00002000:00010000:2.0:1789760678.009915:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20242432, mtime: 0x6aad94a5, atime: 0x0, ctime: 0x6aad94a5, blocks: 0xa007 00010000:00010000:2.0:1789760678.009925:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2c84c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760678.009940:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=657s v=(0 0) 00010000:00010000:2.0:1789760678.009944:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760678.009951:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2c84c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20242432->20246527] (req 20242432->20246527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760678.009965:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2c84c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20242432->20246527] (req 20242432->20246527) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760678.054568:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972261 is committed 00002000:00010000:2.0:1789760678.109753:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20246528 -> 20250624 00010000:00010000:2.0:1789760678.109769:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2c853 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20246528->20250623] (req 20246528->20250623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760678.109782:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2c853 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20246528->20250623] (req 20246528->20250623) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760678.109792:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2c853 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20246528->20250623] (req 20246528->20250623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760678.109819:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760678.109825:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2c853 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20246528->20250623] (req 20246528->20250623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760678.109835:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2c853 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20246528->20250623] (req 20246528->20250623) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760678.109858:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2c853 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20246528->20250623] (req 20246528->20250623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760678.109866:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2c853 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20246528->20250623] (req 20246528->20250623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760678.110166:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febffb9f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512488320:12345-192.168.203.56@tcp:4:dd.0 Request processed in 49060us (49963us total) trans 4294972262 rc 0/0 00000100:00100000:2.0:1789760678.110177:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 4973 00000100:00100000:2.0:1789760678.110187:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 657574417820 time 657625388302 state 0x2 ref 0 00000100:00100000:2.0:1789760678.150456:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760678.150506:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760678.150733:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760678.150763:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512488576 00000100:00100000:2.0:1789760678.150772:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512488576 00000100:00100000:2.0:1789760678.150827:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 372 00000100:00100000:2.0:1789760678.150836:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febe00d5050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:e6cf7267-9b19-462a-a263-14115df15dab+13:2360:x1876699512488576:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:2.0:1789760678.150972:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febe00d5050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:e6cf7267-9b19-462a-a263-14115df15dab+13:2360:x1876699512488576:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 137us (465us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760678.150980:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 372 00000100:00100000:2.0:1789760678.151010:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512488704 00000100:00100000:2.0:1789760678.151013:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512488704 00000100:00100000:2.0:1789760678.151042:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512488832 00000100:00100000:2.0:1789760678.151045:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512488832 00000100:00100000:2.0:1789760678.151063:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 409 00000100:00100000:2.0:1789760678.151069:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febff933480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+8:2360:x1876699512488704:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:2.0:1789760678.151097:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febff933480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+8:2360:x1876699512488704:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 28us (653us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760678.151103:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 409 00000100:00100000:2.0:1789760678.151113:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 410 00000100:00100000:2.0:1789760678.151119:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febff932d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:d2c40150-e97b-40da-952b-cb9f18bf7e96+5:2360:x1876699512488832:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:2.0:1789760678.151141:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febff932d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:d2c40150-e97b-40da-952b-cb9f18bf7e96+5:2360:x1876699512488832:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 22us (408us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760678.151146:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 410 00000100:00080000:2.0:1789760678.152051:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789760678.168758:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760678.224468:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512489088 00000100:00100000:2.0:1789760678.224537:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512489088 00000100:00100000:2.0:1789760678.224592:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 657682391571 time 657733420554 state 0x0 ref 1 00000100:00100000:2.0:1789760678.224607:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 657733680970 time 657733420554 state 0x1 ref 1 00000100:00100000:2.0:1789760678.224672:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 657733680970 time 657788971602 state 0x0 ref 1 00000100:00100000:2.0:1789760678.224685:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febc7b30000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512489088:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760678.224762:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20258816, mtime: 0x6aad94a5, atime: 0x0, ctime: 0x6aad94a5, blocks: 0xa007 00010000:00010000:2.0:1789760678.224771:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2c868 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760678.224786:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=657s v=(0 0) 00010000:00010000:2.0:1789760678.224791:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760678.224797:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2c868 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20258816->20262911] (req 20258816->20262911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760678.224821:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2c868 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20258816->20262911] (req 20258816->20262911) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760678.286066:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000001:00080000:2.0:1789760678.338620:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972266 is committed 00000100:00100000:2.0:1789760678.348639:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760678.409834:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000001:00080000:2.0:1789760678.478670:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972268 is committed 00000100:00100000:2.0:1789760678.482916:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760678.539677:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000001:00080000:2.0:1789760678.584729:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972270 is committed 00000001:00080000:2.0:1789760678.650007:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972271 is committed 00002000:00010000:2.0:1789760678.728268:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20287488 -> 20291584 00010000:00010000:2.0:1789760678.728286:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2c899 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20287488->20291583] (req 20287488->20291583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760678.728298:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2c899 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20287488->20291583] (req 20287488->20291583) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760678.728306:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2c899 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20287488->20291583] (req 20287488->20291583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760678.728313:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760678.728317:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2c899 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20287488->20291583] (req 20287488->20291583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760678.728325:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2c899 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20287488->20291583] (req 20287488->20291583) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760678.728343:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2c899 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20287488->20291583] (req 20287488->20291583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760678.728350:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2c899 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20287488->20291583] (req 20287488->20291583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760678.728448:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febea89b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512489984:12345-192.168.203.56@tcp:4:dd.0 Request processed in 71057us (72459us total) trans 4294972272 rc 0/0 00000100:00100000:2.0:1789760678.728457:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 4983 00000100:00100000:2.0:1789760678.728466:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 658152816209 time 658221663835 state 0x2 ref 0 00002000:00010000:2.0:1789760678.791732:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20291584 -> 20295680 00010000:00010000:2.0:1789760678.791818:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2c8a0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20291584->20295679] (req 20291584->20295679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760678.791832:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2c8a0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20291584->20295679] (req 20291584->20295679) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760678.791842:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2c8a0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20291584->20295679] (req 20291584->20295679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760678.791850:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760678.791855:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2c8a0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20291584->20295679] (req 20291584->20295679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760678.791865:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2c8a0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20291584->20295679] (req 20291584->20295679) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760678.791884:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2c8a0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20291584->20295679] (req 20291584->20295679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760678.791892:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2c8a0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20291584->20295679] (req 20291584->20295679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760678.792083:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febea89b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512490112:12345-192.168.203.56@tcp:4:dd.0 Request processed in 58282us (58766us total) trans 4294972273 rc 0/0 00000100:00100000:2.0:1789760678.792095:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 4984 00000100:00100000:2.0:1789760678.792107:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 658221924251 time 658297828304 state 0x2 ref 0 00000001:00080000:2.0:1789760678.851063:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972274 is committed 00000100:00100000:2.0:1789760678.856257:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512490368 00000100:00100000:2.0:1789760678.856268:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512490368 00000100:00100000:2.0:1789760678.856323:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 658298088720 time 658360699412 state 0x0 ref 1 00000100:00100000:2.0:1789760678.856340:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 658360959828 time 658360699412 state 0x1 ref 1 00000100:00100000:2.0:1789760678.856373:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 658360959828 time 658420672469 state 0x0 ref 1 00000100:00100000:2.0:1789760678.856386:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4059180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512490368:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760678.856440:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20299776, mtime: 0x6aad94a6, atime: 0x0, ctime: 0x6aad94a6, blocks: 0xa007 00010000:00010000:2.0:1789760678.856448:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2c8ae lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760678.856464:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=658s v=(0 0) 00010000:00010000:2.0:1789760678.856468:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760678.856473:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2c8ae lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20299776->20303871] (req 20299776->20303871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760678.856487:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2c8ae lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20299776->20303871] (req 20299776->20303871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760678.909466:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00002000:00010000:2.0:1789760678.972437:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20303872 -> 20307968 00010000:00010000:2.0:1789760678.972455:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2c8b5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20303872->20307967] (req 20303872->20307967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760678.972471:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2c8b5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20303872->20307967] (req 20303872->20307967) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760678.972481:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2c8b5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20303872->20307967] (req 20303872->20307967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760678.972489:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760678.972494:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2c8b5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20303872->20307967] (req 20303872->20307967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760678.972508:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2c8b5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20303872->20307967] (req 20303872->20307967) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760678.972528:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2c8b5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20303872->20307967] (req 20303872->20307967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760678.972535:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2c8b5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20303872->20307967] (req 20303872->20307967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760678.972627:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febfabf5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512490496:12345-192.168.203.56@tcp:4:dd.0 Request processed in 62459us (63165us total) trans 4294972276 rc 0/0 00000100:00100000:2.0:1789760678.972637:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 4987 00000100:00100000:2.0:1789760678.972647:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 658420932885 time 658474451489 state 0x2 ref 0 00000100:00080000:2.0:1789760679.037648:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00010000:2.0:1789760679.094284:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20312064 -> 20316160 00010000:00010000:2.0:1789760679.094305:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2c8c3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20312064->20316159] (req 20312064->20316159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760679.094318:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2c8c3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20312064->20316159] (req 20312064->20316159) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760679.094327:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2c8c3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20312064->20316159] (req 20312064->20316159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760679.094335:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760679.094341:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2c8c3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20312064->20316159] (req 20312064->20316159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760679.094349:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2c8c3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20312064->20316159] (req 20312064->20316159) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760679.094368:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2c8c3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20312064->20316159] (req 20312064->20316159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760679.094376:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2c8c3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20312064->20316159] (req 20312064->20316159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760679.094480:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febea899180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2358:x1876699512490752:12345-192.168.203.56@tcp:4:dd.0 Request processed in 57278us (57605us total) trans 4294972278 rc 0/0 00000100:00100000:2.0:1789760679.094490:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 4989 00000100:00100000:2.0:1789760679.094498:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 658541680532 time 658601477938 state 0x2 ref 0 00000100:00100000:2.0:1789760679.099393:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512490880 00000100:00100000:2.0:1789760679.099404:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512490880 00000100:00100000:2.0:1789760679.099457:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 658541680532 time 658601477938 state 0x0 ref 1 00000100:00100000:2.0:1789760679.099472:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 658601738354 time 658601477938 state 0x1 ref 1 00000100:00100000:2.0:1789760679.099503:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 658601738354 time 658663802461 state 0x0 ref 1 00000100:00100000:2.0:1789760679.099516:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febea89aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512490880:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760679.099564:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20316160, mtime: 0x6aad94a6, atime: 0x0, ctime: 0x6aad94a6, blocks: 0xa007 00010000:00010000:2.0:1789760679.099574:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2c8ca lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760679.099590:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=658s v=(0 0) 00010000:00010000:2.0:1789760679.099594:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760679.099601:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2c8ca lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20316160->20320255] (req 20316160->20320255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760679.099614:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2c8ca lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20316160->20320255] (req 20316160->20320255) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760679.130393:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20316160 -> 20320256 00010000:00010000:2.0:1789760679.130408:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2c8ca lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20316160->20320255] (req 20316160->20320255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760679.130420:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2c8ca lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20316160->20320255] (req 20316160->20320255) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760679.130428:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2c8ca lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20316160->20320255] (req 20316160->20320255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760679.130435:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760679.130439:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2c8ca lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20316160->20320255] (req 20316160->20320255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760679.130447:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2c8ca lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20316160->20320255] (req 20316160->20320255) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760679.130481:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2c8ca lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20316160->20320255] (req 20316160->20320255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760679.130489:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2c8ca lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20316160->20320255] (req 20316160->20320255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760679.130578:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febea89aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512490880:12345-192.168.203.56@tcp:4:dd.0 Request processed in 31059us (31268us total) trans 4294972279 rc 0/0 00000100:00100000:2.0:1789760679.130589:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 4990 00000100:00100000:2.0:1789760679.130598:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 658601738354 time 658663802461 state 0x2 ref 0 00002000:00010000:2.0:1789760679.216900:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20320256 -> 20324352 00010000:00010000:2.0:1789760679.216914:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2c8d1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20320256->20324351] (req 20320256->20324351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760679.216928:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2c8d1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20320256->20324351] (req 20320256->20324351) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760679.216939:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2c8d1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20320256->20324351] (req 20320256->20324351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760679.216947:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760679.216952:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2c8d1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20320256->20324351] (req 20320256->20324351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760679.216966:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2c8d1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20320256->20324351] (req 20320256->20324351) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760679.216985:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2c8d1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20320256->20324351] (req 20320256->20324351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760679.216993:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2c8d1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20320256->20324351] (req 20320256->20324351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760679.217082:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febffb9f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512491008:12345-192.168.203.56@tcp:4:dd.0 Request processed in 79748us (80895us total) trans 4294972280 rc 0/0 00000100:00100000:2.0:1789760679.217091:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 4991 00000100:00100000:2.0:1789760679.217101:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 658664062877 time 658701613724 state 0x2 ref 0 00000100:00100000:2.0:1789760679.225777:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512491136 00000100:00100000:2.0:1789760679.225791:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512491136 00000100:00100000:2.0:1789760679.225864:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 658664062877 time 658701613724 state 0x0 ref 1 00000100:00100000:2.0:1789760679.225882:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 658701874140 time 658701613724 state 0x1 ref 1 00000100:00100000:2.0:1789760679.225917:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 658701874140 time 658790217084 state 0x0 ref 1 00000100:00100000:2.0:1789760679.225930:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febea89b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512491136:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760679.225983:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20324352, mtime: 0x6aad94a6, atime: 0x0, ctime: 0x6aad94a6, blocks: 0xa007 00010000:00010000:2.0:1789760679.225993:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2c8d8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760679.226009:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=658s v=(0 0) 00010000:00010000:2.0:1789760679.226014:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760679.226021:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2c8d8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20324352->20328447] (req 20324352->20328447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760679.226035:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2c8d8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20324352->20328447] (req 20324352->20328447) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760679.285693:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512491264 00000100:00100000:2.0:1789760679.285743:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512491264 00000100:00100000:2.0:1789760679.285815:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 658701874140 time 658790217084 state 0x0 ref 1 00000100:00100000:2.0:1789760679.285830:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 658790477500 time 658790217084 state 0x1 ref 1 00000100:00100000:2.0:1789760679.285863:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 658790477500 time 658850163198 state 0x0 ref 1 00000100:00100000:2.0:1789760679.285876:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febea89b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512491264:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760679.285929:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20328448, mtime: 0x6aad94a6, atime: 0x0, ctime: 0x6aad94a6, blocks: 0xa007 00010000:00010000:2.0:1789760679.285938:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2c8df lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760679.285954:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=658s v=(0 0) 00010000:00010000:2.0:1789760679.285959:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760679.285966:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2c8df lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20328448->20332543] (req 20328448->20332543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760679.285985:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2c8df lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20328448->20332543] (req 20328448->20332543) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760679.344723:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972282 is committed 00000100:00100000:2.0:1789760679.352600:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512491392 00000100:00100000:2.0:1789760679.352610:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512491392 00000100:00100000:2.0:1789760679.352660:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 658790477500 time 658850163198 state 0x0 ref 1 00000100:00100000:2.0:1789760679.352675:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 658850423614 time 658850163198 state 0x1 ref 1 00000100:00100000:2.0:1789760679.352735:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 658850423614 time 658917035054 state 0x0 ref 1 00000100:00100000:2.0:1789760679.352748:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febffb9d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512491392:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760679.352818:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20332544, mtime: 0x6aad94a6, atime: 0x0, ctime: 0x6aad94a6, blocks: 0xa007 00010000:00010000:2.0:1789760679.352827:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2c8e6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760679.352842:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=658s v=(0 0) 00010000:00010000:2.0:1789760679.352847:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760679.352853:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2c8e6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20332544->20336639] (req 20332544->20336639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760679.352866:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2c8e6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20332544->20336639] (req 20332544->20336639) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760679.757113:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972289 is committed 00000001:00080000:2.0:1789760679.827743:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972290 is committed 00000100:00100000:2.0:1789760679.859362:0:3311:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8febe7904a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3311:1876699540211968:0@lo:400:kworker.0 00000100:00100000:2.0:1789760679.859431:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760679.859501:0:3311:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [3->2] 00000100:00100000:2.0:1789760679.859509:0:3311:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8febfabf6300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:6df98d54-eb52-4c5c-8fc9-ee9d74f90f98:3311:1876699540211584:0@lo:400:kworker.0 00000100:00100000:2.0:1789760679.859538:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760679.859562:0:3311:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8febe7906a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3311:1876699540212096:0@lo:400:kworker.0 00000100:00100000:2.0:1789760679.859587:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760679.859612:0:3309:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8feac2fb6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3309:1876699540211712:0@lo:400:kworker.0 00000100:00100000:2.0:1789760679.859634:0:3309:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760679.859651:0:3309:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8febe7907100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3309:1876699540212224:0@lo:400:kworker.0 00000100:00100000:2.0:1789760679.859669:0:3309:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760679.859686:0:3309:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1789760679.859691:0:3309:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8feac2fb4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3309:1876699540211840:0@lo:400:kworker.0 00000100:00100000:2.0:1789760679.859737:0:3309:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760679.859772:0:6705:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540211968 00000100:00100000:2.0:1789760679.859778:0:6705:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540211968 00000100:00100000:2.0:1789760679.859836:0:6705:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540212224 00000100:00100000:2.0:1789760679.859840:0:6705:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540212224 00000100:00100000:2.0:1789760679.859867:0:6705:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 159 00000100:00100000:2.0:1789760679.859875:0:6705:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febe7905c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3311:x1876699540211968:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760679.859925:0:6705:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febe7905c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3311:x1876699540211968:12345-0@lo:400:kworker.0 Request processed in 50us (493us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760679.859936:0:6705:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 159 00000100:00100000:2.0:1789760679.859952:0:6705:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 160 00000100:00100000:2.0:1789760679.859958:0:6705:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febe7905500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3309:x1876699540212224:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760679.859991:0:6705:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febe7905500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3309:x1876699540212224:12345-0@lo:400:kworker.0 Request processed in 33us (320us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760679.859996:0:6705:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 160 00000100:00100000:2.0:1789760679.860014:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540211584 00000100:00100000:2.0:1789760679.860017:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540211584 00000100:00100000:2.0:1789760679.860041:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 373 00000100:00100000:2.0:1789760679.860047:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febe00d4050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:6df98d54-eb52-4c5c-8fc9-ee9d74f90f98+19:3311:x1876699540211584:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760679.860677:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febe00d4050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:6df98d54-eb52-4c5c-8fc9-ee9d74f90f98+19:3311:x1876699540211584:12345-0@lo:400:kworker.0 Request processed in 627us (1135us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760679.860685:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 373 00000100:00100000:2.0:1789760679.860752:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540212096 00000100:00100000:2.0:1789760679.860757:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540212096 00000100:00100000:2.0:1789760679.860784:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540211712 00000100:00100000:2.0:1789760679.860786:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540211712 00000100:00100000:2.0:1789760679.860809:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540211840 00000100:00100000:2.0:1789760679.860812:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540211840 00000100:00100000:2.0:1789760679.860828:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 411 00000100:00100000:2.0:1789760679.860834:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febe7905180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3311:x1876699540212096:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760679.860874:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febe7905180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3311:x1876699540212096:12345-0@lo:400:kworker.0 Request processed in 39us (1286us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760679.860880:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 411 00000100:00100000:2.0:1789760679.860892:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 412 00000100:00100000:2.0:1789760679.860897:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febe7904000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3309:x1876699540211712:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760679.860927:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febe7904000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3309:x1876699540211712:12345-0@lo:400:kworker.0 Request processed in 29us (1292us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760679.860933:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 412 00000100:00100000:2.0:1789760679.860943:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 413 00000100:00100000:2.0:1789760679.860949:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac437dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3309:x1876699540211840:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760679.860973:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac437dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3309:x1876699540211840:12345-0@lo:400:kworker.0 Request processed in 24us (1235us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760679.860978:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 413 00000100:00080000:2.0:1789760679.861004:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:2.0:1789760679.861008:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:2.0:1789760679.861010:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1789760679.861031:0:3310:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8febffb9c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3310:1876699540212352:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789760679.861062:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760679.861100:0:6707:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540212352 00000100:00100000:2.0:1789760679.861104:0:6707:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540212352 00000100:00100000:2.0:1789760679.861127:0:6707:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 172 00000100:00100000:2.0:1789760679.861134:0:6707:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac437c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3310:x1876699540212352:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1789760679.861149:0:6707:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 527630336 pending 1703936 free 3837788160 avail 3812622336 00000100:00100000:2.0:1789760679.861182:0:6707:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac437c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3310:x1876699540212352:12345-0@lo:13:osp-pre-0-0.0 Request processed in 49us (120us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760679.861188:0:6707:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 172 00000100:00100000:2.0:1789760679.861230:0:3310:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8febffb9c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3310:1876699540212352:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789760679.861260:0:3309:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8feac2fb4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3309:1876699540211840:0@lo:400:kworker.0 00000100:00100000:2.0:1789760679.861270:0:3309:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8feac2fb6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3309:1876699540211712:0@lo:400:kworker.0 00000100:00100000:2.0:1789760679.861279:0:3309:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8febe7907100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3309:1876699540212224:0@lo:400:kworker.0 00000100:00100000:2.0:1789760679.863901:0:3311:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8febfabf6300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:6df98d54-eb52-4c5c-8fc9-ee9d74f90f98:3311:1876699540211584:0@lo:400:kworker.0 00000100:00100000:2.0:1789760679.863926:0:3311:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8febe7906a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3311:1876699540212096:0@lo:400:kworker.0 00000100:00100000:2.0:1789760679.863939:0:3311:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8febe7904a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3311:1876699540211968:0@lo:400:kworker.0 00000001:00080000:2.0:1789760679.937120:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972291 is committed 00000001:00080000:2.0:1789760680.028404:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972292 is committed 00000001:00080000:2.0:1789760680.163913:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972294 is committed 00000100:00100000:2.0:1789760680.242377:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512493056 00000100:00100000:2.0:1789760680.242388:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512493056 00000100:00100000:2.0:1789760680.242442:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 659656472651 time 659732372452 state 0x0 ref 1 00000100:00100000:2.0:1789760680.242457:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 659732632868 time 659732372452 state 0x1 ref 1 00000100:00100000:2.0:1789760680.242558:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 659732632868 time 659806856438 state 0x0 ref 1 00000100:00100000:2.0:1789760680.242572:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4058380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512493056:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760680.242617:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20385792, mtime: 0x6aad94a7, atime: 0x0, ctime: 0x6aad94a7, blocks: 0xa007 00010000:00010000:2.0:1789760680.242626:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2c941 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760680.242642:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=659s v=(0 0) 00010000:00010000:2.0:1789760680.242647:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760680.242654:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2c941 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20385792->20389887] (req 20385792->20389887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760680.242668:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2c941 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20385792->20389887] (req 20385792->20389887) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760680.314968:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac405bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512493184:12345-192.168.203.56@tcp:4:dd.0 Request processed in 34638us (35063us total) trans 4294972297 rc 0/0 00000100:00100000:2.0:1789760680.314979:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5008 00000100:00100000:2.0:1789760680.314989:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 659807116854 time 659844613807 state 0x2 ref 0 00000100:00100000:2.0:1789760680.317769:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00002000:00010000:2.0:1789760680.347917:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20393984 -> 20398080 00010000:00010000:2.0:1789760680.347933:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2c94f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20393984->20398079] (req 20393984->20398079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760680.347947:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2c94f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20393984->20398079] (req 20393984->20398079) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760680.347957:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2c94f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20393984->20398079] (req 20393984->20398079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760680.347964:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760680.347969:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2c94f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20393984->20398079] (req 20393984->20398079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760680.347979:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2c94f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20393984->20398079] (req 20393984->20398079) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760680.348001:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2c94f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20393984->20398079] (req 20393984->20398079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760680.348015:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2c94f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20393984->20398079] (req 20393984->20398079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760680.348111:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac437e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512493312:12345-192.168.203.56@tcp:4:dd.0 Request processed in 29997us (30344us total) trans 4294972298 rc 0/0 00000100:00100000:2.0:1789760680.348121:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5009 00000100:00100000:2.0:1789760680.348131:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 659844874223 time 659882395824 state 0x2 ref 0 00000100:00100000:2.0:1789760680.372833:0:6707:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540212480 00000100:00100000:2.0:1789760680.372842:0:6707:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540212480 00000100:00100000:2.0:1789760680.372894:0:6707:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 173 00000100:00100000:2.0:1789760680.372904:0:6707:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4059880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3311:x1876699540212480:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1789760680.372936:0:6707:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3859808256 avail 3857711104 00000100:00100000:2.0:1789760680.372993:0:6707:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4059880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3311:x1876699540212480:12345-0@lo:13:osp-pre-1-0.0 Request processed in 90us (547us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760680.373001:0:6707:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 173 00000100:00100000:2.0:1789760680.416593:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512493568 00000100:00100000:2.0:1789760680.416604:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512493568 00000100:00100000:2.0:1789760680.416657:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 659882656240 time 659917595763 state 0x0 ref 1 00000100:00100000:2.0:1789760680.416671:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 659917856179 time 659917595763 state 0x1 ref 1 00000100:00100000:2.0:1789760680.484675:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512493696 00000100:00100000:2.0:1789760680.484685:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512493696 00000100:00100000:2.0:1789760680.484777:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 659917856179 time 659981158248 state 0x0 ref 1 00000100:00100000:2.0:1789760680.484808:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 659981418664 time 659981158248 state 0x1 ref 1 00000100:00100000:2.0:1789760680.484840:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 659981418664 time 660049140413 state 0x0 ref 1 00000100:00100000:2.0:1789760680.484853:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febea771880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512493696:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760680.484899:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20406272, mtime: 0x6aad94a7, atime: 0x0, ctime: 0x6aad94a7, blocks: 0xa007 00010000:00010000:2.0:1789760680.484908:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2c964 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760680.484925:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=660s v=(0 0) 00010000:00010000:2.0:1789760680.484929:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760680.484935:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2c964 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20406272->20410367] (req 20406272->20410367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760680.484950:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2c964 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20406272->20410367] (req 20406272->20410367) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760680.509536:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972301 is committed 00002000:00010000:2.0:1789760680.509606:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20406272 -> 20410368 00010000:00010000:2.0:1789760680.509622:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2c964 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20406272->20410367] (req 20406272->20410367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760680.509634:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2c964 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20406272->20410367] (req 20406272->20410367) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760680.509642:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2c964 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20406272->20410367] (req 20406272->20410367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760680.509650:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760680.509656:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2c964 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20406272->20410367] (req 20406272->20410367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760680.509665:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2c964 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20406272->20410367] (req 20406272->20410367) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760680.509683:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2c964 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20406272->20410367] (req 20406272->20410367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760680.509691:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2c964 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20406272->20410367] (req 20406272->20410367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760680.509806:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febea771880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512493696:12345-192.168.203.56@tcp:4:dd.0 Request processed in 24937us (25211us total) trans 4294972301 rc 0/0 00000100:00100000:2.0:1789760680.509817:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5012 00000100:00100000:2.0:1789760680.509828:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 659981418664 time 660049140413 state 0x2 ref 0 00000001:00080000:2.0:1789760680.586864:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972302 is committed 00000100:00100000:2.0:1789760680.594813:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760680.683150:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000001:00080000:2.0:1789760680.755894:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972304 is committed 00000100:00100000:2.0:1789760680.763486:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512494208 00000100:00100000:2.0:1789760680.763496:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512494208 00000100:00100000:2.0:1789760680.763547:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 660159566840 time 660247706635 state 0x0 ref 1 00000100:00100000:2.0:1789760680.763566:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 660247967051 time 660247706635 state 0x1 ref 1 00000100:00100000:2.0:1789760680.763598:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 660247967051 time 660327897940 state 0x0 ref 1 00000100:00100000:2.0:1789760680.763611:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2876a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512494208:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760680.763659:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20422656, mtime: 0x6aad94a8, atime: 0x0, ctime: 0x6aad94a8, blocks: 0xa007 00010000:00010000:2.0:1789760680.763668:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2c980 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760680.763683:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=660s v=(0 0) 00010000:00010000:2.0:1789760680.763688:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760680.763730:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2c980 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20422656->20426751] (req 20422656->20426751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760680.763743:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2c980 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20422656->20426751] (req 20422656->20426751) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760680.856827:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972305 is committed 00002000:00010000:2.0:1789760680.856902:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20422656 -> 20426752 00010000:00010000:2.0:1789760680.856918:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2c980 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20422656->20426751] (req 20422656->20426751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760680.856931:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2c980 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20422656->20426751] (req 20422656->20426751) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760680.856940:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2c980 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20422656->20426751] (req 20422656->20426751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760680.856949:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760680.856954:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2c980 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20422656->20426751] (req 20422656->20426751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760680.856962:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2c980 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20422656->20426751] (req 20422656->20426751) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760680.856981:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2c980 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20422656->20426751] (req 20422656->20426751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760680.856989:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2c980 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20422656->20426751] (req 20422656->20426751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760680.857083:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2876a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512494208:12345-192.168.203.56@tcp:4:dd.0 Request processed in 93469us (93708us total) trans 4294972305 rc 0/0 00000100:00100000:2.0:1789760680.857092:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5016 00000100:00100000:2.0:1789760680.857102:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 660247967051 time 660327897940 state 0x2 ref 0 00000100:00100000:2.0:1789760680.861035:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760680.861752:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 660328158356 time 660426049878 state 0x0 ref 1 00000100:00100000:2.0:1789760680.861766:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac437df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512494336:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760680.861823:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20426752, mtime: 0x6aad94a8, atime: 0x0, ctime: 0x6aad94a8, blocks: 0xa007 00010000:00010000:2.0:1789760680.861831:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2c987 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760680.861847:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=660s v=(0 0) 00010000:00010000:2.0:1789760680.861852:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760680.861858:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2c987 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20426752->20430847] (req 20426752->20430847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760680.861871:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2c987 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20426752->20430847] (req 20426752->20430847) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760680.943299:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20426752 -> 20430848 00010000:00010000:2.0:1789760680.943314:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2c987 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20426752->20430847] (req 20426752->20430847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760680.943327:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2c987 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20426752->20430847] (req 20426752->20430847) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760680.943335:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2c987 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20426752->20430847] (req 20426752->20430847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760680.943342:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760680.943347:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2c987 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20426752->20430847] (req 20426752->20430847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760680.943356:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2c987 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20426752->20430847] (req 20426752->20430847) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760680.943373:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2c987 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20426752->20430847] (req 20426752->20430847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760680.943381:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2c987 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20426752->20430847] (req 20426752->20430847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760680.943644:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac437df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512494336:12345-192.168.203.56@tcp:4:dd.0 Request processed in 81874us (82612us total) trans 4294972306 rc 0/0 00000100:00100000:2.0:1789760680.943655:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5017 00000100:00100000:2.0:1789760680.943665:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 660328158356 time 660426049878 state 0x2 ref 0 00000100:00100000:2.0:1789760680.950481:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760681.022945:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00080000:2.0:1789760681.024066:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1789760681.087540:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760681.190905:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760681.332440:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760681.555820:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000001:00080000:2.0:1789760681.603865:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972313 is committed 00000100:00100000:2.0:1789760681.606282:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512495360 00000100:00100000:2.0:1789760681.606293:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512495360 00000100:00100000:2.0:1789760681.606345:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 661000751796 time 661120336492 state 0x0 ref 1 00000100:00100000:2.0:1789760681.606360:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 661120596908 time 661120336492 state 0x1 ref 1 00000100:00100000:2.0:1789760681.606390:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 661120596908 time 661170690482 state 0x0 ref 1 00000100:00100000:2.0:1789760681.606402:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febd1362a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512495360:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760681.606449:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20459520, mtime: 0x6aad94a9, atime: 0x0, ctime: 0x6aad94a9, blocks: 0xa007 00010000:00010000:2.0:1789760681.606468:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2c9bf lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760681.606482:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=661s v=(0 0) 00010000:00010000:2.0:1789760681.606486:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760681.606492:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2c9bf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20459520->20463615] (req 20459520->20463615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760681.606505:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2c9bf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20459520->20463615] (req 20459520->20463615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760681.684426:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20459520 -> 20463616 00010000:00010000:2.0:1789760681.684440:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2c9bf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20459520->20463615] (req 20459520->20463615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760681.684464:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2c9bf lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20459520->20463615] (req 20459520->20463615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760681.684475:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2c9bf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20459520->20463615] (req 20459520->20463615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760681.684483:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760681.684488:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2c9bf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20459520->20463615] (req 20459520->20463615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760681.684496:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2c9bf lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20459520->20463615] (req 20459520->20463615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760681.684515:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2c9bf lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20459520->20463615] (req 20459520->20463615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760681.684522:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2c9bf lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20459520->20463615] (req 20459520->20463615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760681.684598:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febd1362a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512495360:12345-192.168.203.56@tcp:4:dd.0 Request processed in 78194us (78410us total) trans 4294972314 rc 0/0 00000100:00100000:2.0:1789760681.684609:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5025 00000100:00100000:2.0:1789760681.684618:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 661120596908 time 661170690482 state 0x2 ref 0 00000001:00080000:2.0:1789760681.786165:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972316 is committed 00000100:00100000:2.0:1789760681.793987:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512495744 00000100:00100000:2.0:1789760681.794008:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512495744 00000100:00100000:2.0:1789760681.794062:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 661252405716 time 661303374326 state 0x0 ref 1 00000100:00100000:2.0:1789760681.794078:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 661303634742 time 661303374326 state 0x1 ref 1 00000100:00100000:2.0:1789760681.794103:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 661303634742 time 661358403073 state 0x0 ref 1 00000100:00100000:2.0:1789760681.794114:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4074a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512495744:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760681.794161:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20471808, mtime: 0x6aad94a9, atime: 0x0, ctime: 0x6aad94a9, blocks: 0xa007 00010000:00010000:2.0:1789760681.794171:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2c9d4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760681.794192:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=661s v=(0 0) 00010000:00010000:2.0:1789760681.794197:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760681.794203:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2c9d4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20471808->20475903] (req 20471808->20475903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760681.794217:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2c9d4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20471808->20475903] (req 20471808->20475903) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760681.893347:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20475904 -> 20480000 00010000:00010000:2.0:1789760681.893362:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2c9db lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20475904->20479999] (req 20475904->20479999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760681.893376:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2c9db lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20475904->20479999] (req 20475904->20479999) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760681.893385:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2c9db lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20475904->20479999] (req 20475904->20479999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760681.893392:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760681.893396:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2c9db lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20475904->20479999] (req 20475904->20479999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760681.893405:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2c9db lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20475904->20479999] (req 20475904->20479999) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760681.893424:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2c9db lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20475904->20479999] (req 20475904->20479999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760681.893432:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2c9db lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20475904->20479999] (req 20475904->20479999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760681.893536:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febfbeae300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512495872:12345-192.168.203.56@tcp:4:dd.0 Request processed in 32718us (32989us total) trans 4294972318 rc 0/0 00000100:00100000:2.0:1789760681.893546:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5029 00000100:00100000:2.0:1789760681.893556:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 661358663489 time 661425100984 state 0x2 ref 0 00002000:00010000:2.0:1789760681.979859:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20480000 -> 20484096 00010000:00010000:2.0:1789760681.979875:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2c9e2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20480000->20484095] (req 20480000->20484095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760681.979888:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2c9e2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20480000->20484095] (req 20480000->20484095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760681.979897:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2c9e2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20480000->20484095] (req 20480000->20484095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760681.979905:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760681.979910:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2c9e2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20480000->20484095] (req 20480000->20484095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760681.979919:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2c9e2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20480000->20484095] (req 20480000->20484095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760681.979943:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2c9e2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20480000->20484095] (req 20480000->20484095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760681.979951:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2c9e2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20480000->20484095] (req 20480000->20484095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760681.980044:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febea651500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512496000:12345-192.168.203.56@tcp:4:dd.0 Request processed in 73419us (73946us total) trans 4294972319 rc 0/0 00000100:00100000:2.0:1789760681.980054:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5030 00000100:00100000:2.0:1789760681.980064:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 661425361400 time 661470908640 state 0x2 ref 0 00000001:00080000:2.0:1789760682.110440:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972320 is committed 00000100:00100000:2.0:1789760682.114420:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512496256 00000100:00100000:2.0:1789760682.114429:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512496256 00000100:00100000:2.0:1789760682.114491:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 661471169056 time 661547497335 state 0x0 ref 1 00000100:00100000:2.0:1789760682.114507:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 661547757751 time 661547497335 state 0x1 ref 1 00000100:00100000:2.0:1789760682.114541:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 661547757751 time 661678840707 state 0x0 ref 1 00000100:00100000:2.0:1789760682.114553:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febea650380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512496256:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760682.114601:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20488192, mtime: 0x6aad94a9, atime: 0x0, ctime: 0x6aad94a9, blocks: 0xa007 00010000:00010000:2.0:1789760682.114610:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2c9f0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760682.114631:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=661s v=(0 0) 00010000:00010000:2.0:1789760682.114636:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760682.114641:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2c9f0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20488192->20492287] (req 20488192->20492287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760682.114655:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2c9f0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20488192->20492287] (req 20488192->20492287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760682.180130:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512496384 00000100:00100000:2.0:1789760682.180140:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512496384 00000100:00100000:2.0:1789760682.180194:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 661547757751 time 661678840707 state 0x0 ref 1 00000100:00100000:2.0:1789760682.180211:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 661679101123 time 661678840707 state 0x1 ref 1 00000100:00100000:2.0:1789760682.180244:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 661679101123 time 661744542990 state 0x0 ref 1 00000100:00100000:2.0:1789760682.180256:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac3302680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512496384:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760682.180308:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20492288, mtime: 0x6aad94a9, atime: 0x0, ctime: 0x6aad94a9, blocks: 0xa007 00010000:00010000:2.0:1789760682.180317:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2c9f7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760682.180334:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=661s v=(0 0) 00010000:00010000:2.0:1789760682.180339:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760682.180345:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2c9f7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20492288->20496383] (req 20492288->20496383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760682.180364:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2c9f7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20492288->20496383] (req 20492288->20496383) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760682.579977:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512497024 00000100:00100000:2.0:1789760682.579986:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512497024 00000100:00100000:2.0:1789760682.580054:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 662012161717 time 662069679677 state 0x0 ref 1 00000100:00100000:2.0:1789760682.580069:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 662069940093 time 662069679677 state 0x1 ref 1 00000100:00100000:2.0:1789760682.580098:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 662069940093 time 662144398101 state 0x0 ref 1 00000100:00100000:2.0:1789760682.580111:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febc7b33480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512497024:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760682.580159:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20512768, mtime: 0x6aad94aa, atime: 0x0, ctime: 0x6aad94aa, blocks: 0xa007 00010000:00010000:2.0:1789760682.580178:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2ca1a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760682.580194:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=662s v=(0 0) 00010000:00010000:2.0:1789760682.580198:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760682.580204:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2ca1a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20512768->20516863] (req 20512768->20516863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760682.580218:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2ca1a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20512768->20516863] (req 20512768->20516863) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760682.799560:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512497408 00000100:00100000:2.0:1789760682.799570:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512497408 00000100:00100000:2.0:1789760682.799627:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 662232382781 time 662304297707 state 0x0 ref 1 00000100:00100000:2.0:1789760682.799643:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 662304558123 time 662304297707 state 0x1 ref 1 00000100:00100000:2.0:1789760682.799677:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 662304558123 time 662363976850 state 0x0 ref 1 00000100:00100000:2.0:1789760682.799688:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febc7b30000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512497408:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760682.799786:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20525056, mtime: 0x6aad94aa, atime: 0x0, ctime: 0x6aad94aa, blocks: 0xa007 00010000:00010000:2.0:1789760682.799795:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2ca2f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760682.799810:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=662s v=(0 0) 00010000:00010000:2.0:1789760682.799815:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760682.799821:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2ca2f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20525056->20529151] (req 20525056->20529151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760682.799832:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2ca2f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20525056->20529151] (req 20525056->20529151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760682.863979:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00002000:00010000:2.0:1789760682.940817:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20533248 -> 20537344 00010000:00010000:2.0:1789760682.940833:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2ca3d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20533248->20537343] (req 20533248->20537343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760682.940847:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2ca3d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20533248->20537343] (req 20533248->20537343) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760682.940862:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2ca3d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20533248->20537343] (req 20533248->20537343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760682.940870:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760682.940875:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2ca3d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20533248->20537343] (req 20533248->20537343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760682.940883:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2ca3d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20533248->20537343] (req 20533248->20537343) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760682.940902:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2ca3d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20533248->20537343] (req 20533248->20537343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760682.940910:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2ca3d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20533248->20537343] (req 20533248->20537343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760682.941000:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac3302680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512497664:12345-192.168.203.56@tcp:4:dd.0 Request processed in 30265us (30537us total) trans 4294972332 rc 0/0 00000100:00100000:2.0:1789760682.941010:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5043 00000100:00100000:2.0:1789760682.941020:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 662429536235 time 662475018376 state 0x2 ref 0 00000001:00080000:2.0:1789760683.011552:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972334 is committed 00000100:00100000:2.0:1789760683.015589:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760683.015968:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 662508885995 time 662543830190 state 0x0 ref 1 00000100:00100000:2.0:1789760683.015987:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 662544090606 time 662543830190 state 0x1 ref 1 00000100:00100000:2.0:1789760683.016017:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 662544090606 time 662580316585 state 0x0 ref 1 00000100:00100000:2.0:1789760683.016029:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febe7907100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512498048:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760683.016078:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20545536, mtime: 0x6aad94aa, atime: 0x0, ctime: 0x6aad94aa, blocks: 0xa007 00010000:00010000:2.0:1789760683.016088:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2ca52 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760683.016105:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=662s v=(0 0) 00010000:00010000:2.0:1789760683.016111:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760683.016117:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2ca52 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20545536->20549631] (req 20545536->20549631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760683.016131:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2ca52 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20545536->20549631] (req 20545536->20549631) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760683.062980:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000001:00080000:2.0:1789760683.098983:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972336 is committed 00000100:00100000:2.0:1789760683.102461:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760683.144802:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00002000:00010000:2.0:1789760683.191010:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20557824 -> 20561920 00010000:00010000:2.0:1789760683.191022:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2ca67 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20557824->20561919] (req 20557824->20561919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760683.191033:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2ca67 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20557824->20561919] (req 20557824->20561919) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760683.191039:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2ca67 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20557824->20561919] (req 20557824->20561919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760683.191044:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760683.191048:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2ca67 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20557824->20561919] (req 20557824->20561919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760683.191053:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2ca67 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20557824->20561919] (req 20557824->20561919) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760683.191067:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2ca67 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20557824->20561919] (req 20557824->20561919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760683.191072:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2ca67 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20557824->20561919] (req 20557824->20561919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760683.191144:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2fb6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512498432:12345-192.168.203.56@tcp:4:dd.0 Request processed in 42210us (46349us total) trans 4294972338 rc 0/0 00000100:00100000:2.0:1789760683.191154:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5049 00000100:00100000:2.0:1789760683.191163:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 662667255562 time 662713218456 state 0x2 ref 0 00000001:00080000:2.0:1789760683.231980:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972339 is committed 00000100:00100000:2.0:1789760683.271100:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760683.271414:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760683.271506:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00080000:2.0:1789760683.274689:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00002000:00010000:2.0:1789760683.284754:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20566016 -> 20570112 00010000:00010000:2.0:1789760683.284784:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2ca75 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20566016->20570111] (req 20566016->20570111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760683.284801:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2ca75 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20566016->20570111] (req 20566016->20570111) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760683.284809:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2ca75 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20566016->20570111] (req 20566016->20570111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760683.284816:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760683.284821:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2ca75 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20566016->20570111] (req 20566016->20570111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760683.284830:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2ca75 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20566016->20570111] (req 20566016->20570111) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760683.284849:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2ca75 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20566016->20570111] (req 20566016->20570111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760683.284856:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2ca75 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20566016->20570111] (req 20566016->20570111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760683.284957:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febc7b30000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2359:x1876699512498688:12345-192.168.203.56@tcp:4:dd.0 Request processed in 46311us (46890us total) trans 4294972340 rc 0/0 00000100:00100000:2.0:1789760683.284969:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5051 00000100:00100000:2.0:1789760683.284979:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 662759895488 time 662802930283 state 0x2 ref 0 00000100:00100000:2.0:1789760683.294163:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00002000:00010000:2.0:1789760683.325685:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20570112 -> 20574208 00010000:00010000:2.0:1789760683.325732:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2ca7c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20570112->20574207] (req 20570112->20574207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760683.325743:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2ca7c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20570112->20574207] (req 20570112->20574207) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760683.325752:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2ca7c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20570112->20574207] (req 20570112->20574207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760683.325759:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760683.325764:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2ca7c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20570112->20574207] (req 20570112->20574207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760683.325785:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2ca7c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20570112->20574207] (req 20570112->20574207) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760683.325805:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2ca7c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20570112->20574207] (req 20570112->20574207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760683.325813:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2ca7c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20570112->20574207] (req 20570112->20574207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760683.325903:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febfb377100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512499200:12345-192.168.203.56@tcp:4:dd.0 Request processed in 31230us (31744us total) trans 4294972341 rc 0/0 00000100:00100000:2.0:1789760683.325913:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5052 00000100:00100000:2.0:1789760683.325922:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 662803190699 time 662858958225 state 0x2 ref 0 00000001:00080000:2.0:1789760683.362319:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972342 is committed 00000100:00100000:2.0:1789760683.366567:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512499456 00000100:00100000:2.0:1789760683.366576:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512499456 00000100:00100000:2.0:1789760683.366637:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 662859218641 time 662894756136 state 0x0 ref 1 00000100:00100000:2.0:1789760683.366652:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 662895016552 time 662894756136 state 0x1 ref 1 00000100:00100000:2.0:1789760683.366682:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 662895016552 time 662930982168 state 0x0 ref 1 00000100:00100000:2.0:1789760683.366741:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac28a3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512499456:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760683.366801:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20578304, mtime: 0x6aad94aa, atime: 0x0, ctime: 0x6aad94aa, blocks: 0xa007 00010000:00010000:2.0:1789760683.366810:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2ca8a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760683.366824:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=662s v=(0 0) 00010000:00010000:2.0:1789760683.366828:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760683.366835:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2ca8a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20578304->20582399] (req 20578304->20582399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760683.366848:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2ca8a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20578304->20582399] (req 20578304->20582399) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760683.387015:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972343 is committed 00000100:00100000:2.0:1789760683.391388:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00002000:00010000:2.0:1789760683.422952:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20582400 -> 20586496 00010000:00010000:2.0:1789760683.422967:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2ca91 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20582400->20586495] (req 20582400->20586495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760683.422981:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2ca91 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20582400->20586495] (req 20582400->20586495) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760683.422991:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2ca91 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20582400->20586495] (req 20582400->20586495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760683.422998:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760683.423004:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2ca91 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20582400->20586495] (req 20582400->20586495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760683.423013:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2ca91 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20582400->20586495] (req 20582400->20586495) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760683.423032:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2ca91 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20582400->20586495] (req 20582400->20586495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760683.423040:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2ca91 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20582400->20586495] (req 20582400->20586495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760683.423131:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febfb374380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512499584:12345-192.168.203.56@tcp:4:dd.0 Request processed in 31461us (31747us total) trans 4294972344 rc 0/0 00000100:00100000:2.0:1789760683.423142:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5055 00000100:00100000:2.0:1789760683.423152:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 662931242584 time 662955954758 state 0x2 ref 0 00000100:00100000:2.0:1789760683.426141:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00002000:00010000:2.0:1789760683.456867:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20586496 -> 20590592 00010000:00010000:2.0:1789760683.456883:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2ca98 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20586496->20590591] (req 20586496->20590591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760683.456897:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2ca98 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20586496->20590591] (req 20586496->20590591) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760683.456906:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2ca98 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20586496->20590591] (req 20586496->20590591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760683.456914:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760683.456919:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2ca98 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20586496->20590591] (req 20586496->20590591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760683.456928:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2ca98 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20586496->20590591] (req 20586496->20590591) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760683.456949:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2ca98 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20586496->20590591] (req 20586496->20590591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760683.456957:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2ca98 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20586496->20590591] (req 20586496->20590591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760683.457092:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febfb375500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512499712:12345-192.168.203.56@tcp:4:dd.0 Request processed in 29420us (30956us total) trans 4294972345 rc 0/0 00000100:00100000:2.0:1789760683.457104:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5056 00000100:00100000:2.0:1789760683.457113:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 662956215174 time 662991954698 state 0x2 ref 0 00000100:00100000:2.0:1789760683.513744:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760683.561639:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512500096 00000100:00100000:2.0:1789760683.561648:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512500096 00000100:00100000:2.0:1789760683.561754:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 663026162247 time 663078334224 state 0x0 ref 1 00000100:00100000:2.0:1789760683.561780:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 663078594640 time 663078334224 state 0x1 ref 1 00000100:00100000:2.0:1789760683.561811:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 663078594640 time 663126111027 state 0x0 ref 1 00000100:00100000:2.0:1789760683.561823:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2877100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512500096:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760683.561868:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20598784, mtime: 0x6aad94ab, atime: 0x0, ctime: 0x6aad94ab, blocks: 0xa007 00010000:00010000:2.0:1789760683.561877:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2caad lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760683.561892:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=663s v=(0 0) 00010000:00010000:2.0:1789760683.561896:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760683.561903:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2caad lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20598784->20602879] (req 20598784->20602879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760683.561916:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2caad lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20598784->20602879] (req 20598784->20602879) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760683.656473:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512500480 00000100:00100000:2.0:1789760683.656485:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512500480 00000100:00100000:2.0:1789760683.656538:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 663160741469 time 663186733744 state 0x0 ref 1 00000100:00100000:2.0:1789760683.656554:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 663186994160 time 663186733744 state 0x1 ref 1 00000100:00100000:2.0:1789760683.656582:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 663186994160 time 663220881694 state 0x0 ref 1 00000100:00100000:2.0:1789760683.656594:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2877b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512500480:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760683.656641:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20611072, mtime: 0x6aad94ab, atime: 0x0, ctime: 0x6aad94ab, blocks: 0xa007 00010000:00010000:2.0:1789760683.656649:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2cac2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760683.656666:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=663s v=(0 0) 00010000:00010000:2.0:1789760683.656670:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760683.656677:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2cac2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20611072->20615167] (req 20611072->20615167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760683.656691:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2cac2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20611072->20615167] (req 20611072->20615167) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760683.688482:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972351 is committed 00000001:00080000:2.0:1789760683.736116:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972352 is committed 00000100:00100000:2.0:1789760683.742539:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00002000:00010000:2.0:1789760683.745278:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20619264, mtime: 0x6aad94ab, atime: 0x0, ctime: 0x6aad94ab, blocks: 0xa007 00010000:00010000:2.0:1789760683.745292:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2cad0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760683.745314:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=663s v=(0 0) 00010000:00010000:2.0:1789760683.745319:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760683.745325:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2cad0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20619264->20623359] (req 20619264->20623359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760683.745339:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2cad0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20619264->20623359] (req 20619264->20623359) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760683.789832:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20619264 -> 20623360 00010000:00010000:2.0:1789760683.789851:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2cad0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20619264->20623359] (req 20619264->20623359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760683.789933:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2cad0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20619264->20623359] (req 20619264->20623359) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760683.789941:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2cad0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20619264->20623359] (req 20619264->20623359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760683.789947:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760683.789951:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2cad0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20619264->20623359] (req 20619264->20623359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760683.789961:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2cad0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20619264->20623359] (req 20619264->20623359) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760683.789979:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2cad0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20619264->20623359] (req 20619264->20623359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760683.789986:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2cad0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20619264->20623359] (req 20619264->20623359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760683.790076:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac3353800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512500736:12345-192.168.203.56@tcp:4:dd.0 Request processed in 46863us (47541us total) trans 4294972353 rc 0/0 00000100:00100000:2.0:1789760683.790084:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5064 00000100:00100000:2.0:1789760683.790094:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 663255935085 time 663307497322 state 0x2 ref 0 00000100:00100000:2.0:1789760683.793233:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760683.884471:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512501120 00000100:00100000:2.0:1789760683.884481:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512501120 00000100:00100000:2.0:1789760683.884533:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 663357982345 time 663401404202 state 0x0 ref 1 00000100:00100000:2.0:1789760683.884548:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 663401664618 time 663401404202 state 0x1 ref 1 00000100:00100000:2.0:1789760683.884576:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 663401664618 time 663448876287 state 0x0 ref 1 00000100:00100000:2.0:1789760683.884587:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac3300e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512501120:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760683.884635:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20631552, mtime: 0x6aad94ab, atime: 0x0, ctime: 0x6aad94ab, blocks: 0xa007 00010000:00010000:2.0:1789760683.884644:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2cae5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760683.884660:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=663s v=(0 0) 00010000:00010000:2.0:1789760683.884664:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760683.884670:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2cae5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20631552->20635647] (req 20631552->20635647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760683.884684:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2cae5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20631552->20635647] (req 20631552->20635647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760683.940052:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972356 is committed 00000100:00100000:2.0:1789760683.944641:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000001:00080000:2.0:1789760684.000667:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972357 is committed 00000100:00100000:2.0:1789760684.005244:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760684.111296:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 663618624986 time 663675593429 state 0x0 ref 1 00000100:00100000:2.0:1789760684.111313:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac3303100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512501632:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760684.111365:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20647936, mtime: 0x6aad94ab, atime: 0x0, ctime: 0x6aad94ab, blocks: 0xa007 00010000:00010000:2.0:1789760684.111374:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2cb01 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760684.111389:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=663s v=(0 0) 00010000:00010000:2.0:1789760684.111393:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760684.111399:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2cb01 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20647936->20652031] (req 20647936->20652031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760684.111412:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2cb01 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20647936->20652031] (req 20647936->20652031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760684.174322:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000001:00080000:2.0:1789760684.259348:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972361 is committed 00000100:00100000:2.0:1789760684.262473:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00002000:00010000:2.0:1789760684.322601:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20656128 -> 20660224 00010000:00010000:2.0:1789760684.322619:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2cb0f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20656128->20660223] (req 20656128->20660223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760684.322632:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2cb0f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20656128->20660223] (req 20656128->20660223) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760684.322641:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2cb0f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20656128->20660223] (req 20656128->20660223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760684.322648:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760684.322652:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2cb0f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20656128->20660223] (req 20656128->20660223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760684.322662:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2cb0f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20656128->20660223] (req 20656128->20660223) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760684.322681:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2cb0f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20656128->20660223] (req 20656128->20660223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760684.322689:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2cb0f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20656128->20660223] (req 20656128->20660223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760684.322825:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4362d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512501888:12345-192.168.203.56@tcp:4:dd.0 Request processed in 60007us (60356us total) trans 4294972362 rc 0/0 00000100:00100000:2.0:1789760684.322835:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5073 00000100:00100000:2.0:1789760684.322844:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 663739084119 time 663827103002 state 0x2 ref 0 00000100:00100000:2.0:1789760684.326985:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00002000:00010000:2.0:1789760684.390083:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20660224 -> 20664320 00010000:00010000:2.0:1789760684.390097:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2cb16 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20660224->20664319] (req 20660224->20664319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760684.390110:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2cb16 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20660224->20664319] (req 20660224->20664319) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760684.390120:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2cb16 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20660224->20664319] (req 20660224->20664319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760684.390128:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760684.390134:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2cb16 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20660224->20664319] (req 20660224->20664319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760684.390142:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2cb16 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20660224->20664319] (req 20660224->20664319) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760684.390171:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2cb16 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20660224->20664319] (req 20660224->20664319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760684.390180:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2cb16 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20660224->20664319] (req 20660224->20664319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760684.390273:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4361180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512502016:12345-192.168.203.56@tcp:4:dd.0 Request processed in 63085us (63292us total) trans 4294972363 rc 0/0 00000100:00100000:2.0:1789760684.390286:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5074 00000100:00100000:2.0:1789760684.390295:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 663827363418 time 663891473123 state 0x2 ref 0 00002000:00010000:2.0:1789760684.464831:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20664320 -> 20668416 00010000:00010000:2.0:1789760684.464846:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2cb1d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20664320->20668415] (req 20664320->20668415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760684.464860:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2cb1d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20664320->20668415] (req 20664320->20668415) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760684.464870:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2cb1d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20664320->20668415] (req 20664320->20668415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760684.464877:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760684.464882:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2cb1d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20664320->20668415] (req 20664320->20668415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760684.464891:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2cb1d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20664320->20668415] (req 20664320->20668415) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760684.464912:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2cb1d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20664320->20668415] (req 20664320->20668415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760684.464919:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2cb1d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20664320->20668415] (req 20664320->20668415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760684.465050:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2874700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512502144:12345-192.168.203.56@tcp:4:dd.0 Request processed in 69954us (70819us total) trans 4294972364 rc 0/0 00000100:00100000:2.0:1789760684.465060:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5075 00000100:00100000:2.0:1789760684.465071:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 663891733539 time 663959380447 state 0x2 ref 0 00000100:00100000:2.0:1789760684.472273:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512502272 00000100:00100000:2.0:1789760684.472283:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512502272 00000100:00100000:2.0:1789760684.472335:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 663891733539 time 663959380447 state 0x0 ref 1 00000100:00100000:2.0:1789760684.472349:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 663959640863 time 663959380447 state 0x1 ref 1 00000100:00100000:2.0:1789760684.472377:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 663959640863 time 664036677078 state 0x0 ref 1 00000100:00100000:2.0:1789760684.472389:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac3301c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512502272:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760684.472597:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20668416, mtime: 0x6aad94ab, atime: 0x0, ctime: 0x6aad94ab, blocks: 0xa007 00010000:00010000:2.0:1789760684.472607:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2cb24 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760684.472622:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=664s v=(0 0) 00010000:00010000:2.0:1789760684.472626:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760684.472632:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2cb24 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20668416->20672511] (req 20668416->20672511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760684.472646:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2cb24 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20668416->20672511] (req 20668416->20672511) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760684.537894:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20668416 -> 20672512 00010000:00010000:2.0:1789760684.537910:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2cb24 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20668416->20672511] (req 20668416->20672511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760684.537924:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2cb24 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20668416->20672511] (req 20668416->20672511) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760684.537933:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2cb24 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20668416->20672511] (req 20668416->20672511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760684.537941:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760684.537946:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2cb24 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20668416->20672511] (req 20668416->20672511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760684.537955:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2cb24 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20668416->20672511] (req 20668416->20672511) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760684.537975:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2cb24 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20668416->20672511] (req 20668416->20672511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760684.537983:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2cb24 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20668416->20672511] (req 20668416->20672511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760684.538134:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac3301c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512502272:12345-192.168.203.56@tcp:4:dd.0 Request processed in 65742us (66026us total) trans 4294972365 rc 0/0 00000100:00100000:2.0:1789760684.538147:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5076 00000100:00100000:2.0:1789760684.538165:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 663959640863 time 664036677078 state 0x2 ref 0 00000100:00100000:2.0:1789760684.545930:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760684.607575:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512502528 00000100:00100000:2.0:1789760684.607585:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512502528 00000100:00100000:2.0:1789760684.607647:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 664036937494 time 664110691708 state 0x0 ref 1 00000100:00100000:2.0:1789760684.607663:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 664110952124 time 664110691708 state 0x1 ref 1 00000001:00080000:2.0:1789760684.639992:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972367 is committed 00000100:00100000:2.0:1789760684.648230:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00002000:00010000:2.0:1789760684.740851:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20680704 -> 20684800 00010000:00010000:2.0:1789760684.740865:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2cb39 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20680704->20684799] (req 20680704->20684799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760684.740878:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2cb39 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20680704->20684799] (req 20680704->20684799) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760684.740887:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2cb39 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20680704->20684799] (req 20680704->20684799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760684.740893:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760684.740898:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2cb39 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20680704->20684799] (req 20680704->20684799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760684.740906:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2cb39 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20680704->20684799] (req 20680704->20684799) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760684.740924:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2cb39 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20680704->20684799] (req 20680704->20684799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760684.740932:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2cb39 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20680704->20684799] (req 20680704->20684799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760684.741014:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4361500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512502656:12345-192.168.203.56@tcp:4:dd.0 Request processed in 92547us (92790us total) trans 4294972368 rc 0/0 00000100:00100000:2.0:1789760684.741024:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5079 00000100:00100000:2.0:1789760684.741032:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 664172366414 time 664212752675 state 0x2 ref 0 00000100:00100000:2.0:1789760684.746147:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760684.881978:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512503040 00000100:00100000:2.0:1789760684.881990:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512503040 00000100:00100000:2.0:1789760684.882052:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 664311428379 time 664368309630 state 0x0 ref 1 00000100:00100000:2.0:1789760684.882069:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 664368570046 time 664368309630 state 0x1 ref 1 00000100:00100000:2.0:1789760684.882107:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 664368570046 time 664446407225 state 0x0 ref 1 00000100:00100000:2.0:1789760684.882123:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febfbddc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512503040:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760684.882175:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20692992, mtime: 0x6aad94ac, atime: 0x0, ctime: 0x6aad94ac, blocks: 0xa007 00010000:00010000:2.0:1789760684.882185:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2cb4e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760684.882207:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=664s v=(0 0) 00010000:00010000:2.0:1789760684.882212:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760684.882218:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2cb4e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20692992->20697087] (req 20692992->20697087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760684.882233:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2cb4e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20692992->20697087] (req 20692992->20697087) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760684.952437:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760685.162614:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00002000:00010000:2.0:1789760685.341537:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20709376 -> 20713472 00010000:00010000:2.0:1789760685.341557:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2cb6a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20709376->20713471] (req 20709376->20713471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760685.341571:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2cb6a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20709376->20713471] (req 20709376->20713471) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760685.341581:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2cb6a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20709376->20713471] (req 20709376->20713471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760685.341588:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760685.341593:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2cb6a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20709376->20713471] (req 20709376->20713471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760685.341602:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2cb6a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20709376->20713471] (req 20709376->20713471) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760685.341626:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2cb6a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20709376->20713471] (req 20709376->20713471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760685.341634:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2cb6a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20709376->20713471] (req 20709376->20713471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760685.341772:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febea899180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512503552:12345-192.168.203.56@tcp:4:dd.0 Request processed in 109086us (109314us total) trans 4294972375 rc 0/0 00000100:00100000:2.0:1789760685.341783:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5086 00000100:00100000:2.0:1789760685.341792:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 664727791963 time 664796966540 state 0x2 ref 0 00000001:00080000:2.0:1789760685.423958:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972376 is committed 00000100:00100000:2.0:1789760685.505045:0:3310:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8febfb895880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3310:1876699540213376:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789760685.505117:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760685.505322:0:6707:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540213376 00000100:00100000:2.0:1789760685.505329:0:6707:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540213376 00000100:00100000:2.0:1789760685.505377:0:6707:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 175 00000100:00100000:2.0:1789760685.505386:0:6707:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febd02e3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3310:x1876699540213376:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1789760685.505418:0:6707:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3859808256 avail 3857711104 00000100:00100000:2.0:1789760685.505474:0:6707:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febd02e3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3310:x1876699540213376:12345-0@lo:13:osp-pre-1-0.0 Request processed in 88us (357us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760685.505481:0:6707:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 175 00000001:00080000:2.0:1789760685.528476:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972377 is committed 00000100:00100000:2.0:1789760685.636334:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760685.844852:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00002000:00010000:2.0:1789760685.921280:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20746240 -> 20750336 00010000:00010000:2.0:1789760685.921298:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2cba9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20746240->20750335] (req 20746240->20750335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760685.921312:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2cba9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20746240->20750335] (req 20746240->20750335) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760685.921322:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2cba9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20746240->20750335] (req 20746240->20750335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760685.921329:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760685.921334:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2cba9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20746240->20750335] (req 20746240->20750335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760685.921343:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2cba9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20746240->20750335] (req 20746240->20750335) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760685.921364:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2cba9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20746240->20750335] (req 20746240->20750335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760685.921372:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2cba9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20746240->20750335] (req 20746240->20750335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760685.921637:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac38b4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512504704:12345-192.168.203.56@tcp:4:dd.0 Request processed in 26941us (27188us total) trans 4294972384 rc 0/0 00000100:00100000:2.0:1789760685.921648:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5095 00000100:00100000:2.0:1789760685.921659:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 665409620779 time 665458980328 state 0x2 ref 0 00002000:00010000:2.0:1789760686.077723:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20762624 -> 20766720 00010000:00010000:2.0:1789760686.077739:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2cbc5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20762624->20766719] (req 20762624->20766719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760686.077871:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2cbc5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20762624->20766719] (req 20762624->20766719) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760686.077882:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2cbc5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20762624->20766719] (req 20762624->20766719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760686.077890:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760686.077895:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2cbc5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20762624->20766719] (req 20762624->20766719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760686.077905:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2cbc5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20762624->20766719] (req 20762624->20766719) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760686.077923:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2cbc5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20762624->20766719] (req 20762624->20766719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760686.077930:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2cbc5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20762624->20766719] (req 20762624->20766719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760686.078025:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febea4bac50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512505216:12345-192.168.203.56@tcp:4:dd.0 Request processed in 26155us (26542us total) trans 4294972388 rc 0/0 00000100:00100000:2.0:1789760686.078035:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5099 00000100:00100000:2.0:1789760686.078044:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 665569182113 time 665616152084 state 0x2 ref 0 00000100:00100000:2.0:1789760686.082104:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760686.127313:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512505472 00000100:00100000:2.0:1789760686.127323:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512505472 00000100:00100000:2.0:1789760686.127377:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 665616412500 time 665646823662 state 0x0 ref 1 00000100:00100000:2.0:1789760686.127392:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 665647084078 time 665646823662 state 0x1 ref 1 00000100:00100000:2.0:1789760686.127459:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 665647084078 time 665691758075 state 0x0 ref 1 00000100:00100000:2.0:1789760686.127472:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febfb89ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512505472:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760686.127520:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20770816, mtime: 0x6aad94ad, atime: 0x0, ctime: 0x6aad94ad, blocks: 0xa007 00010000:00010000:2.0:1789760686.127529:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2cbd3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760686.127544:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=665s v=(0 0) 00010000:00010000:2.0:1789760686.127549:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760686.127555:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2cbd3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20770816->20774911] (req 20770816->20774911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760686.127568:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2cbd3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20770816->20774911] (req 20770816->20774911) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760686.167971:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20770816 -> 20774912 00010000:00010000:2.0:1789760686.167987:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2cbd3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20770816->20774911] (req 20770816->20774911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760686.168005:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2cbd3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20770816->20774911] (req 20770816->20774911) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760686.168014:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2cbd3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20770816->20774911] (req 20770816->20774911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760686.168022:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760686.168026:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2cbd3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20770816->20774911] (req 20770816->20774911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760686.168036:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2cbd3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20770816->20774911] (req 20770816->20774911) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760686.168053:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2cbd3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20770816->20774911] (req 20770816->20774911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760686.168061:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2cbd3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20770816->20774911] (req 20770816->20774911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760686.168152:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febfb89ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512505472:12345-192.168.203.56@tcp:4:dd.0 Request processed in 40677us (40938us total) trans 4294972390 rc 0/0 00000100:00100000:2.0:1789760686.168162:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5101 00000100:00100000:2.0:1789760686.168170:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 665647084078 time 665691758075 state 0x2 ref 0 00000100:00100000:2.0:1789760686.174270:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512505600 00000100:00100000:2.0:1789760686.174281:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512505600 00000100:00100000:2.0:1789760686.174334:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 665647084078 time 665691758075 state 0x0 ref 1 00000100:00100000:2.0:1789760686.174348:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 665692018491 time 665691758075 state 0x1 ref 1 00000100:00100000:2.0:1789760686.174377:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 665692018491 time 665738676736 state 0x0 ref 1 00000100:00100000:2.0:1789760686.174390:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febffb9f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512505600:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760686.174445:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20774912, mtime: 0x6aad94ad, atime: 0x0, ctime: 0x6aad94ad, blocks: 0xa007 00010000:00010000:2.0:1789760686.174455:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2cbda lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760686.174470:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=665s v=(0 0) 00010000:00010000:2.0:1789760686.174475:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760686.174482:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2cbda lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20774912->20779007] (req 20774912->20779007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760686.174496:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2cbda lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20774912->20779007] (req 20774912->20779007) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760686.290336:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972392 is committed 00000100:00100000:2.0:1789760686.296011:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760686.331314:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000001:00080000:2.0:1789760686.371115:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972394 is committed 00000001:00080000:2.0:1789760686.413938:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972395 is committed 00000100:00100000:2.0:1789760686.420285:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00002000:00010000:2.0:1789760686.475321:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20795392 -> 20799488 00010000:00010000:2.0:1789760686.475338:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2cbfd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20795392->20799487] (req 20795392->20799487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760686.475352:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2cbfd lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20795392->20799487] (req 20795392->20799487) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760686.475362:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2cbfd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20795392->20799487] (req 20795392->20799487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760686.475370:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760686.475376:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2cbfd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20795392->20799487] (req 20795392->20799487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760686.475385:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2cbfd lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20795392->20799487] (req 20795392->20799487) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760686.475405:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2cbfd lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20795392->20799487] (req 20795392->20799487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760686.475413:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2cbfd lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20795392->20799487] (req 20795392->20799487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760686.475504:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2747800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512506240:12345-192.168.203.56@tcp:4:dd.0 Request processed in 54963us (55224us total) trans 4294972396 rc 0/0 00000100:00100000:2.0:1789760686.475515:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5107 00000100:00100000:2.0:1789760686.475525:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 665942345839 time 665984825707 state 0x2 ref 0 00000100:00100000:2.0:1789760686.565351:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000001:00080000:2.0:1789760686.594577:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972399 is committed 00000001:00080000:2.0:1789760686.636476:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972400 is committed 00002000:00010000:2.0:1789760686.636909:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20811776 -> 20815872 00010000:00010000:2.0:1789760686.636928:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2cc19 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20811776->20815871] (req 20811776->20815871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760686.636942:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2cc19 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20811776->20815871] (req 20811776->20815871) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760686.636952:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2cc19 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20811776->20815871] (req 20811776->20815871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760686.636960:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760686.636965:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2cc19 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20811776->20815871] (req 20811776->20815871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760686.636974:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2cc19 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20811776->20815871] (req 20811776->20815871) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760686.636995:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2cc19 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20811776->20815871] (req 20811776->20815871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760686.637004:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2cc19 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20811776->20815871] (req 20811776->20815871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760686.637193:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac38b5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512506752:12345-192.168.203.56@tcp:4:dd.0 Request processed in 38931us (39131us total) trans 4294972400 rc 0/0 00000100:00100000:2.0:1789760686.637206:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5111 00000100:00100000:2.0:1789760686.637216:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 666130225910 time 666162542224 state 0x2 ref 0 00002000:00010000:2.0:1789760686.699625:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20815872 -> 20819968 00010000:00010000:2.0:1789760686.699640:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2cc20 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20815872->20819967] (req 20815872->20819967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760686.699652:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2cc20 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20815872->20819967] (req 20815872->20819967) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760686.699662:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2cc20 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20815872->20819967] (req 20815872->20819967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760686.699670:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760686.699675:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2cc20 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20815872->20819967] (req 20815872->20819967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760686.699685:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2cc20 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20815872->20819967] (req 20815872->20819967) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760686.701271:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2cc20 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20815872->20819967] (req 20815872->20819967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760686.701288:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2cc20 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20815872->20819967] (req 20815872->20819967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760686.701451:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febffb9d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512506880:12345-192.168.203.56@tcp:4:dd.0 Request processed in 59469us (59770us total) trans 4294972401 rc 0/0 00000100:00100000:2.0:1789760686.701462:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5112 00000100:00100000:2.0:1789760686.701472:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 666162802640 time 666206265840 state 0x2 ref 0 00000100:00100000:2.0:1789760686.791965:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512507264 00000100:00100000:2.0:1789760686.791974:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512507264 00000100:00100000:2.0:1789760686.792106:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 666275997277 time 666311293424 state 0x0 ref 1 00000100:00100000:2.0:1789760686.792124:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 666311553840 time 666311293424 state 0x1 ref 1 00000100:00100000:2.0:1789760686.792285:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 666311553840 time 666356582115 state 0x0 ref 1 00000100:00100000:2.0:1789760686.792301:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febffb9c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512507264:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760686.792355:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20828160, mtime: 0x6aad94ae, atime: 0x0, ctime: 0x6aad94ae, blocks: 0xa007 00010000:00010000:2.0:1789760686.792364:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2cc35 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760686.792379:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=666s v=(0 0) 00010000:00010000:2.0:1789760686.792384:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760686.792391:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2cc35 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20828160->20832255] (req 20828160->20832255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760686.792402:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2cc35 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20828160->20832255] (req 20828160->20832255) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760686.902991:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512507648 00000100:00100000:2.0:1789760686.903001:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512507648 00000100:00100000:2.0:1789760686.903126:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 666386420881 time 666419991407 state 0x0 ref 1 00000100:00100000:2.0:1789760686.903207:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 666420251823 time 666419991407 state 0x1 ref 1 00000100:00100000:2.0:1789760686.903239:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 666420251823 time 666467538768 state 0x0 ref 1 00000100:00100000:2.0:1789760686.903251:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febffb9f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512507648:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760686.903296:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20840448, mtime: 0x6aad94ae, atime: 0x0, ctime: 0x6aad94ae, blocks: 0xa007 00010000:00010000:2.0:1789760686.903306:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2cc4a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760686.903322:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=666s v=(0 0) 00010000:00010000:2.0:1789760686.903326:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760686.903332:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2cc4a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20840448->20844543] (req 20840448->20844543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760686.903345:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2cc4a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20840448->20844543] (req 20840448->20844543) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1789760687.003000:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00010000:2.0:1789760687.058820:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20848640 -> 20852736 00010000:00010000:2.0:1789760687.058834:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2cc58 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20848640->20852735] (req 20848640->20852735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760687.058854:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2cc58 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20848640->20852735] (req 20848640->20852735) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760687.058863:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2cc58 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20848640->20852735] (req 20848640->20852735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760687.058870:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760687.058875:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2cc58 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20848640->20852735] (req 20848640->20852735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760687.058884:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2cc58 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20848640->20852735] (req 20848640->20852735) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760687.058901:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2cc58 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20848640->20852735] (req 20848640->20852735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760687.058909:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2cc58 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20848640->20852735] (req 20848640->20852735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760687.059086:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4059180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512507904:12345-192.168.203.56@tcp:4:dd.0 Request processed in 56742us (57172us total) trans 4294972409 rc 0/0 00000100:00100000:2.0:1789760687.059097:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5120 00000100:00100000:2.0:1789760687.059106:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 666514303034 time 666566628047 state 0x2 ref 0 00000100:00100000:2.0:1789760687.120964:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512508288 00000100:00100000:2.0:1789760687.120974:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512508288 00000100:00100000:2.0:1789760687.121023:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 666627049299 time 666655268537 state 0x0 ref 1 00000100:00100000:2.0:1789760687.121037:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 666655528953 time 666655268537 state 0x1 ref 1 00000100:00100000:2.0:1789760687.121068:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 666655528953 time 666685367422 state 0x0 ref 1 00000100:00100000:2.0:1789760687.121080:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac405ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512508288:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760687.121126:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20860928, mtime: 0x6aad94ae, atime: 0x0, ctime: 0x6aad94ae, blocks: 0xa007 00010000:00010000:2.0:1789760687.121207:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2cc6d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760687.121222:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=666s v=(0 0) 00010000:00010000:2.0:1789760687.121226:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760687.121236:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2cc6d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20860928->20865023] (req 20860928->20865023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760687.121251:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2cc6d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20860928->20865023] (req 20860928->20865023) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760687.153749:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20860928 -> 20865024 00010000:00010000:2.0:1789760687.153770:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2cc6d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20860928->20865023] (req 20860928->20865023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760687.153785:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2cc6d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20860928->20865023] (req 20860928->20865023) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760687.153801:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2cc6d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20860928->20865023] (req 20860928->20865023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760687.153808:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760687.153813:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2cc6d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20860928->20865023] (req 20860928->20865023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760687.153823:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2cc6d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20860928->20865023] (req 20860928->20865023) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760687.153842:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2cc6d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20860928->20865023] (req 20860928->20865023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760687.153850:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2cc6d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20860928->20865023] (req 20860928->20865023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760687.153928:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac405ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2360:x1876699512508288:12345-192.168.203.56@tcp:4:dd.0 Request processed in 32847us (33044us total) trans 4294972412 rc 0/0 00000100:00100000:2.0:1789760687.153939:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5123 00000100:00100000:2.0:1789760687.153948:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 666655528953 time 666685367422 state 0x2 ref 0 00000100:00100000:2.0:1789760687.202387:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760687.261116:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512508672 00000100:00100000:2.0:1789760687.261127:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512508672 00000100:00100000:2.0:1789760687.261237:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 666723450899 time 666766951021 state 0x0 ref 1 00000100:00100000:2.0:1789760687.261255:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 666767211437 time 666766951021 state 0x1 ref 1 00000100:00100000:2.0:1789760687.261285:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 666767211437 time 666825584596 state 0x0 ref 1 00000100:00100000:2.0:1789760687.261408:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febfb89fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512508672:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760687.261521:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20873216, mtime: 0x6aad94ae, atime: 0x0, ctime: 0x6aad94ae, blocks: 0xa007 00010000:00010000:2.0:1789760687.261530:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2cc82 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760687.261547:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=666s v=(0 0) 00010000:00010000:2.0:1789760687.261551:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760687.261557:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2cc82 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20873216->20877311] (req 20873216->20877311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760687.261570:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2cc82 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20873216->20877311] (req 20873216->20877311) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760687.313614:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20873216 -> 20877312 00010000:00010000:2.0:1789760687.313635:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2cc82 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20873216->20877311] (req 20873216->20877311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760687.313649:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2cc82 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20873216->20877311] (req 20873216->20877311) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760687.313659:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2cc82 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20873216->20877311] (req 20873216->20877311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760687.313672:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760687.313677:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2cc82 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20873216->20877311] (req 20873216->20877311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760687.313687:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2cc82 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20873216->20877311] (req 20873216->20877311) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760687.313726:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2cc82 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20873216->20877311] (req 20873216->20877311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760687.313734:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2cc82 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20873216->20877311] (req 20873216->20877311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760687.313834:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febfb89fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2361:x1876699512508672:12345-192.168.203.56@tcp:4:dd.0 Request processed in 52534us (52878us total) trans 4294972415 rc 0/0 00000100:00100000:2.0:1789760687.313844:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5126 00000100:00100000:2.0:1789760687.313853:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 666767211437 time 666825584596 state 0x2 ref 0 00000100:00100000:2.0:1789760687.412661:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760687.522647:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000001:00080000:2.0:1789760687.556302:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972421 is committed 00000100:00100000:2.0:1789760687.560673:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512509568 00000100:00100000:2.0:1789760687.560683:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512509568 00000100:00100000:2.0:1789760687.562231:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 667051213355 time 667087262196 state 0x0 ref 1 00000100:00100000:2.0:1789760687.562249:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 667087522612 time 667087262196 state 0x1 ref 1 00000100:00100000:2.0:1789760687.562291:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 667087522612 time 667126591209 state 0x0 ref 1 00000100:00100000:2.0:1789760687.562305:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febfb894700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512509568:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760687.562353:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20901888, mtime: 0x6aad94af, atime: 0x0, ctime: 0x6aad94af, blocks: 0xa007 00010000:00010000:2.0:1789760687.562361:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2ccb3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760687.562377:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=667s v=(0 0) 00010000:00010000:2.0:1789760687.562382:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760687.562388:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2ccb3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20901888->20905983] (req 20901888->20905983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760687.562402:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2ccb3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20901888->20905983] (req 20901888->20905983) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760687.600875:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20901888 -> 20905984 00010000:00010000:2.0:1789760687.600890:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2ccb3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20901888->20905983] (req 20901888->20905983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760687.600903:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2ccb3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20901888->20905983] (req 20901888->20905983) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760687.600911:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2ccb3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20901888->20905983] (req 20901888->20905983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760687.600918:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760687.600924:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2ccb3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20901888->20905983] (req 20901888->20905983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760687.600933:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2ccb3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20901888->20905983] (req 20901888->20905983) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760687.600950:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2ccb3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20901888->20905983] (req 20901888->20905983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760687.600957:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2ccb3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20901888->20905983] (req 20901888->20905983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760687.601216:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febfb894700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512509568:12345-192.168.203.56@tcp:4:dd.0 Request processed in 38909us (40664us total) trans 4294972422 rc 0/0 00000100:00100000:2.0:1789760687.601227:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5133 00000100:00100000:2.0:1789760687.601236:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 667087522612 time 667126591209 state 0x2 ref 0 00000100:00100000:2.0:1789760687.605405:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 667126851625 time 667169702209 state 0x0 ref 1 00000100:00100000:2.0:1789760687.605433:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febea772680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512509696:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760687.605484:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20905984, mtime: 0x6aad94af, atime: 0x0, ctime: 0x6aad94af, blocks: 0xa007 00010000:00010000:2.0:1789760687.605494:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2ccba lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760687.605510:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=667s v=(0 0) 00010000:00010000:2.0:1789760687.605515:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760687.605522:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2ccba lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20905984->20910079] (req 20905984->20910079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760687.605536:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2ccba lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20905984->20910079] (req 20905984->20910079) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760687.684797:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00002000:00010000:2.0:1789760687.722369:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20914176 -> 20918272 00010000:00010000:2.0:1789760687.722384:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2ccc8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20914176->20918271] (req 20914176->20918271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760687.722397:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2ccc8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20914176->20918271] (req 20914176->20918271) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760687.722468:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2ccc8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20914176->20918271] (req 20914176->20918271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760687.722478:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760687.722483:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2ccc8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20914176->20918271] (req 20914176->20918271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760687.722492:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2ccc8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20914176->20918271] (req 20914176->20918271) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760687.722511:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2ccc8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20914176->20918271] (req 20914176->20918271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760687.722520:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2ccc8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20914176->20918271] (req 20914176->20918271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760687.722618:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2745880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512509952:12345-192.168.203.56@tcp:4:dd.0 Request processed in 37511us (37827us total) trans 4294972425 rc 0/0 00000100:00100000:2.0:1789760687.722630:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5136 00000100:00100000:2.0:1789760687.722640:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 667206837590 time 667249390621 state 0x2 ref 0 00000100:00100000:2.0:1789760687.725833:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00002000:00010000:2.0:1789760687.764995:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20918272 -> 20922368 00010000:00010000:2.0:1789760687.765086:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2cccf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20918272->20922367] (req 20918272->20922367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760687.765102:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2cccf lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20918272->20922367] (req 20918272->20922367) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760687.765112:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2cccf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20918272->20922367] (req 20918272->20922367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760687.765121:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760687.765191:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2cccf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20918272->20922367] (req 20918272->20922367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760687.765204:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2cccf lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20918272->20922367] (req 20918272->20922367) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760687.765225:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2cccf lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20918272->20922367] (req 20918272->20922367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760687.765239:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2cccf lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20918272->20922367] (req 20918272->20922367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760687.765374:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2745180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512510080:12345-192.168.203.56@tcp:4:dd.0 Request processed in 39250us (39544us total) trans 4294972426 rc 0/0 00000100:00100000:2.0:1789760687.765385:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5137 00000100:00100000:2.0:1789760687.765395:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 667249651037 time 667290409079 state 0x2 ref 0 00000100:00100000:2.0:1789760687.768418:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760687.809120:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512510336 00000100:00100000:2.0:1789760687.809138:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512510336 00000100:00100000:2.0:1789760687.809189:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 667290669495 time 667332885546 state 0x0 ref 1 00000100:00100000:2.0:1789760687.809204:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 667333145962 time 667332885546 state 0x1 ref 1 00000100:00100000:2.0:1789760687.809237:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 667333145962 time 667373537096 state 0x0 ref 1 00000100:00100000:2.0:1789760687.809248:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febea770380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512510336:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760687.809294:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20926464, mtime: 0x6aad94af, atime: 0x0, ctime: 0x6aad94af, blocks: 0xa007 00010000:00010000:2.0:1789760687.809302:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2ccdd lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760687.809317:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=667s v=(0 0) 00010000:00010000:2.0:1789760687.809322:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760687.809328:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2ccdd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20926464->20930559] (req 20926464->20930559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760687.809342:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2ccdd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20926464->20930559] (req 20926464->20930559) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760687.848483:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20926464 -> 20930560 00010000:00010000:2.0:1789760687.848498:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2ccdd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20926464->20930559] (req 20926464->20930559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760687.848512:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2ccdd lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20926464->20930559] (req 20926464->20930559) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760687.848521:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2ccdd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20926464->20930559] (req 20926464->20930559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760687.848529:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760687.848534:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2ccdd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20926464->20930559] (req 20926464->20930559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760687.848544:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2ccdd lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20926464->20930559] (req 20926464->20930559) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760687.848562:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2ccdd lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20926464->20930559] (req 20926464->20930559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760687.848570:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2ccdd lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20926464->20930559] (req 20926464->20930559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760687.848752:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febea770380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512510336:12345-192.168.203.56@tcp:4:dd.0 Request processed in 39491us (39791us total) trans 4294972428 rc 0/0 00000100:00100000:2.0:1789760687.848764:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5139 00000100:00100000:2.0:1789760687.848773:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 667333145962 time 667373537096 state 0x2 ref 0 00000100:00100000:2.0:1789760687.851914:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760687.928320:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00010000:00010000:3.0:1789760687.960003:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2ccf2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20938752->20942847] (req 20938752->20942847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760687.960094:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2ccf2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20938752->20942847] (req 20938752->20942847) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760687.960104:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2ccf2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20938752->20942847] (req 20938752->20942847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760687.960112:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760687.960117:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2ccf2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20938752->20942847] (req 20938752->20942847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760687.960198:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2ccf2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20938752->20942847] (req 20938752->20942847) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760687.960221:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2ccf2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20938752->20942847] (req 20938752->20942847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760687.960229:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2ccf2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20938752->20942847] (req 20938752->20942847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760687.960326:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2746d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2360:x1876699512510720:12345-192.168.203.56@tcp:4:dd.0 Request processed in 31678us (32012us total) trans 4294972431 rc 0/0 00000100:00100000:3.0:1789760687.960337:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5142 00000100:00100000:3.0:1789760687.960347:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 667457002749 time 667492932266 state 0x2 ref 0 00000100:00100000:2.0:1789760687.963994:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760687.964520:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512510848 00000100:00100000:3.0:1789760687.964532:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512510848 00000100:00100000:2.0:1789760687.964799:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 667457002749 time 667492932266 state 0x0 ref 1 00000100:00100000:2.0:1789760687.964816:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 667493192682 time 667492932266 state 0x1 ref 1 00000100:00100000:2.0:1789760687.964886:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 667493192682 time 667529185706 state 0x0 ref 1 00000100:00100000:2.0:1789760687.964899:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2746300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512510848:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760687.964945:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20942848, mtime: 0x6aad94af, atime: 0x0, ctime: 0x6aad94af, blocks: 0xa007 00010000:00010000:2.0:1789760687.964954:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2ccf9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760687.964971:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=667s v=(0 0) 00010000:00010000:2.0:1789760687.964975:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760687.964981:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2ccf9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20942848->20946943] (req 20942848->20946943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760687.964994:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2ccf9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20942848->20946943] (req 20942848->20946943) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760688.000586:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760688.000672:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512510976 00000100:00100000:3.0:1789760688.000682:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512510976 00000100:00100000:3.0:1789760688.000784:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 667493192682 time 667529185706 state 0x0 ref 1 00000100:00100000:3.0:1789760688.000799:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 667529446122 time 667529185706 state 0x1 ref 1 00000100:00100000:3.0:1789760688.000831:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 667529446122 time 667565130374 state 0x0 ref 1 00000100:00100000:3.0:1789760688.000843:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2745c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512510976:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760688.000890:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20946944, mtime: 0x6aad94af, atime: 0x0, ctime: 0x6aad94af, blocks: 0xa007 00010000:00010000:3.0:1789760688.000898:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2cd00 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760688.000914:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=667s v=(0 0) 00010000:00010000:3.0:1789760688.000919:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760688.000925:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2cd00 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20946944->20951039] (req 20946944->20951039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760688.000940:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2cd00 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20946944->20951039] (req 20946944->20951039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760688.034780:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20946944 -> 20951040 00010000:00010000:3.0:1789760688.034796:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2cd00 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20946944->20951039] (req 20946944->20951039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760688.034810:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2cd00 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20946944->20951039] (req 20946944->20951039) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760688.034820:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2cd00 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20946944->20951039] (req 20946944->20951039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760688.034828:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760688.034833:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2cd00 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20946944->20951039] (req 20946944->20951039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760688.034844:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2cd00 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20946944->20951039] (req 20946944->20951039) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760688.034863:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2cd00 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20946944->20951039] (req 20946944->20951039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760688.034872:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2cd00 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20946944->20951039] (req 20946944->20951039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760688.034966:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2745c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512510976:12345-192.168.203.56@tcp:4:dd.0 Request processed in 34120us (34384us total) trans 4294972433 rc 0/0 00000100:00100000:3.0:1789760688.034976:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5144 00000100:00100000:3.0:1789760688.034986:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 667529446122 time 667565130374 state 0x2 ref 0 00000100:00100000:3.0:1789760688.037363:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760688.068901:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512511232 00000100:00100000:3.0:1789760688.068912:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512511232 00000100:00100000:3.0:1789760688.068975:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 667565390790 time 667602177093 state 0x0 ref 1 00000100:00100000:3.0:1789760688.068994:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 667602437509 time 667602177093 state 0x1 ref 1 00000100:00100000:3.0:1789760688.069028:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 667602437509 time 667633327311 state 0x0 ref 1 00000100:00100000:3.0:1789760688.069042:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febfb895880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512511232:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760688.069096:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20955136, mtime: 0x6aad94af, atime: 0x0, ctime: 0x6aad94af, blocks: 0xa007 00010000:00010000:3.0:1789760688.069106:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2cd0e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760688.069123:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=667s v=(0 0) 00010000:00010000:3.0:1789760688.069137:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760688.069144:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2cd0e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20955136->20959231] (req 20955136->20959231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760688.069158:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2cd0e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20955136->20959231] (req 20955136->20959231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760688.109217:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760688.109360:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512511360 00000100:00100000:3.0:1789760688.109368:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512511360 00000100:00100000:3.0:1789760688.109416:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 667602437509 time 667633327311 state 0x0 ref 1 00000100:00100000:3.0:1789760688.109433:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 667633587727 time 667633327311 state 0x1 ref 1 00000100:00100000:3.0:1789760688.109459:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 667633587727 time 667673759197 state 0x0 ref 1 00000100:00100000:3.0:1789760688.109472:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2744e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512511360:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760688.109527:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20959232, mtime: 0x6aad94af, atime: 0x0, ctime: 0x6aad94af, blocks: 0xa007 00010000:00010000:3.0:1789760688.109536:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2cd15 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760688.109552:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=667s v=(0 0) 00010000:00010000:3.0:1789760688.109557:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760688.109564:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2cd15 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20959232->20963327] (req 20959232->20963327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760688.109577:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2cd15 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20959232->20963327] (req 20959232->20963327) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760688.169420:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20959232 -> 20963328 00010000:00010000:3.0:1789760688.169442:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2cd15 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20959232->20963327] (req 20959232->20963327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760688.169469:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2cd15 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20959232->20963327] (req 20959232->20963327) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760688.169479:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2cd15 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20959232->20963327] (req 20959232->20963327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760688.169486:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760688.169491:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2cd15 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20959232->20963327] (req 20959232->20963327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760688.169501:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2cd15 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20959232->20963327] (req 20959232->20963327) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760688.169521:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2cd15 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20959232->20963327] (req 20959232->20963327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760688.169528:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2cd15 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20959232->20963327] (req 20959232->20963327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760688.169631:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2744e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2359:x1876699512511360:12345-192.168.203.56@tcp:4:dd.0 Request processed in 60156us (60417us total) trans 4294972436 rc 0/0 00000100:00100000:3.0:1789760688.169641:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5147 00000100:00100000:3.0:1789760688.169650:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 667633587727 time 667673759197 state 0x2 ref 0 00000100:00100000:2.0:1789760688.174818:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00002000:00010000:3.0:1789760688.233013:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20963328 -> 20967424 00010000:00010000:3.0:1789760688.233033:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2cd1c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20963328->20967423] (req 20963328->20967423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760688.233045:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2cd1c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20963328->20967423] (req 20963328->20967423) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760688.233054:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2cd1c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20963328->20967423] (req 20963328->20967423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760688.233061:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760688.233067:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2cd1c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20963328->20967423] (req 20963328->20967423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760688.233080:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2cd1c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20963328->20967423] (req 20963328->20967423) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760688.233098:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2cd1c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20963328->20967423] (req 20963328->20967423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760688.233106:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2cd1c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20963328->20967423] (req 20963328->20967423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760688.233203:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2747100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2358:x1876699512511488:12345-192.168.203.56@tcp:4:dd.0 Request processed in 58026us (58390us total) trans 4294972437 rc 0/0 00000100:00100000:3.0:1789760688.233213:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5148 00000100:00100000:3.0:1789760688.233222:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 667674019613 time 667739461841 state 0x2 ref 0 00000100:00100000:2.0:1789760688.237425:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00002000:00010000:2.0:1789760688.297568:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20967424 -> 20971520 00010000:00010000:2.0:1789760688.297588:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2cd23 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20967424->20971519] (req 20967424->20971519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760688.297602:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2cd23 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20967424->20971519] (req 20967424->20971519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760688.297622:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2cd23 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20967424->20971519] (req 20967424->20971519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760688.297629:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760688.297634:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2cd23 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20967424->20971519] (req 20967424->20971519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760688.297642:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2cd23 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20967424->20971519] (req 20967424->20971519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760688.297662:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2cd23 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20967424->20971519] (req 20967424->20971519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760688.297670:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2cd23 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20967424->20971519] (req 20967424->20971519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760688.297812:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2745f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2359:x1876699512511616:12345-192.168.203.56@tcp:4:dd.0 Request processed in 59957us (60390us total) trans 4294972438 rc 0/0 00000100:00100000:2.0:1789760688.297822:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5149 00000100:00100000:2.0:1789760688.297831:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 667739722257 time 667802139977 state 0x2 ref 0 00000100:00100000:2.0:1789760688.300821:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760688.301287:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512511744 00000100:00100000:3.0:1789760688.301297:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512511744 00000100:00100000:3.0:1789760688.301354:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 667739722257 time 667802139977 state 0x0 ref 1 00000100:00100000:3.0:1789760688.301369:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 667802400393 time 667802139977 state 0x1 ref 1 00000100:00100000:3.0:1789760688.301399:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 667802400393 time 667865698958 state 0x0 ref 1 00000100:00100000:3.0:1789760688.301412:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febe7907100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512511744:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760688.301617:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20971520, mtime: 0x6aad94af, atime: 0x0, ctime: 0x6aad94af, blocks: 0xa007 00010000:00010000:3.0:1789760688.301628:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2cd2a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760688.301646:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=667s v=(0 0) 00010000:00010000:3.0:1789760688.301650:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760688.301657:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2cd2a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20971520->20975615] (req 20971520->20975615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760688.301670:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2cd2a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20971520->20975615] (req 20971520->20975615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760688.349533:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20971520 -> 20975616 00002000:00010000:2.0:1789760688.349538:0:13062:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb blocks from disk: 40967 -> 43015 00010000:00010000:2.0:1789760688.349551:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2cd2a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20971520->20975615] (req 20971520->20975615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760688.349563:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2cd2a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20971520->20975615] (req 20971520->20975615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760688.349572:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2cd2a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20971520->20975615] (req 20971520->20975615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760688.349578:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760688.349584:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2cd2a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20971520->20975615] (req 20971520->20975615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760688.349593:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2cd2a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20971520->20975615] (req 20971520->20975615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760688.349673:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2cd2a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20971520->20975615] (req 20971520->20975615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760688.349683:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2cd2a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20971520->20975615] (req 20971520->20975615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760688.353183:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512511872 00000100:00100000:3.0:1789760688.353194:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512511872 00000100:00100000:3.0:1789760688.353246:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 667802400393 time 667865698958 state 0x0 ref 1 00000100:00100000:3.0:1789760688.353261:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 667865959374 time 667865698958 state 0x1 ref 1 00000100:00100000:3.0:1789760688.353294:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 667865959374 time 667917593563 state 0x0 ref 1 00000100:00100000:3.0:1789760688.353307:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febff929f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512511872:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760688.353357:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20975616, mtime: 0x6aad94af, atime: 0x0, ctime: 0x6aad94af, blocks: 0xa807 00010000:00010000:3.0:1789760688.353365:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2cd31 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760688.353380:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=667s v=(0 0) 00010000:00010000:3.0:1789760688.353385:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760688.353391:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2cd31 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20975616->20979711] (req 20975616->20979711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760688.353411:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2cd31 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20975616->20979711] (req 20975616->20979711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760688.393272:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760688.393316:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760688.393339:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760688.393365:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512512000 00000100:00100000:2.0:1789760688.393375:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512512000 00000100:00100000:2.0:1789760688.393439:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 376 00000100:00100000:2.0:1789760688.393449:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febe00d6050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:e6cf7267-9b19-462a-a263-14115df15dab+13:2359:x1876699512512000:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:2.0:1789760688.393515:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febe00d6050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:e6cf7267-9b19-462a-a263-14115df15dab+13:2359:x1876699512512000:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 68us (198us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760688.393523:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 376 00000100:00100000:2.0:1789760688.393555:0:5848:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512512128 00000100:00100000:2.0:1789760688.393559:0:5848:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512512128 00000100:00100000:2.0:1789760688.393590:0:5848:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512512256 00000100:00100000:2.0:1789760688.393594:0:5848:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512512256 00000100:00100000:2.0:1789760688.393615:0:5848:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 419 00000100:00100000:2.0:1789760688.394183:0:5848:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febe7904a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+8:2359:x1876699512512128:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:2.0:1789760688.394224:0:5848:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febe7904a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+8:2359:x1876699512512128:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 602us (960us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760688.394230:0:5848:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 419 00000100:00100000:2.0:1789760688.394246:0:5848:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 420 00000100:00100000:2.0:1789760688.394252:0:5848:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febe7906a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:d2c40150-e97b-40da-952b-cb9f18bf7e96+5:2359:x1876699512512256:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:2.0:1789760688.394277:0:5848:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febe7906a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:d2c40150-e97b-40da-952b-cb9f18bf7e96+5:2359:x1876699512512256:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 26us (937us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760688.394282:0:5848:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 420 00000100:00080000:2.0:1789760688.394397:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789760688.402068:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760688.402227:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512512384 00000100:00100000:3.0:1789760688.402237:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512512384 00000100:00100000:3.0:1789760688.402289:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 667865959374 time 667917593563 state 0x0 ref 1 00000100:00100000:3.0:1789760688.402304:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 667917853979 time 667917593563 state 0x1 ref 1 00000100:00100000:3.0:1789760688.402367:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 667917853979 time 667966666906 state 0x0 ref 1 00000100:00100000:3.0:1789760688.402381:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2fb6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512512384:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760688.402439:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20979712, mtime: 0x6aad94af, atime: 0x0, ctime: 0x6aad94af, blocks: 0xa807 00010000:00010000:3.0:1789760688.402448:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2cd38 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760688.402473:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=667s v=(0 0) 00010000:00010000:3.0:1789760688.402478:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760688.402485:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2cd38 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20979712->20983807] (req 20979712->20983807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760688.402499:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2cd38 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20979712->20983807] (req 20979712->20983807) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760688.460606:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512512512 00000100:00100000:3.0:1789760688.460616:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512512512 00000100:00100000:3.0:1789760688.460676:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 667917853979 time 667966666906 state 0x0 ref 1 00000100:00100000:3.0:1789760688.460692:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 667966927322 time 667966666906 state 0x1 ref 1 00000100:00100000:2.0:1789760688.460922:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 667966927322 time 668025218835 state 0x0 ref 1 00000100:00100000:2.0:1789760688.460938:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febff92b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512512512:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760688.460989:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20983808, mtime: 0x6aad94af, atime: 0x0, ctime: 0x6aad94af, blocks: 0xa807 00010000:00010000:2.0:1789760688.460998:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2cd3f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760688.461019:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=668s v=(0 0) 00010000:00010000:2.0:1789760688.461023:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760688.461030:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2cd3f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20983808->20987903] (req 20983808->20987903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760688.461044:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2cd3f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20983808->20987903] (req 20983808->20987903) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760688.497149:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20983808 -> 20987904 00010000:00010000:3.0:1789760688.497167:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2cd3f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20983808->20987903] (req 20983808->20987903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760688.497185:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2cd3f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20983808->20987903] (req 20983808->20987903) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760688.497194:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2cd3f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20983808->20987903] (req 20983808->20987903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760688.497201:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760688.497206:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2cd3f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20983808->20987903] (req 20983808->20987903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760688.497214:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2cd3f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20983808->20987903] (req 20983808->20987903) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760688.497231:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2cd3f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20983808->20987903] (req 20983808->20987903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760688.497237:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2cd3f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20983808->20987903] (req 20983808->20987903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760688.497362:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febff92b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512512512:12345-192.168.203.56@tcp:4:dd.0 Request processed in 36424us (37135us total) trans 4294972442 rc 0/0 00000100:00100000:3.0:1789760688.497372:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5153 00000100:00100000:3.0:1789760688.497381:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 667966927322 time 668025218835 state 0x2 ref 0 00000100:00100000:2.0:1789760688.505031:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760688.555667:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760688.629227:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512512896 00000100:00100000:3.0:1789760688.629237:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512512896 00000100:00100000:3.0:1789760688.629294:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 668070142948 time 668122338278 state 0x0 ref 1 00000100:00100000:3.0:1789760688.629311:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 668122598694 time 668122338278 state 0x1 ref 1 00000100:00100000:3.0:1789760688.629390:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 668122598694 time 668193688730 state 0x0 ref 1 00000100:00100000:3.0:1789760688.629415:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febea4bb050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512512896:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760688.629463:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20996096, mtime: 0x6aad94b0, atime: 0x0, ctime: 0x6aad94b0, blocks: 0xa807 00010000:00010000:3.0:1789760688.629474:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2cd54 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760688.629491:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=668s v=(0 0) 00010000:00010000:3.0:1789760688.629496:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760688.629502:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2cd54 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20996096->21000191] (req 20996096->21000191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760688.629531:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2cd54 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20996096->21000191] (req 20996096->21000191) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760688.665064:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20996096 -> 21000192 00010000:00010000:3.0:1789760688.665079:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2cd54 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20996096->21000191] (req 20996096->21000191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760688.665092:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2cd54 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20996096->21000191] (req 20996096->21000191) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760688.665101:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2cd54 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20996096->21000191] (req 20996096->21000191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760688.665108:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760688.665113:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2cd54 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20996096->21000191] (req 20996096->21000191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760688.665121:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2cd54 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20996096->21000191] (req 20996096->21000191) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760688.665140:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2cd54 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20996096->21000191] (req 20996096->21000191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760688.665147:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2cd54 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20996096->21000191] (req 20996096->21000191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760688.665249:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febea4bb050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512512896:12345-192.168.203.56@tcp:4:dd.0 Request processed in 35843us (36236us total) trans 4294972445 rc 0/0 00000100:00100000:3.0:1789760688.665262:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5156 00000100:00100000:3.0:1789760688.665272:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 668122598694 time 668193688730 state 0x2 ref 0 00000100:00100000:2.0:1789760688.671117:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00002000:00010000:2.0:1789760688.722430:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21000192 -> 21004288 00010000:00010000:2.0:1789760688.722449:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2cd5b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21000192->21004287] (req 21000192->21004287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760688.722480:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2cd5b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21000192->21004287] (req 21000192->21004287) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760688.722490:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2cd5b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21000192->21004287] (req 21000192->21004287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760688.722498:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760688.722503:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2cd5b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21000192->21004287] (req 21000192->21004287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760688.722512:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2cd5b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21000192->21004287] (req 21000192->21004287) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760688.722530:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2cd5b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21000192->21004287] (req 21000192->21004287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760688.722538:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2cd5b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21000192->21004287] (req 21000192->21004287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760688.722644:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febc7b33480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2359:x1876699512513024:12345-192.168.203.56@tcp:4:dd.0 Request processed in 50851us (51532us total) trans 4294972446 rc 0/0 00000100:00100000:2.0:1789760688.722655:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5157 00000100:00100000:2.0:1789760688.722665:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 668193949146 time 668236076431 state 0x2 ref 0 00000100:00100000:2.0:1789760688.726218:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512513152 00000100:00100000:2.0:1789760688.726228:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512513152 00000100:00100000:2.0:1789760688.726280:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 668193949146 time 668236076431 state 0x0 ref 1 00000100:00100000:2.0:1789760688.726296:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 668236336847 time 668236076431 state 0x1 ref 1 00000100:00100000:2.0:1789760688.726373:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 668236336847 time 668290671225 state 0x0 ref 1 00000100:00100000:2.0:1789760688.726386:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febea771880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2361:x1876699512513152:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760688.726446:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21004288, mtime: 0x6aad94b0, atime: 0x0, ctime: 0x6aad94b0, blocks: 0xa807 00010000:00010000:2.0:1789760688.726465:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2cd62 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760688.726488:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=668s v=(0 0) 00010000:00010000:2.0:1789760688.726492:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760688.726499:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2cd62 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21004288->21008383] (req 21004288->21008383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760688.726514:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2cd62 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21004288->21008383] (req 21004288->21008383) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760688.771148:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972447 is committed 00002000:00010000:2.0:1789760688.771783:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21004288 -> 21008384 00010000:00010000:2.0:1789760688.771800:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2cd62 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21004288->21008383] (req 21004288->21008383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760688.771815:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2cd62 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21004288->21008383] (req 21004288->21008383) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760688.771831:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2cd62 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21004288->21008383] (req 21004288->21008383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760688.771839:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760688.771844:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2cd62 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21004288->21008383] (req 21004288->21008383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760688.771854:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2cd62 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21004288->21008383] (req 21004288->21008383) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760688.771874:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2cd62 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21004288->21008383] (req 21004288->21008383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760688.771883:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2cd62 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21004288->21008383] (req 21004288->21008383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760688.772046:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febea771880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512513152:12345-192.168.203.56@tcp:4:dd.0 Request processed in 45656us (46074us total) trans 4294972447 rc 0/0 00000100:00100000:2.0:1789760688.772056:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5158 00000100:00100000:2.0:1789760688.772067:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 668236336847 time 668290671225 state 0x2 ref 0 00000100:00100000:3.0:1789760688.776892:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512513280 00000100:00100000:3.0:1789760688.776902:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512513280 00000100:00100000:2.0:1789760688.777034:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 668236336847 time 668290671225 state 0x0 ref 1 00000100:00100000:2.0:1789760688.777049:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 668290931641 time 668290671225 state 0x1 ref 1 00000100:00100000:2.0:1789760688.777128:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 668290931641 time 668341418610 state 0x0 ref 1 00000100:00100000:2.0:1789760688.777141:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febff92aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512513280:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760688.777193:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21008384, mtime: 0x6aad94b0, atime: 0x0, ctime: 0x6aad94b0, blocks: 0xa807 00010000:00010000:2.0:1789760688.777202:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2cd69 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760688.777219:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=668s v=(0 0) 00010000:00010000:2.0:1789760688.777224:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760688.777230:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2cd69 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21008384->21012479] (req 21008384->21012479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760688.777244:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2cd69 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21008384->21012479] (req 21008384->21012479) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760688.812338:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21008384 -> 21012480 00010000:00010000:2.0:1789760688.812353:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2cd69 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21008384->21012479] (req 21008384->21012479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760688.812367:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2cd69 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21008384->21012479] (req 21008384->21012479) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760688.812376:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2cd69 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21008384->21012479] (req 21008384->21012479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760688.812383:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760688.812389:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2cd69 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21008384->21012479] (req 21008384->21012479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760688.812458:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2cd69 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21008384->21012479] (req 21008384->21012479) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760688.812478:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2cd69 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21008384->21012479] (req 21008384->21012479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760688.812487:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2cd69 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21008384->21012479] (req 21008384->21012479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760688.812690:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febff92aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512513280:12345-192.168.203.56@tcp:4:dd.0 Request processed in 35546us (36154us total) trans 4294972448 rc 0/0 00000100:00100000:2.0:1789760688.812751:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5159 00000100:00100000:2.0:1789760688.812761:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 668290931641 time 668341418610 state 0x2 ref 0 00000100:00100000:3.0:1789760688.816449:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760688.816514:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512513408 00000100:00100000:2.0:1789760688.816523:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512513408 00000100:00100000:2.0:1789760688.816585:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 668290931641 time 668341418610 state 0x0 ref 1 00000100:00100000:2.0:1789760688.816601:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 668341679026 time 668341418610 state 0x1 ref 1 00000100:00100000:2.0:1789760688.816638:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 668341679026 time 668380938304 state 0x0 ref 1 00000100:00100000:2.0:1789760688.816651:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac38b5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512513408:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760688.816740:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21012480, mtime: 0x6aad94b0, atime: 0x0, ctime: 0x6aad94b0, blocks: 0xa807 00010000:00010000:2.0:1789760688.816749:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2cd70 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760688.816765:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=668s v=(0 0) 00010000:00010000:2.0:1789760688.816770:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760688.816776:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2cd70 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21012480->21016575] (req 21012480->21016575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760688.816791:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2cd70 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21012480->21016575] (req 21012480->21016575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760688.925763:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21012480 -> 21016576 00010000:00010000:3.0:1789760688.925778:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2cd70 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21012480->21016575] (req 21012480->21016575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760688.925792:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2cd70 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21012480->21016575] (req 21012480->21016575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760688.925801:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2cd70 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21012480->21016575] (req 21012480->21016575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760688.925809:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760688.925813:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2cd70 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21012480->21016575] (req 21012480->21016575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760688.925822:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2cd70 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21012480->21016575] (req 21012480->21016575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760688.925842:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2cd70 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21012480->21016575] (req 21012480->21016575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760688.925855:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2cd70 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21012480->21016575] (req 21012480->21016575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760688.926019:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac38b5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512513408:12345-192.168.203.56@tcp:4:dd.0 Request processed in 109295us (109505us total) trans 4294972449 rc 0/0 00000100:00100000:3.0:1789760688.926287:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5160 00000100:00100000:3.0:1789760688.926298:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 668341679026 time 668380938304 state 0x2 ref 0 00000001:00080000:2.0:1789760688.971934:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972450 is committed 00002000:00010000:3.0:1789760688.972058:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21016576 -> 21020672 00010000:00010000:3.0:1789760688.972077:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2cd77 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21016576->21020671] (req 21016576->21020671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760688.972091:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2cd77 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21016576->21020671] (req 21016576->21020671) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760688.972100:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2cd77 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21016576->21020671] (req 21016576->21020671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760688.972107:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760688.972112:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2cd77 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21016576->21020671] (req 21016576->21020671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760688.972133:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2cd77 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21016576->21020671] (req 21016576->21020671) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760688.972160:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2cd77 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21016576->21020671] (req 21016576->21020671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760688.972168:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2cd77 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21016576->21020671] (req 21016576->21020671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760688.972591:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febea772680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512513536:12345-192.168.203.56@tcp:4:dd.0 Request processed in 40687us (41217us total) trans 4294972450 rc 0/0 00000100:00100000:3.0:1789760688.972603:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5161 00000100:00100000:3.0:1789760688.972614:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 668381198720 time 668496186036 state 0x2 ref 0 00000100:00100000:2.0:1789760688.976655:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512513664 00000100:00100000:2.0:1789760688.976665:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512513664 00000100:00100000:2.0:1789760688.978295:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 668381198720 time 668496186036 state 0x0 ref 1 00000100:00100000:2.0:1789760688.978313:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 668496446452 time 668496186036 state 0x1 ref 1 00000100:00100000:2.0:1789760688.978357:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 668496446452 time 668542655920 state 0x0 ref 1 00000100:00100000:2.0:1789760688.978376:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febff929f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512513664:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760688.978428:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21020672, mtime: 0x6aad94b0, atime: 0x0, ctime: 0x6aad94b0, blocks: 0xa807 00010000:00010000:2.0:1789760688.978437:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2cd7e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760688.978452:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=668s v=(0 0) 00010000:00010000:2.0:1789760688.978457:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760688.978464:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2cd7e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21020672->21024767] (req 21020672->21024767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760688.978477:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2cd7e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21020672->21024767] (req 21020672->21024767) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760689.035723:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21020672 -> 21024768 00010000:00010000:2.0:1789760689.035752:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2cd7e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21020672->21024767] (req 21020672->21024767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760689.035766:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2cd7e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21020672->21024767] (req 21020672->21024767) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760689.035776:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2cd7e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21020672->21024767] (req 21020672->21024767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760689.035783:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760689.035789:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2cd7e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21020672->21024767] (req 21020672->21024767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760689.035799:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2cd7e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21020672->21024767] (req 21020672->21024767) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760689.035815:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2cd7e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21020672->21024767] (req 21020672->21024767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760689.035822:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2cd7e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21020672->21024767] (req 21020672->21024767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760689.035909:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febff929f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2361:x1876699512513664:12345-192.168.203.56@tcp:4:dd.0 Request processed in 57536us (59526us total) trans 4294972451 rc 0/0 00000100:00100000:2.0:1789760689.035919:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5162 00000100:00100000:2.0:1789760689.035930:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 668496446452 time 668542655920 state 0x2 ref 0 00000001:00080000:3.0:1789760689.036644:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972451 is committed 00000100:00100000:3.0:1789760689.040306:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512513792 00000100:00100000:3.0:1789760689.040316:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512513792 00000100:00080000:2.0:1789760689.040499:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1789760689.040527:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 668496446452 time 668542655920 state 0x0 ref 1 00000100:00100000:3.0:1789760689.040545:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 668542916336 time 668542655920 state 0x1 ref 1 00000100:00100000:3.0:1789760689.040575:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 668542916336 time 668604874649 state 0x0 ref 1 00000100:00100000:3.0:1789760689.040591:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febea770380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512513792:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760689.040641:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21024768, mtime: 0x6aad94b0, atime: 0x0, ctime: 0x6aad94b0, blocks: 0xa807 00010000:00010000:3.0:1789760689.040655:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2cd85 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760689.040677:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=668s v=(0 0) 00010000:00010000:3.0:1789760689.040682:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760689.040688:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2cd85 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21024768->21028863] (req 21024768->21028863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760689.040851:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2cd85 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21024768->21028863] (req 21024768->21028863) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760689.117105:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972452 is committed 00000100:00100000:3.0:1789760689.125440:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512513920 00000100:00100000:3.0:1789760689.125452:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512513920 00000100:00100000:3.0:1789760689.125514:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 668542916336 time 668604874649 state 0x0 ref 1 00000100:00100000:3.0:1789760689.125531:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 668605135065 time 668604874649 state 0x1 ref 1 00000100:00100000:3.0:1789760689.125569:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 668605135065 time 668689868765 state 0x0 ref 1 00000100:00100000:3.0:1789760689.125582:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febff92b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512513920:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760689.125636:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21028864, mtime: 0x6aad94b0, atime: 0x0, ctime: 0x6aad94b0, blocks: 0xa807 00010000:00010000:3.0:1789760689.125646:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2cd8c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760689.125663:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=668s v=(0 0) 00010000:00010000:3.0:1789760689.125667:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760689.125674:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2cd8c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21028864->21032959] (req 21028864->21032959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760689.125689:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2cd8c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21028864->21032959] (req 21028864->21032959) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760689.201886:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972453 is committed 00002000:00010000:2.0:1789760689.204611:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21028864 -> 21032960 00010000:00010000:2.0:1789760689.204630:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2cd8c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21028864->21032959] (req 21028864->21032959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760689.204643:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2cd8c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21028864->21032959] (req 21028864->21032959) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760689.204651:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2cd8c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21028864->21032959] (req 21028864->21032959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760689.204658:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760689.204663:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2cd8c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21028864->21032959] (req 21028864->21032959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760689.204671:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2cd8c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21028864->21032959] (req 21028864->21032959) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760689.204688:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2cd8c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21028864->21032959] (req 21028864->21032959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760689.204751:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2cd8c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21028864->21032959] (req 21028864->21032959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760689.204842:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febff92b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512513920:12345-192.168.203.56@tcp:4:dd.0 Request processed in 79258us (80261us total) trans 4294972453 rc 0/0 00000100:00100000:2.0:1789760689.204852:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5164 00000100:00100000:2.0:1789760689.204861:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 668605135065 time 668689868765 state 0x2 ref 0 00000100:00100000:2.0:1789760689.209071:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512514048 00000100:00100000:2.0:1789760689.209081:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512514048 00000100:00100000:2.0:1789760689.209200:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 668605135065 time 668689868765 state 0x0 ref 1 00000100:00100000:2.0:1789760689.209282:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 668690129181 time 668689868765 state 0x1 ref 1 00000100:00100000:2.0:1789760689.209347:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 668690129181 time 668773645981 state 0x0 ref 1 00000100:00100000:2.0:1789760689.209361:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febea771c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512514048:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760689.209415:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21032960, mtime: 0x6aad94b0, atime: 0x0, ctime: 0x6aad94b0, blocks: 0xa807 00010000:00010000:2.0:1789760689.209423:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2cd93 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760689.209440:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=668s v=(0 0) 00010000:00010000:2.0:1789760689.209445:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760689.209452:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2cd93 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21032960->21037055] (req 21032960->21037055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760689.209470:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2cd93 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21032960->21037055] (req 21032960->21037055) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760689.268285:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972454 is committed 00002000:00010000:2.0:1789760689.268592:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21032960 -> 21037056 00010000:00010000:2.0:1789760689.268608:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2cd93 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21032960->21037055] (req 21032960->21037055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760689.268621:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2cd93 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21032960->21037055] (req 21032960->21037055) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760689.268631:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2cd93 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21032960->21037055] (req 21032960->21037055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760689.268638:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760689.268643:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2cd93 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21032960->21037055] (req 21032960->21037055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760689.268652:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2cd93 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21032960->21037055] (req 21032960->21037055) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760689.268670:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2cd93 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21032960->21037055] (req 21032960->21037055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760689.268678:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2cd93 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21032960->21037055] (req 21032960->21037055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760689.268809:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febea771c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512514048:12345-192.168.203.56@tcp:4:dd.0 Request processed in 59447us (59841us total) trans 4294972454 rc 0/0 00000100:00100000:2.0:1789760689.268831:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5165 00000100:00100000:2.0:1789760689.268840:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 668690129181 time 668773645981 state 0x2 ref 0 00000100:00100000:3.0:1789760689.273168:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512514176 00000100:00100000:3.0:1789760689.273178:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512514176 00000100:00100000:3.0:1789760689.273242:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 668690129181 time 668773645981 state 0x0 ref 1 00000100:00100000:3.0:1789760689.273258:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 668773906397 time 668773645981 state 0x1 ref 1 00000100:00100000:3.0:1789760689.273290:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 668773906397 time 668837590208 state 0x0 ref 1 00000100:00100000:3.0:1789760689.273304:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4217100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512514176:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760689.273350:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21037056, mtime: 0x6aad94b0, atime: 0x0, ctime: 0x6aad94b0, blocks: 0xa807 00010000:00010000:3.0:1789760689.273359:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2cd9a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760689.273373:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=668s v=(0 0) 00010000:00010000:3.0:1789760689.273379:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760689.273385:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2cd9a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21037056->21041151] (req 21037056->21041151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760689.273560:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2cd9a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21037056->21041151] (req 21037056->21041151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760689.315409:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972455 is committed 00002000:00010000:2.0:1789760689.315776:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21037056 -> 21041152 00010000:00010000:2.0:1789760689.315794:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2cd9a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21037056->21041151] (req 21037056->21041151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760689.315808:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2cd9a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21037056->21041151] (req 21037056->21041151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760689.315820:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2cd9a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21037056->21041151] (req 21037056->21041151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760689.315829:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760689.315835:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2cd9a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21037056->21041151] (req 21037056->21041151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760689.315845:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2cd9a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21037056->21041151] (req 21037056->21041151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760689.315865:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2cd9a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21037056->21041151] (req 21037056->21041151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760689.315880:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2cd9a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21037056->21041151] (req 21037056->21041151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760689.315993:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4217100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512514176:12345-192.168.203.56@tcp:4:dd.0 Request processed in 42687us (42919us total) trans 4294972455 rc 0/0 00000100:00100000:2.0:1789760689.316003:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5166 00000100:00100000:2.0:1789760689.316013:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 668773906397 time 668837590208 state 0x2 ref 0 00000100:00100000:2.0:1789760689.321203:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512514304 00000100:00100000:2.0:1789760689.321222:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512514304 00000100:00100000:2.0:1789760689.321275:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 668773906397 time 668837590208 state 0x0 ref 1 00000100:00100000:2.0:1789760689.321289:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 668837850624 time 668837590208 state 0x1 ref 1 00000100:00100000:2.0:1789760689.321320:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 668837850624 time 668885619153 state 0x0 ref 1 00000100:00100000:2.0:1789760689.321333:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2fbb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512514304:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760689.321382:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21041152, mtime: 0x6aad94b0, atime: 0x0, ctime: 0x6aad94b0, blocks: 0xa807 00010000:00010000:2.0:1789760689.321391:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2cda1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760689.321417:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=668s v=(0 0) 00010000:00010000:2.0:1789760689.321422:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760689.321428:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2cda1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21041152->21045247] (req 21041152->21045247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760689.321448:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2cda1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21041152->21045247] (req 21041152->21045247) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760689.399331:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972456 is committed 00002000:00010000:2.0:1789760689.399455:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21041152 -> 21045248 00010000:00010000:2.0:1789760689.399469:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2cda1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21041152->21045247] (req 21041152->21045247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760689.399481:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2cda1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21041152->21045247] (req 21041152->21045247) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760689.399490:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2cda1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21041152->21045247] (req 21041152->21045247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760689.399497:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760689.399502:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2cda1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21041152->21045247] (req 21041152->21045247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760689.399511:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2cda1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21041152->21045247] (req 21041152->21045247) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760689.399528:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2cda1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21041152->21045247] (req 21041152->21045247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760689.399536:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2cda1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21041152->21045247] (req 21041152->21045247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760689.399631:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2fbb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512514304:12345-192.168.203.56@tcp:4:dd.0 Request processed in 78296us (78525us total) trans 4294972456 rc 0/0 00000100:00100000:2.0:1789760689.399642:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5167 00000100:00100000:2.0:1789760689.399653:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 668837850624 time 668885619153 state 0x2 ref 0 00000100:00100000:2.0:1789760689.403648:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512514432 00000100:00100000:2.0:1789760689.403659:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512514432 00000100:00100000:2.0:1789760689.403750:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 668837850624 time 668885619153 state 0x0 ref 1 00000100:00100000:2.0:1789760689.403766:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 668885879569 time 668885619153 state 0x1 ref 1 00000100:00100000:2.0:1789760689.403792:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 668885879569 time 668968091933 state 0x0 ref 1 00000100:00100000:2.0:1789760689.403805:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac437e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512514432:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760689.403865:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21045248, mtime: 0x6aad94b0, atime: 0x0, ctime: 0x6aad94b0, blocks: 0xa807 00010000:00010000:2.0:1789760689.403874:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2cda8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760689.403890:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=668s v=(0 0) 00010000:00010000:2.0:1789760689.403894:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760689.403900:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2cda8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21045248->21049343] (req 21045248->21049343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760689.403915:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2cda8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21045248->21049343] (req 21045248->21049343) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760689.452411:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21045248 -> 21049344 00010000:00010000:2.0:1789760689.452433:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2cda8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21045248->21049343] (req 21045248->21049343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760689.452447:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2cda8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21045248->21049343] (req 21045248->21049343) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760689.452456:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2cda8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21045248->21049343] (req 21045248->21049343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760689.452464:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760689.452469:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2cda8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21045248->21049343] (req 21045248->21049343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760689.452480:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2cda8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21045248->21049343] (req 21045248->21049343) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760689.452500:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2cda8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21045248->21049343] (req 21045248->21049343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760689.452507:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2cda8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21045248->21049343] (req 21045248->21049343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760689.452613:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac437e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512514432:12345-192.168.203.56@tcp:4:dd.0 Request processed in 48805us (49170us total) trans 4294972457 rc 0/0 00000100:00100000:2.0:1789760689.452627:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5168 00000100:00100000:2.0:1789760689.452637:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 668885879569 time 668968091933 state 0x2 ref 0 00000100:00100000:3.0:1789760689.459946:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760689.460316:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512514560 00000100:00100000:2.0:1789760689.460326:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512514560 00000100:00100000:2.0:1789760689.460385:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 668885879569 time 668968091933 state 0x0 ref 1 00000100:00100000:2.0:1789760689.460410:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 668968352349 time 668968091933 state 0x1 ref 1 00000100:00100000:2.0:1789760689.460442:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 668968352349 time 669024741864 state 0x0 ref 1 00000100:00100000:2.0:1789760689.460455:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac38b5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512514560:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760689.460503:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21049344, mtime: 0x6aad94b0, atime: 0x0, ctime: 0x6aad94b0, blocks: 0xa807 00010000:00010000:2.0:1789760689.460511:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2cdaf lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760689.460528:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=669s v=(0 0) 00010000:00010000:2.0:1789760689.460533:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760689.460539:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2cdaf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21049344->21053439] (req 21049344->21053439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760689.460553:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2cdaf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21049344->21053439] (req 21049344->21053439) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760689.493607:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972458 is committed 00002000:00010000:2.0:1789760689.493748:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21049344 -> 21053440 00010000:00010000:2.0:1789760689.493763:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2cdaf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21049344->21053439] (req 21049344->21053439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760689.493777:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2cdaf lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21049344->21053439] (req 21049344->21053439) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760689.493785:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2cdaf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21049344->21053439] (req 21049344->21053439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760689.493792:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760689.493797:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2cdaf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21049344->21053439] (req 21049344->21053439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760689.493806:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2cdaf lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21049344->21053439] (req 21049344->21053439) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760689.493823:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2cdaf lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21049344->21053439] (req 21049344->21053439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760689.493830:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2cdaf lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21049344->21053439] (req 21049344->21053439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760689.493914:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac38b5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512514560:12345-192.168.203.56@tcp:4:dd.0 Request processed in 33457us (33974us total) trans 4294972458 rc 0/0 00000100:00100000:2.0:1789760689.493924:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5169 00000100:00100000:2.0:1789760689.493933:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 668968352349 time 669024741864 state 0x2 ref 0 00000100:00100000:2.0:1789760689.500778:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512514688 00000100:00100000:2.0:1789760689.500789:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512514688 00000100:00100000:2.0:1789760689.500843:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 668968352349 time 669024741864 state 0x0 ref 1 00000100:00100000:2.0:1789760689.500859:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 669025002280 time 669024741864 state 0x1 ref 1 00000100:00100000:2.0:1789760689.500893:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 669025002280 time 669065192793 state 0x0 ref 1 00000100:00100000:2.0:1789760689.500906:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2fb8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512514688:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760689.500967:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21053440, mtime: 0x6aad94b0, atime: 0x0, ctime: 0x6aad94b0, blocks: 0xa807 00010000:00010000:2.0:1789760689.500976:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2cdb6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760689.500993:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=669s v=(0 0) 00010000:00010000:2.0:1789760689.500997:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760689.501004:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2cdb6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21053440->21057535] (req 21053440->21057535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760689.501018:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2cdb6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21053440->21057535] (req 21053440->21057535) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760689.570640:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21053440 -> 21057536 00002000:00010000:2.0:1789760689.570645:0:6708:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760688 -> 1789760689 00002000:00010000:2.0:1789760689.570648:0:6708:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760688 -> 1789760689 00010000:00010000:2.0:1789760689.570662:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2cdb6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21053440->21057535] (req 21053440->21057535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760689.570676:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2cdb6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21053440->21057535] (req 21053440->21057535) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760689.570685:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2cdb6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21053440->21057535] (req 21053440->21057535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760689.570693:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760689.570719:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2cdb6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21053440->21057535] (req 21053440->21057535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760689.570739:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2cdb6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21053440->21057535] (req 21053440->21057535) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760689.570758:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2cdb6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21053440->21057535] (req 21053440->21057535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760689.570767:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2cdb6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21053440->21057535] (req 21053440->21057535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760689.570878:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2fb8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512514688:12345-192.168.203.56@tcp:4:dd.0 Request processed in 69969us (70516us total) trans 4294972459 rc 0/0 00000100:00100000:2.0:1789760689.570888:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5170 00000100:00100000:2.0:1789760689.570899:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 669025002280 time 669065192793 state 0x2 ref 0 00000100:00100000:2.0:1789760689.575241:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512514816 00000100:00100000:2.0:1789760689.575251:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512514816 00000100:00100000:2.0:1789760689.575305:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 669025002280 time 669065192793 state 0x0 ref 1 00000100:00100000:2.0:1789760689.575321:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 669065453209 time 669065192793 state 0x1 ref 1 00000100:00100000:2.0:1789760689.575358:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 669065453209 time 669139657712 state 0x0 ref 1 00000100:00100000:2.0:1789760689.575371:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac437e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512514816:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760689.575422:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21057536, mtime: 0x6aad94b1, atime: 0x0, ctime: 0x6aad94b1, blocks: 0xa807 00010000:00010000:2.0:1789760689.575432:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2cdbd lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760689.575449:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=669s v=(0 0) 00010000:00010000:2.0:1789760689.575453:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760689.575460:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2cdbd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21057536->21061631] (req 21057536->21061631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760689.575475:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2cdbd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21057536->21061631] (req 21057536->21061631) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760689.630621:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21057536 -> 21061632 00010000:00010000:2.0:1789760689.630643:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2cdbd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21057536->21061631] (req 21057536->21061631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760689.630657:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2cdbd lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21057536->21061631] (req 21057536->21061631) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760689.630666:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2cdbd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21057536->21061631] (req 21057536->21061631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760689.630673:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760689.630679:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2cdbd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21057536->21061631] (req 21057536->21061631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760689.630688:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2cdbd lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21057536->21061631] (req 21057536->21061631) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760689.630740:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2cdbd lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21057536->21061631] (req 21057536->21061631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760689.630749:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2cdbd lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21057536->21061631] (req 21057536->21061631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760689.630995:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac437e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2360:x1876699512514816:12345-192.168.203.56@tcp:4:dd.0 Request processed in 55620us (55949us total) trans 4294972460 rc 0/0 00000100:00100000:2.0:1789760689.631006:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5171 00000100:00100000:2.0:1789760689.631016:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 669065453209 time 669139657712 state 0x2 ref 0 00000100:00100000:2.0:1789760689.634739:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512514944 00000100:00100000:2.0:1789760689.634750:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512514944 00000100:00100000:2.0:1789760689.634811:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 669065453209 time 669139657712 state 0x0 ref 1 00000100:00100000:2.0:1789760689.634826:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 669139918128 time 669139657712 state 0x1 ref 1 00000100:00100000:2.0:1789760689.634847:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 669139918128 time 669199146872 state 0x0 ref 1 00000100:00100000:2.0:1789760689.634860:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac437ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512514944:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760689.634907:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21061632, mtime: 0x6aad94b1, atime: 0x0, ctime: 0x6aad94b1, blocks: 0xa807 00010000:00010000:2.0:1789760689.634918:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2cdc4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760689.634934:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=669s v=(0 0) 00010000:00010000:2.0:1789760689.634938:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760689.634944:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2cdc4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21061632->21065727] (req 21061632->21065727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760689.634958:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2cdc4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21061632->21065727] (req 21061632->21065727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760689.713501:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21061632 -> 21065728 00010000:00010000:3.0:1789760689.713521:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2cdc4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21061632->21065727] (req 21061632->21065727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760689.713536:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2cdc4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21061632->21065727] (req 21061632->21065727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760689.713544:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2cdc4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21061632->21065727] (req 21061632->21065727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760689.713551:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760689.713556:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2cdc4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21061632->21065727] (req 21061632->21065727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760689.713565:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2cdc4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21061632->21065727] (req 21061632->21065727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760689.713582:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2cdc4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21061632->21065727] (req 21061632->21065727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760689.713588:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2cdc4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21061632->21065727] (req 21061632->21065727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760689.713694:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac437ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2360:x1876699512514944:12345-192.168.203.56@tcp:4:dd.0 Request processed in 78831us (79132us total) trans 4294972461 rc 0/0 00000100:00100000:3.0:1789760689.713742:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5172 00000100:00100000:3.0:1789760689.713752:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 669139918128 time 669199146872 state 0x2 ref 0 00000100:00100000:2.0:1789760689.726134:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512515072 00000100:00100000:2.0:1789760689.726143:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512515072 00000100:00100000:2.0:1789760689.726389:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 669139918128 time 669199146872 state 0x0 ref 1 00000100:00100000:2.0:1789760689.726405:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 669199407288 time 669199146872 state 0x1 ref 1 00000100:00100000:2.0:1789760689.726478:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 669199407288 time 669290777032 state 0x0 ref 1 00000100:00100000:2.0:1789760689.726491:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac437df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512515072:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760689.726539:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21065728, mtime: 0x6aad94b1, atime: 0x0, ctime: 0x6aad94b1, blocks: 0xa807 00010000:00010000:2.0:1789760689.726548:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2cdcb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760689.726563:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=669s v=(0 0) 00010000:00010000:2.0:1789760689.726568:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760689.726574:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2cdcb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21065728->21069823] (req 21065728->21069823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760689.726588:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2cdcb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21065728->21069823] (req 21065728->21069823) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760689.773508:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21065728 -> 21069824 00010000:00010000:2.0:1789760689.773532:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2cdcb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21065728->21069823] (req 21065728->21069823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760689.773546:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2cdcb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21065728->21069823] (req 21065728->21069823) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760689.773555:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2cdcb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21065728->21069823] (req 21065728->21069823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760689.773563:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760689.773568:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2cdcb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21065728->21069823] (req 21065728->21069823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760689.773577:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2cdcb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21065728->21069823] (req 21065728->21069823) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760689.773599:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2cdcb lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21065728->21069823] (req 21065728->21069823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760689.773608:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2cdcb lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21065728->21069823] (req 21065728->21069823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760689.773759:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac437df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2360:x1876699512515072:12345-192.168.203.56@tcp:4:dd.0 Request processed in 47264us (47829us total) trans 4294972462 rc 0/0 00000100:00100000:2.0:1789760689.773771:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5173 00000100:00100000:2.0:1789760689.773780:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 669199407288 time 669290777032 state 0x2 ref 0 00000001:00080000:3.0:1789760689.773937:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972462 is committed 00000100:00100000:2.0:1789760689.777452:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00002000:00010000:2.0:1789760689.840934:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21069824 -> 21073920 00010000:00010000:2.0:1789760689.840963:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2cdd2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21069824->21073919] (req 21069824->21073919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760689.840978:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2cdd2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21069824->21073919] (req 21069824->21073919) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760689.840987:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2cdd2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21069824->21073919] (req 21069824->21073919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760689.840994:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760689.840999:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2cdd2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21069824->21073919] (req 21069824->21073919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760689.841009:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2cdd2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21069824->21073919] (req 21069824->21073919) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760689.841027:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2cdd2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21069824->21073919] (req 21069824->21073919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760689.841041:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2cdd2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21069824->21073919] (req 21069824->21073919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760689.841135:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febc7b30000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2359:x1876699512515200:12345-192.168.203.56@tcp:4:dd.0 Request processed in 63308us (63689us total) trans 4294972463 rc 0/0 00000100:00100000:2.0:1789760689.841145:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5174 00000100:00100000:2.0:1789760689.841155:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 669291037448 time 669342110429 state 0x2 ref 0 00000001:00080000:3.0:1789760689.841352:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972463 is committed 00000100:00100000:3.0:1789760689.848022:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512515328 00000100:00100000:3.0:1789760689.848032:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512515328 00000100:00100000:3.0:1789760689.848091:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 669291037448 time 669342110429 state 0x0 ref 1 00000100:00100000:3.0:1789760689.848106:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 669342370845 time 669342110429 state 0x1 ref 1 00000100:00100000:3.0:1789760689.848148:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 669342370845 time 669412446602 state 0x0 ref 1 00000100:00100000:3.0:1789760689.848160:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2fb9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512515328:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760689.848209:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21073920, mtime: 0x6aad94b1, atime: 0x0, ctime: 0x6aad94b1, blocks: 0xa807 00010000:00010000:3.0:1789760689.848219:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2cdd9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760689.848236:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=669s v=(0 0) 00010000:00010000:3.0:1789760689.848240:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760689.848247:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2cdd9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21073920->21078015] (req 21073920->21078015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760689.848267:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2cdd9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21073920->21078015] (req 21073920->21078015) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760689.908587:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21073920 -> 21078016 00010000:00010000:3.0:1789760689.908609:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2cdd9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21073920->21078015] (req 21073920->21078015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760689.908622:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2cdd9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21073920->21078015] (req 21073920->21078015) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760689.908631:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2cdd9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21073920->21078015] (req 21073920->21078015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760689.908638:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760689.908643:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2cdd9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21073920->21078015] (req 21073920->21078015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760689.908651:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2cdd9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21073920->21078015] (req 21073920->21078015) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760689.908670:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2cdd9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21073920->21078015] (req 21073920->21078015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760689.908678:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2cdd9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21073920->21078015] (req 21073920->21078015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760689.909001:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2fb9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2360:x1876699512515328:12345-192.168.203.56@tcp:4:dd.0 Request processed in 60838us (61091us total) trans 4294972464 rc 0/0 00000100:00100000:3.0:1789760689.909012:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5175 00000100:00100000:3.0:1789760689.909023:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 669342370845 time 669412446602 state 0x2 ref 0 00000001:00080000:3.0:1789760689.909061:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972464 is committed 00000100:00100000:2.0:1789760689.912689:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512515456 00000100:00100000:2.0:1789760689.912916:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512515456 00000100:00100000:2.0:1789760689.912967:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 669342370845 time 669412446602 state 0x0 ref 1 00000100:00100000:2.0:1789760689.912982:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 669412707018 time 669412446602 state 0x1 ref 1 00000100:00100000:2.0:1789760689.913056:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 669412707018 time 669477354862 state 0x0 ref 1 00000100:00100000:2.0:1789760689.913068:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac437fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512515456:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760689.913123:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21078016, mtime: 0x6aad94b1, atime: 0x0, ctime: 0x6aad94b1, blocks: 0xa807 00010000:00010000:2.0:1789760689.913131:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2cde0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760689.913147:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=669s v=(0 0) 00010000:00010000:2.0:1789760689.913151:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760689.913158:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2cde0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21078016->21082111] (req 21078016->21082111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760689.913177:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2cde0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21078016->21082111] (req 21078016->21082111) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760689.960223:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512515584 00000100:00100000:2.0:1789760689.960233:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512515584 00000100:00100000:2.0:1789760689.960282:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 669412707018 time 669477354862 state 0x0 ref 1 00000100:00100000:2.0:1789760689.960297:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 669477615278 time 669477354862 state 0x1 ref 1 00000100:00100000:2.0:1789760689.960363:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 669477615278 time 669524661685 state 0x0 ref 1 00000100:00100000:2.0:1789760689.960375:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac437f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512515584:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760689.960506:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21082112, mtime: 0x6aad94b1, atime: 0x0, ctime: 0x6aad94b1, blocks: 0xa807 00010000:00010000:2.0:1789760689.960515:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2cde7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760689.960530:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=669s v=(0 0) 00010000:00010000:2.0:1789760689.960535:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760689.960541:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2cde7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21082112->21086207] (req 21082112->21086207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760689.960556:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2cde7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21082112->21086207] (req 21082112->21086207) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760689.997566:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21082112 -> 21086208 00010000:00010000:2.0:1789760689.997584:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2cde7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21082112->21086207] (req 21082112->21086207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760689.997597:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2cde7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21082112->21086207] (req 21082112->21086207) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760689.997606:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2cde7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21082112->21086207] (req 21082112->21086207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760689.997614:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760689.997620:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2cde7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21082112->21086207] (req 21082112->21086207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760689.997628:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2cde7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21082112->21086207] (req 21082112->21086207) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760689.997647:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2cde7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21082112->21086207] (req 21082112->21086207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760689.997654:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2cde7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21082112->21086207] (req 21082112->21086207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760689.997863:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac437f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512515584:12345-192.168.203.56@tcp:4:dd.0 Request processed in 37485us (37796us total) trans 4294972466 rc 0/0 00000100:00100000:2.0:1789760689.997874:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5177 00000100:00100000:2.0:1789760689.997884:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 669477615278 time 669524661685 state 0x2 ref 0 00000100:00100000:2.0:1789760690.003078:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760690.003198:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512515712 00000100:00100000:3.0:1789760690.003207:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512515712 00000100:00100000:3.0:1789760690.003259:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 669477615278 time 669524661685 state 0x0 ref 1 00000100:00100000:3.0:1789760690.003273:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 669524922101 time 669524661685 state 0x1 ref 1 00000100:00100000:3.0:1789760690.003305:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 669524922101 time 669567604818 state 0x0 ref 1 00000100:00100000:3.0:1789760690.003319:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febc7b33800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512515712:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760690.003366:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21086208, mtime: 0x6aad94b1, atime: 0x0, ctime: 0x6aad94b1, blocks: 0xa807 00010000:00010000:3.0:1789760690.003376:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2cdee lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760690.003403:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=669s v=(0 0) 00010000:00010000:3.0:1789760690.003407:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760690.003413:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2cdee lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21086208->21090303] (req 21086208->21090303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760690.003428:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2cdee lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21086208->21090303] (req 21086208->21090303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760690.039067:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21086208 -> 21090304 00010000:00010000:3.0:1789760690.039089:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2cdee lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21086208->21090303] (req 21086208->21090303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760690.039103:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2cdee lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21086208->21090303] (req 21086208->21090303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760690.039250:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2cdee lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21086208->21090303] (req 21086208->21090303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760690.039259:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760690.039264:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2cdee lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21086208->21090303] (req 21086208->21090303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760690.039273:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2cdee lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21086208->21090303] (req 21086208->21090303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760690.039291:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2cdee lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21086208->21090303] (req 21086208->21090303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760690.039299:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2cdee lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21086208->21090303] (req 21086208->21090303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760690.039523:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febc7b33800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2358:x1876699512515712:12345-192.168.203.56@tcp:4:dd.0 Request processed in 36202us (36449us total) trans 4294972467 rc 0/0 00000100:00100000:3.0:1789760690.039539:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5178 00000100:00100000:3.0:1789760690.039549:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 669524922101 time 669567604818 state 0x2 ref 0 00000100:00100000:2.0:1789760690.044864:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512515840 00000100:00100000:2.0:1789760690.044873:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512515840 00000100:00100000:2.0:1789760690.044929:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 669524922101 time 669567604818 state 0x0 ref 1 00000100:00100000:2.0:1789760690.044946:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 669567865234 time 669567604818 state 0x1 ref 1 00000100:00100000:2.0:1789760690.044977:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 669567865234 time 669609276209 state 0x0 ref 1 00000100:00100000:2.0:1789760690.044989:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac437d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512515840:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760690.045037:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21090304, mtime: 0x6aad94b1, atime: 0x0, ctime: 0x6aad94b1, blocks: 0xa807 00010000:00010000:2.0:1789760690.045046:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2cdf5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760690.045062:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=669s v=(0 0) 00010000:00010000:2.0:1789760690.045067:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760690.045072:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2cdf5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21090304->21094399] (req 21090304->21094399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760690.045086:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2cdf5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21090304->21094399] (req 21090304->21094399) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760690.077041:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972468 is committed 00002000:00010000:2.0:1789760690.077253:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21090304 -> 21094400 00010000:00010000:2.0:1789760690.077270:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2cdf5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21090304->21094399] (req 21090304->21094399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760690.077289:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2cdf5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21090304->21094399] (req 21090304->21094399) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760690.077297:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2cdf5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21090304->21094399] (req 21090304->21094399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760690.077304:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760690.077308:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2cdf5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21090304->21094399] (req 21090304->21094399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760690.077318:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2cdf5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21090304->21094399] (req 21090304->21094399) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760690.077336:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2cdf5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21090304->21094399] (req 21090304->21094399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760690.077343:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2cdf5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21090304->21094399] (req 21090304->21094399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760690.077482:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac437d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512515840:12345-192.168.203.56@tcp:4:dd.0 Request processed in 32490us (32743us total) trans 4294972468 rc 0/0 00000100:00100000:2.0:1789760690.077492:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5179 00000100:00100000:2.0:1789760690.077501:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 669567865234 time 669609276209 state 0x2 ref 0 00000100:00100000:2.0:1789760690.081056:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512515968 00000100:00100000:2.0:1789760690.081068:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512515968 00000100:00100000:2.0:1789760690.081131:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 669567865234 time 669609276209 state 0x0 ref 1 00000100:00100000:2.0:1789760690.081146:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 669609536625 time 669609276209 state 0x1 ref 1 00000100:00100000:2.0:1789760690.081173:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 669609536625 time 669645472290 state 0x0 ref 1 00000100:00100000:2.0:1789760690.081185:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2fbaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512515968:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760690.081238:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21094400, mtime: 0x6aad94b1, atime: 0x0, ctime: 0x6aad94b1, blocks: 0xa807 00010000:00010000:2.0:1789760690.081247:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2cdfc lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760690.081263:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=669s v=(0 0) 00010000:00010000:2.0:1789760690.081268:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760690.081274:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2cdfc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21094400->21098495] (req 21094400->21098495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760690.081287:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2cdfc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21094400->21098495] (req 21094400->21098495) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760690.098961:0:3309:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8feac38b6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3309:1876699540213760:0@lo:400:kworker.0 00000100:00100000:3.0:1789760690.099030:0:3309:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760690.099083:0:3309:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1789760690.099090:0:3309:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8feac38b6300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3309:1876699540213888:0@lo:400:kworker.0 00000100:00100000:3.0:1789760690.099128:0:3309:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760690.099170:0:5848:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540213760 00000100:00100000:3.0:1789760690.099177:0:5848:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540213760 00000100:00100000:3.0:1789760690.099232:0:5848:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 421 00000100:00100000:3.0:1789760690.099241:0:5848:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac38b7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3309:x1876699540213760:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760690.099292:0:5848:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac38b7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3309:x1876699540213760:12345-0@lo:400:kworker.0 Request processed in 52us (262us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760690.099298:0:5848:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 421 00000100:00100000:3.0:1789760690.099319:0:13064:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540213888 00000100:00100000:3.0:1789760690.099322:0:13064:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540213888 00000100:00100000:3.0:1789760690.099348:0:13064:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 162 00000100:00100000:3.0:1789760690.099355:0:13064:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac38b5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3309:x1876699540213888:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760690.099385:0:13064:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac38b5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3309:x1876699540213888:12345-0@lo:400:kworker.0 Request processed in 30us (256us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760690.099403:0:13064:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 162 00000100:00080000:3.0:1789760690.099419:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:3.0:1789760690.099424:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789760690.099456:0:3309:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8feac38b6300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3309:1876699540213888:0@lo:400:kworker.0 00000100:00100000:3.0:1789760690.099466:0:3309:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8feac38b6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3309:1876699540213760:0@lo:400:kworker.0 00000100:00100000:3.0:1789760690.100532:0:8229:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540213632 00000100:00100000:3.0:1789760690.100537:0:8229:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540213632 00000100:00100000:3.0:1789760690.100569:0:8229:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540214016 00000100:00100000:3.0:1789760690.100572:0:8229:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540214016 00000100:00100000:3.0:1789760690.100605:0:8229:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 422 00000100:00100000:3.0:1789760690.100611:0:8229:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febfabf5c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3312:x1876699540213632:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760690.100651:0:8229:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febfabf5c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3312:x1876699540213632:12345-0@lo:400:kworker.0 Request processed in 40us (823us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760690.100656:0:8229:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 422 00000100:00100000:3.0:1789760690.100670:0:8229:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 423 00000100:00100000:3.0:1789760690.100676:0:8229:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febc6bd6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3312:x1876699540214016:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760690.100737:0:8229:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febc6bd6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3312:x1876699540214016:12345-0@lo:400:kworker.0 Request processed in 62us (762us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760690.100743:0:8229:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 423 00000100:00100000:3.0:1789760690.100904:0:3310:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1789760690.100915:0:3310:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8febc6bd6300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3310:1876699540214272:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1789760690.100946:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760690.101322:0:3312:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8feac38b5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3312:1876699540214016:0@lo:400:kworker.0 00000100:00100000:3.0:1789760690.101333:0:3312:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8feac38b4a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3312:1876699540213632:0@lo:400:kworker.0 00000100:00100000:2.0:1789760690.101517:0:10102:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540214144 00000100:00100000:2.0:1789760690.101522:0:10102:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540214144 00000100:00100000:2.0:1789760690.101574:0:10102:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 163 00000100:00100000:2.0:1789760690.101582:0:10102:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febfabf6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3312:x1876699540214144:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760690.101627:0:10102:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febfabf6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3312:x1876699540214144:12345-0@lo:400:kworker.0 Request processed in 46us (1754us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760690.101633:0:10102:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 163 00000100:00100000:3.0:1789760690.102906:0:3312:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8feac38b4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3312:1876699540214144:0@lo:400:kworker.0 00000100:00100000:3.0:1789760690.103443:0:3310:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8febc6bd6300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3310:1876699540214272:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1789760690.127085:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512516096 00000100:00100000:3.0:1789760690.127095:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512516096 00000100:00100000:3.0:1789760690.127159:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 669609536625 time 669645472290 state 0x0 ref 1 00000100:00100000:3.0:1789760690.127177:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 669645732706 time 669645472290 state 0x1 ref 1 00000100:00100000:3.0:1789760690.127208:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 669645732706 time 669691507891 state 0x0 ref 1 00000100:00100000:3.0:1789760690.127222:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2fb8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512516096:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760690.127271:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21098496, mtime: 0x6aad94b1, atime: 0x0, ctime: 0x6aad94b1, blocks: 0xa807 00010000:00010000:3.0:1789760690.127281:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2ce03 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760690.127300:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=669s v=(0 0) 00010000:00010000:3.0:1789760690.127304:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760690.127310:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2ce03 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21098496->21102591] (req 21098496->21102591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760690.127324:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2ce03 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21098496->21102591] (req 21098496->21102591) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760690.160307:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21098496 -> 21102592 00010000:00010000:3.0:1789760690.160326:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2ce03 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21098496->21102591] (req 21098496->21102591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760690.160340:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2ce03 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21098496->21102591] (req 21098496->21102591) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760690.160349:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2ce03 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21098496->21102591] (req 21098496->21102591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760690.160361:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760690.160366:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2ce03 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21098496->21102591] (req 21098496->21102591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760690.160376:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2ce03 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21098496->21102591] (req 21098496->21102591) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760690.160455:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2ce03 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21098496->21102591] (req 21098496->21102591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760690.160465:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2ce03 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21098496->21102591] (req 21098496->21102591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760690.160562:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2fb8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2358:x1876699512516096:12345-192.168.203.56@tcp:4:dd.0 Request processed in 33336us (33756us total) trans 4294972470 rc 0/0 00000100:00100000:3.0:1789760690.160572:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5181 00000100:00100000:3.0:1789760690.160583:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 669645732706 time 669691507891 state 0x2 ref 0 00000100:00100000:3.0:1789760690.165058:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512516224 00000100:00100000:3.0:1789760690.165068:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512516224 00000100:00100000:3.0:1789760690.165187:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 669645732706 time 669691507891 state 0x0 ref 1 00000100:00100000:3.0:1789760690.165206:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 669691768307 time 669691507891 state 0x1 ref 1 00000100:00100000:3.0:1789760690.165237:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 669691768307 time 669729536854 state 0x0 ref 1 00000100:00100000:3.0:1789760690.165254:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2fb9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512516224:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760690.165309:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21102592, mtime: 0x6aad94b1, atime: 0x0, ctime: 0x6aad94b1, blocks: 0xa807 00010000:00010000:3.0:1789760690.165321:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2ce0a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760690.165341:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=669s v=(0 0) 00010000:00010000:3.0:1789760690.165347:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760690.165354:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2ce0a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21102592->21106687] (req 21102592->21106687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760690.165372:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2ce0a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21102592->21106687] (req 21102592->21106687) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760690.197323:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21102592 -> 21106688 00010000:00010000:3.0:1789760690.197342:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2ce0a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21102592->21106687] (req 21102592->21106687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760690.197355:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2ce0a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21102592->21106687] (req 21102592->21106687) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760690.197364:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2ce0a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21102592->21106687] (req 21102592->21106687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760690.197372:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760690.197377:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2ce0a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21102592->21106687] (req 21102592->21106687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760690.197386:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2ce0a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21102592->21106687] (req 21102592->21106687) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760690.197414:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2ce0a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21102592->21106687] (req 21102592->21106687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760690.197423:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2ce0a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21102592->21106687] (req 21102592->21106687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760690.197511:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2fb9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2358:x1876699512516224:12345-192.168.203.56@tcp:4:dd.0 Request processed in 32255us (32811us total) trans 4294972471 rc 0/0 00000100:00100000:3.0:1789760690.197521:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5182 00000100:00100000:3.0:1789760690.197531:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 669691768307 time 669729536854 state 0x2 ref 0 00000100:00100000:3.0:1789760690.200745:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000001:00080000:2.0:1789760690.202102:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972471 is committed 00002000:00010000:2.0:1789760690.236999:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21106688 -> 21110784 00010000:00010000:2.0:1789760690.237023:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2ce11 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21106688->21110783] (req 21106688->21110783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760690.237039:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2ce11 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21106688->21110783] (req 21106688->21110783) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760690.237048:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2ce11 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21106688->21110783] (req 21106688->21110783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760690.237056:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760690.237061:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2ce11 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21106688->21110783] (req 21106688->21110783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760690.237070:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2ce11 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21106688->21110783] (req 21106688->21110783) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760690.237090:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2ce11 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21106688->21110783] (req 21106688->21110783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760690.237098:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2ce11 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21106688->21110783] (req 21106688->21110783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760690.237207:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac38b4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2358:x1876699512516352:12345-192.168.203.56@tcp:4:dd.0 Request processed in 36166us (36509us total) trans 4294972472 rc 0/0 00000100:00100000:2.0:1789760690.237219:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5183 00000100:00100000:2.0:1789760690.237229:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 669729797270 time 669765325029 state 0x2 ref 0 00000001:00080000:3.0:1789760690.238978:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972472 is committed 00000100:00100000:3.0:1789760690.242110:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512516480 00000100:00100000:3.0:1789760690.242130:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512516480 00000100:00100000:3.0:1789760690.242197:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 669729797270 time 669765325029 state 0x0 ref 1 00000100:00100000:3.0:1789760690.242213:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 669765585445 time 669765325029 state 0x1 ref 1 00000100:00100000:3.0:1789760690.242241:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 669765585445 time 669806541177 state 0x0 ref 1 00000100:00100000:3.0:1789760690.242254:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2fb8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512516480:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760690.242304:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21110784, mtime: 0x6aad94b1, atime: 0x0, ctime: 0x6aad94b1, blocks: 0xa807 00010000:00010000:3.0:1789760690.242313:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2ce18 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760690.242329:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=669s v=(0 0) 00010000:00010000:3.0:1789760690.242334:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760690.242341:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2ce18 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21110784->21114879] (req 21110784->21114879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760690.242354:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2ce18 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21110784->21114879] (req 21110784->21114879) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760690.275296:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21110784 -> 21114880 00010000:00010000:3.0:1789760690.275313:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2ce18 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21110784->21114879] (req 21110784->21114879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760690.275327:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2ce18 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21110784->21114879] (req 21110784->21114879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760690.275338:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2ce18 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21110784->21114879] (req 21110784->21114879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760690.275346:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760690.275351:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2ce18 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21110784->21114879] (req 21110784->21114879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760690.275360:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2ce18 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21110784->21114879] (req 21110784->21114879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760690.275381:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2ce18 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21110784->21114879] (req 21110784->21114879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760690.275461:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2ce18 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21110784->21114879] (req 21110784->21114879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760690.275595:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2fb8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512516480:12345-192.168.203.56@tcp:4:dd.0 Request processed in 33336us (33680us total) trans 4294972473 rc 0/0 00000100:00100000:3.0:1789760690.275607:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5184 00000100:00100000:3.0:1789760690.275617:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 669765585445 time 669806541177 state 0x2 ref 0 00000100:00100000:3.0:1789760690.278758:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512516608 00000100:00100000:3.0:1789760690.278769:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512516608 00000100:00100000:3.0:1789760690.278823:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 669765585445 time 669806541177 state 0x0 ref 1 00000100:00100000:3.0:1789760690.278839:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 669806801593 time 669806541177 state 0x1 ref 1 00000100:00100000:3.0:1789760690.278871:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 669806801593 time 669843171342 state 0x0 ref 1 00000100:00100000:3.0:1789760690.278885:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2fba680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512516608:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760690.278935:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21114880, mtime: 0x6aad94b1, atime: 0x0, ctime: 0x6aad94b1, blocks: 0xa807 00010000:00010000:3.0:1789760690.278944:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2ce1f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760690.278961:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=669s v=(0 0) 00010000:00010000:3.0:1789760690.278967:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760690.278973:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2ce1f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21114880->21118975] (req 21114880->21118975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760690.278987:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2ce1f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21114880->21118975] (req 21114880->21118975) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760690.331812:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21114880 -> 21118976 00010000:00010000:3.0:1789760690.331829:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2ce1f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21114880->21118975] (req 21114880->21118975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760690.331840:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2ce1f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21114880->21118975] (req 21114880->21118975) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760690.331850:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2ce1f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21114880->21118975] (req 21114880->21118975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760690.331857:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760690.331861:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2ce1f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21114880->21118975] (req 21114880->21118975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760690.331871:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2ce1f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21114880->21118975] (req 21114880->21118975) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760690.331889:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2ce1f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21114880->21118975] (req 21114880->21118975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760690.331896:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2ce1f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21114880->21118975] (req 21114880->21118975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760690.332105:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2fba680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512516608:12345-192.168.203.56@tcp:4:dd.0 Request processed in 53218us (53459us total) trans 4294972474 rc 0/0 00000100:00100000:3.0:1789760690.332115:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5185 00000100:00100000:3.0:1789760690.332124:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 669806801593 time 669843171342 state 0x2 ref 0 00000100:00100000:2.0:1789760690.336194:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512516736 00000100:00100000:2.0:1789760690.336213:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512516736 00000100:00100000:2.0:1789760690.336274:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 669806801593 time 669843171342 state 0x0 ref 1 00000100:00100000:2.0:1789760690.336291:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 669843431758 time 669843171342 state 0x1 ref 1 00000100:00100000:2.0:1789760690.336323:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 669843431758 time 669900622634 state 0x0 ref 1 00000100:00100000:2.0:1789760690.336336:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac38b2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512516736:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760690.336401:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21118976, mtime: 0x6aad94b1, atime: 0x0, ctime: 0x6aad94b1, blocks: 0xa807 00010000:00010000:2.0:1789760690.336411:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2ce26 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760690.336427:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=669s v=(0 0) 00010000:00010000:2.0:1789760690.336432:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760690.336437:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2ce26 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21118976->21123071] (req 21118976->21123071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760690.336460:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2ce26 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21118976->21123071] (req 21118976->21123071) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760690.399921:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512516864 00000100:00100000:3.0:1789760690.399930:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512516864 00000100:00100000:3.0:1789760690.399992:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 669843431758 time 669900622634 state 0x0 ref 1 00000100:00100000:3.0:1789760690.400006:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 669900883050 time 669900622634 state 0x1 ref 1 00000100:00100000:3.0:1789760690.400034:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 669900883050 time 669964334017 state 0x0 ref 1 00000100:00100000:3.0:1789760690.400046:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac38b3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512516864:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760690.400097:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21123072, mtime: 0x6aad94b1, atime: 0x0, ctime: 0x6aad94b1, blocks: 0xa807 00010000:00010000:3.0:1789760690.400106:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2ce2d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760690.400122:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=669s v=(0 0) 00010000:00010000:3.0:1789760690.400126:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760690.400132:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2ce2d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21123072->21127167] (req 21123072->21127167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760690.400499:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2ce2d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21123072->21127167] (req 21123072->21127167) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760690.474635:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972476 is committed 00002000:00010000:2.0:1789760690.476973:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21123072 -> 21127168 00010000:00010000:2.0:1789760690.476993:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2ce2d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21123072->21127167] (req 21123072->21127167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760690.477011:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2ce2d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21123072->21127167] (req 21123072->21127167) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760690.477019:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2ce2d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21123072->21127167] (req 21123072->21127167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760690.477035:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760690.477039:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2ce2d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21123072->21127167] (req 21123072->21127167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760690.477048:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2ce2d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21123072->21127167] (req 21123072->21127167) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760690.477069:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2ce2d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21123072->21127167] (req 21123072->21127167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760690.477075:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2ce2d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21123072->21127167] (req 21123072->21127167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760690.477176:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac38b3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512516864:12345-192.168.203.56@tcp:4:dd.0 Request processed in 77128us (77483us total) trans 4294972476 rc 0/0 00000100:00100000:2.0:1789760690.477186:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5187 00000100:00100000:2.0:1789760690.477196:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 669900883050 time 669964334017 state 0x2 ref 0 00000100:00100000:3.0:1789760690.482303:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512516992 00000100:00100000:3.0:1789760690.482312:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512516992 00000100:00100000:3.0:1789760690.482364:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 669900883050 time 669964334017 state 0x0 ref 1 00000100:00100000:3.0:1789760690.482380:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 669964594433 time 669964334017 state 0x1 ref 1 00000100:00100000:3.0:1789760690.482412:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 669964594433 time 670046711945 state 0x0 ref 1 00000100:00100000:3.0:1789760690.482426:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febc6bd6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512516992:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760690.482473:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21127168, mtime: 0x6aad94b1, atime: 0x0, ctime: 0x6aad94b1, blocks: 0xa807 00010000:00010000:3.0:1789760690.482482:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2ce34 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760690.482497:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=670s v=(0 0) 00010000:00010000:3.0:1789760690.482502:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760690.482508:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2ce34 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21127168->21131263] (req 21127168->21131263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760690.482522:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2ce34 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21127168->21131263] (req 21127168->21131263) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760690.525465:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972477 is committed 00002000:00010000:3.0:1789760690.525547:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21127168 -> 21131264 00010000:00010000:3.0:1789760690.525563:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2ce34 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21127168->21131263] (req 21127168->21131263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760690.525582:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2ce34 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21127168->21131263] (req 21127168->21131263) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760690.525592:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2ce34 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21127168->21131263] (req 21127168->21131263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760690.525599:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760690.525604:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2ce34 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21127168->21131263] (req 21127168->21131263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760690.525613:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2ce34 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21127168->21131263] (req 21127168->21131263) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760690.525633:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2ce34 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21127168->21131263] (req 21127168->21131263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760690.525642:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2ce34 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21127168->21131263] (req 21127168->21131263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760690.525781:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febc6bd6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512516992:12345-192.168.203.56@tcp:4:dd.0 Request processed in 43353us (43668us total) trans 4294972477 rc 0/0 00000100:00100000:3.0:1789760690.525791:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5188 00000100:00100000:3.0:1789760690.525801:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 669964594433 time 670046711945 state 0x2 ref 0 00000100:00100000:2.0:1789760690.532742:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512517120 00000100:00100000:2.0:1789760690.532752:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512517120 00000100:00100000:2.0:1789760690.532803:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 669964594433 time 670046711945 state 0x0 ref 1 00000100:00100000:2.0:1789760690.532819:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 670046972361 time 670046711945 state 0x1 ref 1 00000100:00100000:2.0:1789760690.532849:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 670046972361 time 670097149033 state 0x0 ref 1 00000100:00100000:2.0:1789760690.532860:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac38b3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512517120:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760690.532906:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21131264, mtime: 0x6aad94b1, atime: 0x0, ctime: 0x6aad94b1, blocks: 0xa807 00010000:00010000:2.0:1789760690.532916:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2ce3b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760690.532930:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=670s v=(0 0) 00010000:00010000:2.0:1789760690.532935:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760690.532941:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2ce3b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21131264->21135359] (req 21131264->21135359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760690.532953:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2ce3b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21131264->21135359] (req 21131264->21135359) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760690.590198:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972478 is committed 00000100:00100000:2.0:1789760690.599588:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512517248 00000100:00100000:2.0:1789760690.599601:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512517248 00000100:00100000:2.0:1789760690.599663:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 670046972361 time 670097149033 state 0x0 ref 1 00000100:00100000:2.0:1789760690.599684:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 670097409449 time 670097149033 state 0x1 ref 1 00000100:00100000:2.0:1789760690.599743:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 670097409449 time 670164042296 state 0x0 ref 1 00000100:00100000:2.0:1789760690.599758:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4074700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512517248:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760690.599809:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21135360, mtime: 0x6aad94b2, atime: 0x0, ctime: 0x6aad94b2, blocks: 0xa807 00010000:00010000:2.0:1789760690.599818:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2ce42 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760690.599832:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=670s v=(0 0) 00010000:00010000:2.0:1789760690.599836:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760690.599842:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2ce42 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21135360->21139455] (req 21135360->21139455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760690.599856:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2ce42 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21135360->21139455] (req 21135360->21139455) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760690.612084:0:3310:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8feac4074a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3310:1876699540214400:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1789760690.612150:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00002000:00010000:3.0:1789760690.646966:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21135360 -> 21139456 00010000:00010000:3.0:1789760690.646984:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2ce42 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21135360->21139455] (req 21135360->21139455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760690.646998:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2ce42 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21135360->21139455] (req 21135360->21139455) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760690.647006:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2ce42 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21135360->21139455] (req 21135360->21139455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760690.647013:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760690.647018:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2ce42 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21135360->21139455] (req 21135360->21139455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760690.647027:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2ce42 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21135360->21139455] (req 21135360->21139455) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760690.647047:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2ce42 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21135360->21139455] (req 21135360->21139455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760690.647055:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2ce42 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21135360->21139455] (req 21135360->21139455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760690.647305:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4074700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512517248:12345-192.168.203.56@tcp:4:dd.0 Request processed in 47546us (49346us total) trans 4294972479 rc 0/0 00000100:00100000:3.0:1789760690.647316:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5190 00000100:00100000:3.0:1789760690.647327:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 670097409449 time 670164042296 state 0x2 ref 0 00000100:00100000:3.0:1789760690.650273:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512517376 00000100:00100000:3.0:1789760690.650283:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512517376 00000100:00100000:3.0:1789760690.650339:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 670097409449 time 670164042296 state 0x0 ref 1 00000100:00100000:3.0:1789760690.650355:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 670164302712 time 670164042296 state 0x1 ref 1 00000100:00100000:3.0:1789760690.650385:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 670164302712 time 670214684925 state 0x0 ref 1 00000100:00100000:3.0:1789760690.650507:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4074a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512517376:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760690.650554:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21139456, mtime: 0x6aad94b2, atime: 0x0, ctime: 0x6aad94b2, blocks: 0xa807 00010000:00010000:3.0:1789760690.650562:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2ce49 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760690.650577:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=670s v=(0 0) 00010000:00010000:3.0:1789760690.650582:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760690.650589:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2ce49 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21139456->21143551] (req 21139456->21143551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760690.650603:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2ce49 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21139456->21143551] (req 21139456->21143551) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760690.673954:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972480 is committed 00000100:00100000:2.0:1789760690.676894:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512517504 00000100:00100000:2.0:1789760690.676905:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512517504 00000100:00100000:2.0:1789760690.676954:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 670164302712 time 670214684925 state 0x0 ref 1 00000100:00100000:2.0:1789760690.676970:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 670214945341 time 670214684925 state 0x1 ref 1 00000100:00100000:2.0:1789760690.677031:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 670214945341 time 670241326301 state 0x0 ref 1 00000100:00100000:2.0:1789760690.677043:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac38b0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512517504:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760690.677092:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21143552, mtime: 0x6aad94b2, atime: 0x0, ctime: 0x6aad94b2, blocks: 0xa807 00010000:00010000:2.0:1789760690.677100:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2ce50 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760690.677114:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=670s v=(0 0) 00010000:00010000:2.0:1789760690.677118:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760690.677124:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2ce50 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21143552->21147647] (req 21143552->21147647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760690.677139:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2ce50 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21143552->21147647] (req 21143552->21147647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760690.708624:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512517632 00000100:00100000:3.0:1789760690.708634:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512517632 00000100:00100000:3.0:1789760690.708689:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 670214945341 time 670241326301 state 0x0 ref 1 00000100:00100000:3.0:1789760690.708835:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 670241586717 time 670241326301 state 0x1 ref 1 00000100:00100000:3.0:1789760690.708895:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 670241586717 time 670273193659 state 0x0 ref 1 00000100:00100000:3.0:1789760690.708909:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febfbeae300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512517632:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760690.708959:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21147648, mtime: 0x6aad94b2, atime: 0x0, ctime: 0x6aad94b2, blocks: 0xa807 00010000:00010000:3.0:1789760690.708968:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2ce57 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760690.708984:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=670s v=(0 0) 00010000:00010000:3.0:1789760690.708989:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760690.708996:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2ce57 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21147648->21151743] (req 21147648->21151743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760690.709010:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2ce57 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21147648->21151743] (req 21147648->21151743) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760690.767680:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972483 is committed 00000100:00100000:3.0:1789760690.770991:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000001:00080000:3.0:1789760690.795485:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972484 is committed 00002000:00010000:2.0:1789760690.795611:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21155840 -> 21159936 00010000:00010000:2.0:1789760690.795628:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2ce65 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21155840->21159935] (req 21155840->21159935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760690.795641:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2ce65 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21155840->21159935] (req 21155840->21159935) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760690.795650:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2ce65 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21155840->21159935] (req 21155840->21159935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760690.795658:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760690.795663:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2ce65 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21155840->21159935] (req 21155840->21159935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760690.795672:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2ce65 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21155840->21159935] (req 21155840->21159935) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760690.795693:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2ce65 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21155840->21159935] (req 21155840->21159935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760690.795734:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2ce65 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21155840->21159935] (req 21155840->21159935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760690.795866:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac38b5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512517888:12345-192.168.203.56@tcp:4:dd.0 Request processed in 24630us (24882us total) trans 4294972484 rc 0/0 00000100:00100000:2.0:1789760690.795877:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5195 00000100:00100000:2.0:1789760690.795886:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 670301138151 time 670335521097 state 0x2 ref 0 00000100:00100000:2.0:1789760690.799871:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00002000:00010000:3.0:1789760690.844958:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21159936 -> 21164032 00010000:00010000:3.0:1789760690.844988:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2ce6c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21159936->21164031] (req 21159936->21164031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760690.845000:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2ce6c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21159936->21164031] (req 21159936->21164031) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760690.845009:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2ce6c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21159936->21164031] (req 21159936->21164031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760690.845017:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760690.845022:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2ce6c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21159936->21164031] (req 21159936->21164031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760690.845031:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2ce6c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21159936->21164031] (req 21159936->21164031) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760690.845050:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2ce6c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21159936->21164031] (req 21159936->21164031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760690.845058:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2ce6c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21159936->21164031] (req 21159936->21164031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760690.845159:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febc7b32680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512518016:12345-192.168.203.56@tcp:4:dd.0 Request processed in 45003us (45292us total) trans 4294972485 rc 0/0 00000100:00100000:3.0:1789760690.845169:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5196 00000100:00100000:3.0:1789760690.845179:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 670335781513 time 670364439451 state 0x2 ref 0 00000100:00100000:3.0:1789760690.848602:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00002000:00010000:3.0:1789760690.883869:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21164032 -> 21168128 00010000:00010000:3.0:1789760690.883884:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2ce73 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21164032->21168127] (req 21164032->21168127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760690.883897:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2ce73 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21164032->21168127] (req 21164032->21168127) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760690.883912:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2ce73 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21164032->21168127] (req 21164032->21168127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760690.883920:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760690.883925:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2ce73 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21164032->21168127] (req 21164032->21168127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760690.883934:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2ce73 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21164032->21168127] (req 21164032->21168127) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760690.883952:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2ce73 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21164032->21168127] (req 21164032->21168127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760690.883960:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2ce73 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21164032->21168127] (req 21164032->21168127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760690.884048:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac38b6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512518144:12345-192.168.203.56@tcp:4:dd.0 Request processed in 35172us (35450us total) trans 4294972486 rc 0/0 00000100:00100000:3.0:1789760690.884057:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5197 00000100:00100000:3.0:1789760690.884066:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 670364699867 time 670413161087 state 0x2 ref 0 00000100:00100000:2.0:1789760690.888067:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512518272 00000100:00100000:2.0:1789760690.888078:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512518272 00000100:00100000:2.0:1789760690.888128:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 670364699867 time 670413161087 state 0x0 ref 1 00000100:00100000:2.0:1789760690.888144:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 670413421503 time 670413161087 state 0x1 ref 1 00000100:00100000:2.0:1789760690.888174:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 670413421503 time 670452473216 state 0x0 ref 1 00000100:00100000:2.0:1789760690.888187:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac38b3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512518272:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760690.888235:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21168128, mtime: 0x6aad94b2, atime: 0x0, ctime: 0x6aad94b2, blocks: 0xa807 00010000:00010000:2.0:1789760690.888244:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2ce7a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760690.888266:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=670s v=(0 0) 00010000:00010000:2.0:1789760690.888271:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760690.888277:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2ce7a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21168128->21172223] (req 21168128->21172223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760690.888291:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2ce7a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21168128->21172223] (req 21168128->21172223) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760690.946002:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21168128 -> 21172224 00010000:00010000:2.0:1789760690.946018:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2ce7a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21168128->21172223] (req 21168128->21172223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760690.946092:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2ce7a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21168128->21172223] (req 21168128->21172223) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760690.946103:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2ce7a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21168128->21172223] (req 21168128->21172223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760690.946111:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760690.946116:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2ce7a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21168128->21172223] (req 21168128->21172223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760690.946126:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2ce7a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21168128->21172223] (req 21168128->21172223) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760690.946150:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2ce7a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21168128->21172223] (req 21168128->21172223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760690.946159:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2ce7a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21168128->21172223] (req 21168128->21172223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760690.946287:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac38b3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512518272:12345-192.168.203.56@tcp:4:dd.0 Request processed in 58098us (58371us total) trans 4294972487 rc 0/0 00000100:00100000:2.0:1789760690.946298:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5198 00000100:00100000:2.0:1789760690.946307:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 670413421503 time 670452473216 state 0x2 ref 0 00000100:00100000:3.0:1789760690.955262:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512518400 00000100:00100000:3.0:1789760690.955276:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512518400 00000100:00100000:3.0:1789760690.955333:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 670413421503 time 670452473216 state 0x0 ref 1 00000100:00100000:3.0:1789760690.955347:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 670452733632 time 670452473216 state 0x1 ref 1 00000100:00100000:3.0:1789760690.955398:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 670452733632 time 670519697299 state 0x0 ref 1 00000100:00100000:3.0:1789760690.955411:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febd1362300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512518400:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760690.955457:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21172224, mtime: 0x6aad94b2, atime: 0x0, ctime: 0x6aad94b2, blocks: 0xa807 00010000:00010000:3.0:1789760690.955465:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2ce81 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760690.955479:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=670s v=(0 0) 00010000:00010000:3.0:1789760690.955484:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760690.955490:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2ce81 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21172224->21176319] (req 21172224->21176319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760690.955504:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2ce81 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21172224->21176319] (req 21172224->21176319) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760691.043016:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21172224 -> 21176320 00010000:00010000:2.0:1789760691.043038:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2ce81 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21172224->21176319] (req 21172224->21176319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760691.043052:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2ce81 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21172224->21176319] (req 21172224->21176319) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760691.043061:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2ce81 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21172224->21176319] (req 21172224->21176319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760691.043067:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760691.043072:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2ce81 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21172224->21176319] (req 21172224->21176319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760691.043080:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2ce81 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21172224->21176319] (req 21172224->21176319) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760691.043102:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2ce81 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21172224->21176319] (req 21172224->21176319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760691.043111:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2ce81 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21172224->21176319] (req 21172224->21176319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760691.043203:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febd1362300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512518400:12345-192.168.203.56@tcp:4:dd.0 Request processed in 87790us (89243us total) trans 4294972488 rc 0/0 00000100:00100000:2.0:1789760691.043213:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5199 00000100:00100000:2.0:1789760691.043223:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 670452733632 time 670519697299 state 0x2 ref 0 00000100:00100000:3.0:1789760691.048225:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00080000:3.0:1789760691.052626:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00010000:3.0:1789760691.097334:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21176320 -> 21180416 00010000:00010000:3.0:1789760691.097350:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2ce88 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21176320->21180415] (req 21176320->21180415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760691.097363:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2ce88 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21176320->21180415] (req 21176320->21180415) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760691.097475:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2ce88 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21176320->21180415] (req 21176320->21180415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760691.097489:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760691.097494:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2ce88 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21176320->21180415] (req 21176320->21180415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760691.097503:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2ce88 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21176320->21180415] (req 21176320->21180415) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760691.097524:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2ce88 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21176320->21180415] (req 21176320->21180415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760691.097531:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2ce88 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21176320->21180415] (req 21176320->21180415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760691.097649:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac38b4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512518528:12345-192.168.203.56@tcp:4:dd.0 Request processed in 46001us (49429us total) trans 4294972489 rc 0/0 00000100:00100000:3.0:1789760691.097659:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5200 00000100:00100000:3.0:1789760691.097669:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 670519957715 time 670615933389 state 0x2 ref 0 00000100:00100000:2.0:1789760691.107994:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512518656 00000100:00100000:2.0:1789760691.108005:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512518656 00000100:00100000:2.0:1789760691.108060:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 670519957715 time 670615933389 state 0x0 ref 1 00000100:00100000:2.0:1789760691.108211:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 670616193805 time 670615933389 state 0x1 ref 1 00000100:00100000:2.0:1789760691.108253:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 670616193805 time 670672552883 state 0x0 ref 1 00000100:00100000:2.0:1789760691.108266:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac38b2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512518656:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760691.108316:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21180416, mtime: 0x6aad94b2, atime: 0x0, ctime: 0x6aad94b2, blocks: 0xa807 00010000:00010000:2.0:1789760691.108325:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2ce8f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760691.108341:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=670s v=(0 0) 00010000:00010000:2.0:1789760691.108347:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760691.108353:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2ce8f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21180416->21184511] (req 21180416->21184511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760691.108366:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2ce8f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21180416->21184511] (req 21180416->21184511) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760691.150539:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21180416 -> 21184512 00010000:00010000:3.0:1789760691.150556:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2ce8f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21180416->21184511] (req 21180416->21184511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760691.150571:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2ce8f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21180416->21184511] (req 21180416->21184511) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760691.150588:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2ce8f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21180416->21184511] (req 21180416->21184511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760691.150596:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760691.150602:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2ce8f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21180416->21184511] (req 21180416->21184511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760691.150612:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2ce8f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21180416->21184511] (req 21180416->21184511) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760691.150632:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2ce8f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21180416->21184511] (req 21180416->21184511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760691.150641:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2ce8f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21180416->21184511] (req 21180416->21184511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760691.150816:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac38b2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512518656:12345-192.168.203.56@tcp:4:dd.0 Request processed in 42547us (43433us total) trans 4294972490 rc 0/0 00000100:00100000:3.0:1789760691.150827:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5201 00000100:00100000:3.0:1789760691.150838:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 670616193805 time 670672552883 state 0x2 ref 0 00000100:00100000:2.0:1789760691.153815:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760691.153911:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512518784 00000100:00100000:3.0:1789760691.153921:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512518784 00000100:00100000:3.0:1789760691.153979:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 670616193805 time 670672552883 state 0x0 ref 1 00000100:00100000:3.0:1789760691.153994:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 670672813299 time 670672552883 state 0x1 ref 1 00000100:00100000:3.0:1789760691.154015:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 670672813299 time 670718314921 state 0x0 ref 1 00000100:00100000:3.0:1789760691.154027:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febfb377100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512518784:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760691.154070:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21184512, mtime: 0x6aad94b2, atime: 0x0, ctime: 0x6aad94b2, blocks: 0xa807 00010000:00010000:3.0:1789760691.154079:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2ce96 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760691.154095:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=670s v=(0 0) 00010000:00010000:3.0:1789760691.154099:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760691.154117:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2ce96 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21184512->21188607] (req 21184512->21188607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760691.154132:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2ce96 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21184512->21188607] (req 21184512->21188607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760691.206991:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512518912 00000100:00100000:3.0:1789760691.207001:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512518912 00000100:00100000:3.0:1789760691.207052:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 670672813299 time 670718314921 state 0x0 ref 1 00000100:00100000:3.0:1789760691.207067:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 670718575337 time 670718314921 state 0x1 ref 1 00000100:00100000:3.0:1789760691.207097:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 670718575337 time 670771396663 state 0x0 ref 1 00000100:00100000:3.0:1789760691.207119:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febea651500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512518912:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760691.207165:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21188608, mtime: 0x6aad94b2, atime: 0x0, ctime: 0x6aad94b2, blocks: 0xa807 00010000:00010000:3.0:1789760691.207174:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2ce9d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760691.207190:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=670s v=(0 0) 00010000:00010000:3.0:1789760691.207316:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760691.207323:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2ce9d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21188608->21192703] (req 21188608->21192703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760691.207337:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2ce9d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21188608->21192703] (req 21188608->21192703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760691.280809:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972492 is committed 00000100:00100000:2.0:1789760691.285607:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760691.285729:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512519040 00000100:00100000:3.0:1789760691.285740:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512519040 00000100:00100000:3.0:1789760691.285795:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 670718575337 time 670771396663 state 0x0 ref 1 00000100:00100000:3.0:1789760691.285811:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 670771657079 time 670771396663 state 0x1 ref 1 00000100:00100000:3.0:1789760691.285833:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 670771657079 time 670850132514 state 0x0 ref 1 00000100:00100000:3.0:1789760691.285846:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febfb374380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512519040:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760691.285893:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21192704, mtime: 0x6aad94b2, atime: 0x0, ctime: 0x6aad94b2, blocks: 0xa807 00010000:00010000:3.0:1789760691.285901:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2cea4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760691.285919:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=670s v=(0 0) 00010000:00010000:3.0:1789760691.285924:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760691.285930:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2cea4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21192704->21196799] (req 21192704->21196799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760691.285944:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2cea4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21192704->21196799] (req 21192704->21196799) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760691.333167:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972493 is committed 00000100:00100000:2.0:1789760691.345056:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000001:00080000:2.0:1789760691.369520:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972494 is committed 00000001:00080000:3.0:1789760691.400781:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972495 is committed 00000100:00100000:2.0:1789760691.404913:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760691.441909:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512519552 00000100:00100000:3.0:1789760691.441918:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512519552 00000100:00100000:3.0:1789760691.441975:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 670941922705 time 670969385883 state 0x0 ref 1 00000100:00100000:3.0:1789760691.441990:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 670969646299 time 670969385883 state 0x1 ref 1 00000100:00100000:3.0:1789760691.442060:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 670969646299 time 671006359570 state 0x0 ref 1 00000100:00100000:3.0:1789760691.442211:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac38b1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512519552:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760691.442259:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21209088, mtime: 0x6aad94b2, atime: 0x0, ctime: 0x6aad94b2, blocks: 0xa807 00010000:00010000:3.0:1789760691.442268:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2cec0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760691.442284:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=671s v=(0 0) 00010000:00010000:3.0:1789760691.442289:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760691.442295:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2cec0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21209088->21213183] (req 21209088->21213183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760691.442311:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2cec0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21209088->21213183] (req 21209088->21213183) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760691.470485:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972497 is committed 00000100:00100000:3.0:1789760691.473879:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00002000:00010000:2.0:1789760691.500651:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21213184 -> 21217280 00010000:00010000:2.0:1789760691.500665:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2cec7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21213184->21217279] (req 21213184->21217279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760691.500674:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2cec7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21213184->21217279] (req 21213184->21217279) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760691.500681:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2cec7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21213184->21217279] (req 21213184->21217279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760691.500685:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760691.500689:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2cec7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21213184->21217279] (req 21213184->21217279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760691.500753:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2cec7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21213184->21217279] (req 21213184->21217279) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760691.500773:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2cec7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21213184->21217279] (req 21213184->21217279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760691.500782:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2cec7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21213184->21217279] (req 21213184->21217279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760691.500876:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac38b6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512519680:12345-192.168.203.56@tcp:4:dd.0 Request processed in 26649us (27000us total) trans 4294972498 rc 0/0 00000100:00100000:2.0:1789760691.500886:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5209 00000100:00100000:2.0:1789760691.500896:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 671006619986 time 671038509763 state 0x2 ref 0 00000100:00100000:2.0:1789760691.503192:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760691.542363:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512519936 00000100:00100000:3.0:1789760691.542374:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512519936 00000100:00100000:3.0:1789760691.542436:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 671038770179 time 671067634930 state 0x0 ref 1 00000100:00100000:3.0:1789760691.542451:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 671067895346 time 671067634930 state 0x1 ref 1 00000100:00100000:3.0:1789760691.542481:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 671067895346 time 671106780904 state 0x0 ref 1 00000100:00100000:3.0:1789760691.542494:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac38b1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512519936:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760691.542542:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21221376, mtime: 0x6aad94b3, atime: 0x0, ctime: 0x6aad94b3, blocks: 0xa807 00010000:00010000:3.0:1789760691.542552:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2ced5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760691.542567:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=671s v=(0 0) 00010000:00010000:3.0:1789760691.542572:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760691.542578:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2ced5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21221376->21225471] (req 21221376->21225471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760691.542592:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2ced5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21221376->21225471] (req 21221376->21225471) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760691.603278:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 671067895346 time 671106780904 state 0x2 ref 0 00000100:00100000:3.0:1789760691.609880:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760691.610880:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512520064 00000100:00100000:2.0:1789760691.610892:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512520064 00000100:00100000:2.0:1789760691.610946:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 671067895346 time 671106780904 state 0x0 ref 1 00000100:00100000:2.0:1789760691.610962:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 671107041320 time 671106780904 state 0x1 ref 1 00000100:00100000:2.0:1789760691.610998:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 671107041320 time 671175297852 state 0x0 ref 1 00000100:00100000:2.0:1789760691.611011:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac38b5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512520064:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760691.611069:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21225472, mtime: 0x6aad94b3, atime: 0x0, ctime: 0x6aad94b3, blocks: 0xa807 00010000:00010000:2.0:1789760691.611079:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2cedc lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760691.611093:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=671s v=(0 0) 00010000:00010000:2.0:1789760691.611097:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760691.611104:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2cedc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21225472->21229567] (req 21225472->21229567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760691.611117:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2cedc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21225472->21229567] (req 21225472->21229567) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760691.649849:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972501 is committed 00002000:00010000:3.0:1789760691.650337:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21225472 -> 21229568 00010000:00010000:3.0:1789760691.650353:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2cedc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21225472->21229567] (req 21225472->21229567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760691.650514:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2cedc lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21225472->21229567] (req 21225472->21229567) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760691.650526:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2cedc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21225472->21229567] (req 21225472->21229567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760691.650532:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760691.650536:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2cedc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21225472->21229567] (req 21225472->21229567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760691.650543:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2cedc lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21225472->21229567] (req 21225472->21229567) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760691.650558:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2cedc lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21225472->21229567] (req 21225472->21229567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760691.650562:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2cedc lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21225472->21229567] (req 21225472->21229567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760691.650646:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac38b5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512520064:12345-192.168.203.56@tcp:4:dd.0 Request processed in 39633us (40771us total) trans 4294972501 rc 0/0 00000100:00100000:3.0:1789760691.650656:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5212 00000100:00100000:3.0:1789760691.650665:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 671107041320 time 671175297852 state 0x2 ref 0 00000100:00100000:2.0:1789760691.653284:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760691.653465:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512520192 00000100:00100000:0.0:1789760691.653475:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512520192 00000100:00100000:0.0:1789760691.653527:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 671107041320 time 671175297852 state 0x0 ref 1 00000100:00100000:0.0:1789760691.653542:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 671175558268 time 671175297852 state 0x1 ref 1 00000100:00100000:0.0:1789760691.653573:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 671175558268 time 671217872738 state 0x0 ref 1 00000100:00100000:0.0:1789760691.653587:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febea4ba850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512520192:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760691.653638:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21229568, mtime: 0x6aad94b3, atime: 0x0, ctime: 0x6aad94b3, blocks: 0xa807 00010000:00010000:0.0:1789760691.653646:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2cee3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760691.653661:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=671s v=(0 0) 00010000:00010000:0.0:1789760691.653666:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760691.653672:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2cee3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21229568->21233663] (req 21229568->21233663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760691.653685:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2cee3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21229568->21233663] (req 21229568->21233663) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760691.690408:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21229568 -> 21233664 00010000:00010000:0.0:1789760691.690427:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2cee3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21229568->21233663] (req 21229568->21233663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760691.690449:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2cee3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21229568->21233663] (req 21229568->21233663) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760691.690459:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2cee3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21229568->21233663] (req 21229568->21233663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760691.690467:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760691.690472:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2cee3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21229568->21233663] (req 21229568->21233663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760691.690483:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2cee3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21229568->21233663] (req 21229568->21233663) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760691.690501:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2cee3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21229568->21233663] (req 21229568->21233663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760691.690509:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2cee3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21229568->21233663] (req 21229568->21233663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760691.690602:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febea4ba850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2361:x1876699512520192:12345-192.168.203.56@tcp:4:dd.0 Request processed in 37012us (37320us total) trans 4294972502 rc 0/0 00000100:00100000:0.0:1789760691.690612:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5213 00000100:00100000:0.0:1789760691.690622:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 671175558268 time 671217872738 state 0x2 ref 0 00000100:00100000:0.0:1789760691.695482:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512520320 00000100:00100000:0.0:1789760691.695492:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512520320 00000100:00100000:0.0:1789760691.695545:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 671175558268 time 671217872738 state 0x0 ref 1 00000100:00100000:0.0:1789760691.695560:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 671218133154 time 671217872738 state 0x1 ref 1 00000100:00100000:0.0:1789760691.695589:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 671218133154 time 671259887978 state 0x0 ref 1 00000100:00100000:0.0:1789760691.695601:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac38b2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512520320:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760691.695649:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21233664, mtime: 0x6aad94b3, atime: 0x0, ctime: 0x6aad94b3, blocks: 0xa807 00010000:00010000:0.0:1789760691.695658:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2ceea lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760691.695673:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=671s v=(0 0) 00010000:00010000:0.0:1789760691.695677:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760691.695683:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2ceea lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21233664->21237759] (req 21233664->21237759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760691.695745:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2ceea lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21233664->21237759] (req 21233664->21237759) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760691.721126:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21233664 -> 21237760 00010000:00010000:3.0:1789760691.721142:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2ceea lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21233664->21237759] (req 21233664->21237759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760691.721156:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2ceea lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21233664->21237759] (req 21233664->21237759) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760691.721165:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2ceea lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21233664->21237759] (req 21233664->21237759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760691.721173:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760691.721178:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2ceea lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21233664->21237759] (req 21233664->21237759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760691.721187:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2ceea lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21233664->21237759] (req 21233664->21237759) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760691.721206:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2ceea lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21233664->21237759] (req 21233664->21237759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760691.721214:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2ceea lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21233664->21237759] (req 21233664->21237759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760691.721302:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac38b2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512520320:12345-192.168.203.56@tcp:4:dd.0 Request processed in 25699us (25903us total) trans 4294972503 rc 0/0 00000100:00100000:3.0:1789760691.721312:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5214 00000100:00100000:3.0:1789760691.721322:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 671218133154 time 671259887978 state 0x2 ref 0 00000100:00100000:3.0:1789760691.726355:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000001:00080000:2.0:1789760691.753590:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972504 is committed 00000100:00100000:2.0:1789760691.761948:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760691.762016:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512520576 00000100:00100000:0.0:1789760691.762027:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512520576 00000100:00100000:0.0:1789760691.762208:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 671260148394 time 671290994070 state 0x0 ref 1 00000100:00100000:0.0:1789760691.762224:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 671291254486 time 671290994070 state 0x1 ref 1 00000100:00100000:0.0:1789760691.762257:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 671291254486 time 671326556258 state 0x0 ref 1 00000100:00100000:0.0:1789760691.762273:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac28a2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512520576:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760691.762318:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21241856, mtime: 0x6aad94b3, atime: 0x0, ctime: 0x6aad94b3, blocks: 0xa807 00010000:00010000:0.0:1789760691.762327:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2cef8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760691.762342:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=671s v=(0 0) 00010000:00010000:0.0:1789760691.762346:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760691.762352:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2cef8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21241856->21245951] (req 21241856->21245951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760691.762365:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2cef8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21241856->21245951] (req 21241856->21245951) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760691.779359:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21241856 -> 21245952 00010000:00010000:0.0:1789760691.779379:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2cef8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21241856->21245951] (req 21241856->21245951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760691.779393:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2cef8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21241856->21245951] (req 21241856->21245951) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760691.779402:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2cef8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21241856->21245951] (req 21241856->21245951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760691.779410:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760691.779415:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2cef8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21241856->21245951] (req 21241856->21245951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760691.779425:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2cef8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21241856->21245951] (req 21241856->21245951) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760691.779431:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972505 is committed 00010000:00010000:0.0:1789760691.779443:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2cef8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21241856->21245951] (req 21241856->21245951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760691.779450:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2cef8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21241856->21245951] (req 21241856->21245951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760691.779534:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac28a2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512520576:12345-192.168.203.56@tcp:4:dd.0 Request processed in 17259us (17589us total) trans 4294972505 rc 0/0 00000100:00100000:0.0:1789760691.779545:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5216 00000100:00100000:0.0:1789760691.779555:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 671291254486 time 671326556258 state 0x2 ref 0 00000100:00100000:2.0:1789760691.781959:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760691.782080:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512520704 00000100:00100000:3.0:1789760691.782090:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512520704 00000100:00100000:3.0:1789760691.782148:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 671291254486 time 671326556258 state 0x0 ref 1 00000100:00100000:3.0:1789760691.782174:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 671326816674 time 671326556258 state 0x1 ref 1 00000100:00100000:3.0:1789760691.782206:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 671326816674 time 671346505756 state 0x0 ref 1 00000100:00100000:3.0:1789760691.782220:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac28a0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512520704:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760691.782264:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21245952, mtime: 0x6aad94b3, atime: 0x0, ctime: 0x6aad94b3, blocks: 0xa807 00010000:00010000:3.0:1789760691.782278:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2ceff lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760691.782292:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=671s v=(0 0) 00010000:00010000:3.0:1789760691.782297:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760691.782304:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2ceff lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21245952->21250047] (req 21245952->21250047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760691.782318:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2ceff lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21245952->21250047] (req 21245952->21250047) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760691.803796:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972506 is committed 00002000:00010000:3.0:1789760691.803942:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21245952 -> 21250048 00010000:00010000:3.0:1789760691.803958:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2ceff lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21245952->21250047] (req 21245952->21250047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760691.803975:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2ceff lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21245952->21250047] (req 21245952->21250047) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760691.803984:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2ceff lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21245952->21250047] (req 21245952->21250047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760691.803990:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760691.803995:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2ceff lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21245952->21250047] (req 21245952->21250047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760691.804003:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2ceff lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21245952->21250047] (req 21245952->21250047) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760691.804023:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2ceff lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21245952->21250047] (req 21245952->21250047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760691.804030:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2ceff lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21245952->21250047] (req 21245952->21250047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760691.804159:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac28a0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512520704:12345-192.168.203.56@tcp:4:dd.0 Request processed in 21938us (22205us total) trans 4294972506 rc 0/0 00000100:00100000:3.0:1789760691.804171:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5217 00000100:00100000:3.0:1789760691.804180:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 671326816674 time 671346505756 state 0x2 ref 0 00000100:00100000:3.0:1789760691.806596:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000001:00080000:2.0:1789760691.847822:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972507 is committed 00002000:00010000:0.0:1789760691.847844:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21250048 -> 21254144 00010000:00010000:0.0:1789760691.847859:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2cf06 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21250048->21254143] (req 21250048->21254143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760691.847873:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2cf06 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21250048->21254143] (req 21250048->21254143) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760691.847882:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2cf06 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21250048->21254143] (req 21250048->21254143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760691.847935:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760691.847939:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2cf06 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21250048->21254143] (req 21250048->21254143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760691.847948:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2cf06 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21250048->21254143] (req 21250048->21254143) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760691.848000:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2cf06 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21250048->21254143] (req 21250048->21254143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760691.848008:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2cf06 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21250048->21254143] (req 21250048->21254143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760691.848096:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac38b4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512520832:12345-192.168.203.56@tcp:4:dd.0 Request processed in 41239us (41504us total) trans 4294972507 rc 0/0 00000100:00100000:0.0:1789760691.848106:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5218 00000100:00100000:0.0:1789760691.848115:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 671346766172 time 671371141713 state 0x2 ref 0 00000100:00100000:2.0:1789760691.854006:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760691.854692:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512520960 00000100:00100000:0.0:1789760691.854733:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512520960 00000100:00100000:0.0:1789760691.854790:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 671346766172 time 671371141713 state 0x0 ref 1 00000100:00100000:0.0:1789760691.854806:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 671371402129 time 671371141713 state 0x1 ref 1 00000001:00080000:2.0:1789760691.918839:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972508 is committed 00002000:00010000:3.0:1789760691.918948:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21254144 -> 21258240 00010000:00010000:3.0:1789760691.918964:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2cf0d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21254144->21258239] (req 21254144->21258239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760691.919566:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2cf0d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21254144->21258239] (req 21254144->21258239) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760691.919581:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2cf0d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21254144->21258239] (req 21254144->21258239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760691.919589:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760691.919593:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2cf0d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21254144->21258239] (req 21254144->21258239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760691.919602:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2cf0d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21254144->21258239] (req 21254144->21258239) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760691.919622:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2cf0d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21254144->21258239] (req 21254144->21258239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760691.919628:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2cf0d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21254144->21258239] (req 21254144->21258239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760691.919785:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac28a3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512520960:12345-192.168.203.56@tcp:4:dd.0 Request processed in 60569us (65783us total) trans 4294972508 rc 0/0 00000100:00100000:3.0:1789760691.919796:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5219 00000100:00100000:3.0:1789760691.919806:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 671371402129 time 671422115905 state 0x2 ref 0 00000100:00100000:2.0:1789760691.928089:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760691.928240:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512521088 00000100:00100000:3.0:1789760691.928250:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512521088 00000100:00100000:3.0:1789760691.928303:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 671371402129 time 671422115905 state 0x0 ref 1 00000100:00100000:3.0:1789760691.928318:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 671422376321 time 671422115905 state 0x1 ref 1 00000100:00100000:3.0:1789760691.928405:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 671422376321 time 671492703784 state 0x0 ref 1 00000100:00100000:3.0:1789760691.928418:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac28a1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512521088:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760691.928463:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21258240, mtime: 0x6aad94b3, atime: 0x0, ctime: 0x6aad94b3, blocks: 0xa807 00010000:00010000:3.0:1789760691.928470:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2cf14 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760691.928485:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=671s v=(0 0) 00010000:00010000:3.0:1789760691.928489:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760691.928495:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2cf14 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21258240->21262335] (req 21258240->21262335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760691.928507:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2cf14 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21258240->21262335] (req 21258240->21262335) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760691.974289:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972509 is committed 00000100:00100000:0.0:1789760691.977616:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00002000:00010000:2.0:1789760692.004023:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21262336 -> 21266432 00010000:00010000:2.0:1789760692.004046:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2cf1b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21262336->21266431] (req 21262336->21266431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.004064:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2cf1b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21262336->21266431] (req 21262336->21266431) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.004074:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2cf1b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21262336->21266431] (req 21262336->21266431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.004083:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760692.004087:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2cf1b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21262336->21266431] (req 21262336->21266431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760692.004090:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972510 is committed 00010000:00010000:2.0:1789760692.004096:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2cf1b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21262336->21266431] (req 21262336->21266431) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.004114:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2cf1b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21262336->21266431] (req 21262336->21266431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.004122:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2cf1b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21262336->21266431] (req 21262336->21266431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760692.004202:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febea651500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512521216:12345-192.168.203.56@tcp:4:dd.0 Request processed in 26371us (26588us total) trans 4294972510 rc 0/0 00000100:00100000:2.0:1789760692.004212:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5221 00000100:00100000:2.0:1789760692.004221:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 671492964200 time 671542116960 state 0x2 ref 0 00000100:00100000:2.0:1789760692.006171:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760692.006350:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512521344 00000100:00100000:3.0:1789760692.006359:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512521344 00000100:00100000:3.0:1789760692.006468:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 671492964200 time 671542116960 state 0x0 ref 1 00000100:00100000:3.0:1789760692.006483:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 671542377376 time 671542116960 state 0x1 ref 1 00000100:00100000:3.0:1789760692.006513:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 671542377376 time 671570813194 state 0x0 ref 1 00000100:00100000:3.0:1789760692.006524:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febfbfe7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512521344:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760692.006561:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21266432, mtime: 0x6aad94b3, atime: 0x0, ctime: 0x6aad94b3, blocks: 0xa807 00010000:00010000:3.0:1789760692.006579:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2cf22 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760692.006592:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=671s v=(0 0) 00010000:00010000:3.0:1789760692.006595:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760692.006753:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2cf22 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21266432->21270527] (req 21266432->21270527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760692.006770:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2cf22 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21266432->21270527] (req 21266432->21270527) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760692.044497:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972511 is committed 00000100:00100000:3.0:1789760692.048117:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512521472 00000100:00100000:3.0:1789760692.048128:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512521472 00000100:00100000:3.0:1789760692.048180:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 671542377376 time 671570813194 state 0x0 ref 1 00000100:00100000:3.0:1789760692.048194:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 671571073610 time 671570813194 state 0x1 ref 1 00000100:00100000:3.0:1789760692.048324:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 671571073610 time 671612556949 state 0x0 ref 1 00000100:00100000:3.0:1789760692.048338:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac38b3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512521472:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760692.048393:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21270528, mtime: 0x6aad94b3, atime: 0x0, ctime: 0x6aad94b3, blocks: 0xa807 00010000:00010000:3.0:1789760692.048401:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2cf29 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760692.048416:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=671s v=(0 0) 00010000:00010000:3.0:1789760692.048420:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760692.048426:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2cf29 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21270528->21274623] (req 21270528->21274623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760692.048440:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2cf29 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21270528->21274623] (req 21270528->21274623) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760692.067010:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972512 is committed 00000100:00100000:3.0:1789760692.070076:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760692.070204:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512521600 00000100:00100000:2.0:1789760692.070213:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512521600 00000100:00100000:2.0:1789760692.070265:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 671571073610 time 671612556949 state 0x0 ref 1 00000100:00100000:2.0:1789760692.070280:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 671612817365 time 671612556949 state 0x1 ref 1 00000100:00100000:2.0:1789760692.070313:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 671612817365 time 671634613021 state 0x0 ref 1 00000100:00100000:2.0:1789760692.070324:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac38b7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512521600:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760692.070384:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21274624, mtime: 0x6aad94b3, atime: 0x0, ctime: 0x6aad94b3, blocks: 0xa807 00010000:00010000:2.0:1789760692.070394:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2cf30 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.070411:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=671s v=(0 0) 00010000:00010000:2.0:1789760692.070415:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760692.070422:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2cf30 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21274624->21278719] (req 21274624->21278719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.070437:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2cf30 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21274624->21278719] (req 21274624->21278719) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760692.092828:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972513 is committed 00000100:00100000:2.0:1789760692.095338:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000001:00080000:3.0:1789760692.120853:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972514 is committed 00002000:00010000:2.0:1789760692.121236:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21278720 -> 21282816 00010000:00010000:2.0:1789760692.121252:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2cf37 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21278720->21282815] (req 21278720->21282815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.121266:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2cf37 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21278720->21282815] (req 21278720->21282815) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.121275:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2cf37 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21278720->21282815] (req 21278720->21282815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.121288:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760692.121293:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2cf37 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21278720->21282815] (req 21278720->21282815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.121302:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2cf37 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21278720->21282815] (req 21278720->21282815) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.121322:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2cf37 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21278720->21282815] (req 21278720->21282815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.121330:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2cf37 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21278720->21282815] (req 21278720->21282815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760692.121435:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac3352300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512521728:12345-192.168.203.56@tcp:4:dd.0 Request processed in 25887us (26100us total) trans 4294972514 rc 0/0 00000100:00100000:2.0:1789760692.121445:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5225 00000100:00100000:2.0:1789760692.121454:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 671634873437 time 671659833399 state 0x2 ref 0 00000100:00100000:2.0:1789760692.125149:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760692.126207:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 671660093815 time 671690366844 state 0x0 ref 1 00000100:00100000:0.0:1789760692.126222:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac3353800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512521856:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760692.126269:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21282816, mtime: 0x6aad94b3, atime: 0x0, ctime: 0x6aad94b3, blocks: 0xa807 00010000:00010000:0.0:1789760692.126277:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2cf3e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.126292:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=671s v=(0 0) 00010000:00010000:0.0:1789760692.126296:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760692.126302:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2cf3e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21282816->21286911] (req 21282816->21286911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.126315:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2cf3e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21282816->21286911] (req 21282816->21286911) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760692.150684:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21282816 -> 21286912 00000001:00080000:3.0:1789760692.150730:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972515 is committed 00010000:00010000:2.0:1789760692.150733:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2cf3e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21282816->21286911] (req 21282816->21286911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.150745:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2cf3e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21282816->21286911] (req 21282816->21286911) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.150760:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2cf3e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21282816->21286911] (req 21282816->21286911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.150767:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760692.150772:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2cf3e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21282816->21286911] (req 21282816->21286911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.150781:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2cf3e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21282816->21286911] (req 21282816->21286911) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.150798:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2cf3e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21282816->21286911] (req 21282816->21286911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.150805:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2cf3e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21282816->21286911] (req 21282816->21286911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760692.150886:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac3353800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512521856:12345-192.168.203.56@tcp:4:dd.0 Request processed in 24662us (25740us total) trans 4294972515 rc 0/0 00000100:00100000:2.0:1789760692.150896:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5226 00000100:00100000:2.0:1789760692.150905:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 671660093815 time 671690366844 state 0x2 ref 0 00000100:00100000:3.0:1789760692.157417:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00002000:00010000:2.0:1789760692.179862:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21286912 -> 21291008 00010000:00010000:2.0:1789760692.179884:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2cf45 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21286912->21291007] (req 21286912->21291007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.179904:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2cf45 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21286912->21291007] (req 21286912->21291007) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.179914:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2cf45 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21286912->21291007] (req 21286912->21291007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.179921:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760692.179927:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2cf45 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21286912->21291007] (req 21286912->21291007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.179936:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2cf45 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21286912->21291007] (req 21286912->21291007) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.179957:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2cf45 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21286912->21291007] (req 21286912->21291007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.179965:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2cf45 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21286912->21291007] (req 21286912->21291007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760692.180098:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac38b4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2361:x1876699512521984:12345-192.168.203.56@tcp:4:dd.0 Request processed in 22330us (22684us total) trans 4294972516 rc 0/0 00000100:00100000:2.0:1789760692.180109:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5227 00000100:00100000:2.0:1789760692.180119:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 671690627260 time 671722050595 state 0x2 ref 0 00000001:00080000:3.0:1789760692.180253:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972516 is committed 00000100:00100000:2.0:1789760692.187264:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512522112 00000100:00100000:2.0:1789760692.187274:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512522112 00000100:00100000:2.0:1789760692.187480:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 671690627260 time 671722050595 state 0x0 ref 1 00000100:00100000:2.0:1789760692.187497:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 671722311011 time 671722050595 state 0x1 ref 1 00000100:00100000:2.0:1789760692.187560:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 671722311011 time 671751858913 state 0x0 ref 1 00000100:00100000:2.0:1789760692.187573:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac38b3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512522112:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760692.187624:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21291008, mtime: 0x6aad94b3, atime: 0x0, ctime: 0x6aad94b3, blocks: 0xa807 00010000:00010000:2.0:1789760692.187633:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2cf4c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.187648:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=671s v=(0 0) 00010000:00010000:2.0:1789760692.187653:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760692.187660:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2cf4c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21291008->21295103] (req 21291008->21295103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.187674:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2cf4c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21291008->21295103] (req 21291008->21295103) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760692.212126:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21291008 -> 21295104 00010000:00010000:0.0:1789760692.212146:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2cf4c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21291008->21295103] (req 21291008->21295103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.212158:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2cf4c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21291008->21295103] (req 21291008->21295103) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.212167:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2cf4c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21291008->21295103] (req 21291008->21295103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.212174:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760692.212179:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2cf4c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21291008->21295103] (req 21291008->21295103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.212188:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2cf4c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21291008->21295103] (req 21291008->21295103) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.212206:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2cf4c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21291008->21295103] (req 21291008->21295103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.212214:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2cf4c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21291008->21295103] (req 21291008->21295103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760692.212305:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac38b3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2361:x1876699512522112:12345-192.168.203.56@tcp:4:dd.0 Request processed in 24729us (25199us total) trans 4294972517 rc 0/0 00000100:00100000:0.0:1789760692.212316:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5228 00000100:00100000:0.0:1789760692.212327:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 671722311011 time 671751858913 state 0x2 ref 0 00000001:00080000:3.0:1789760692.212402:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972517 is committed 00000100:00100000:2.0:1789760692.215468:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512522240 00000100:00100000:2.0:1789760692.215478:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512522240 00000100:00100000:2.0:1789760692.215530:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 671722311011 time 671751858913 state 0x0 ref 1 00000100:00100000:2.0:1789760692.215545:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 671752119329 time 671751858913 state 0x1 ref 1 00000100:00100000:2.0:1789760692.215573:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 671752119329 time 671779873277 state 0x0 ref 1 00000100:00100000:2.0:1789760692.215589:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac38b0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512522240:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760692.215633:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21295104, mtime: 0x6aad94b3, atime: 0x0, ctime: 0x6aad94b3, blocks: 0xa807 00010000:00010000:2.0:1789760692.215642:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2cf53 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.215658:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=671s v=(0 0) 00010000:00010000:2.0:1789760692.215662:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760692.215667:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2cf53 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21295104->21299199] (req 21295104->21299199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.215680:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2cf53 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21295104->21299199] (req 21295104->21299199) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760692.245162:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21295104 -> 21299200 00010000:00010000:2.0:1789760692.245178:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2cf53 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21295104->21299199] (req 21295104->21299199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.245191:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2cf53 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21295104->21299199] (req 21295104->21299199) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.245200:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2cf53 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21295104->21299199] (req 21295104->21299199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.245207:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760692.245212:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2cf53 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21295104->21299199] (req 21295104->21299199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.245222:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2cf53 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21295104->21299199] (req 21295104->21299199) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.245241:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2cf53 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21295104->21299199] (req 21295104->21299199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.245249:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2cf53 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21295104->21299199] (req 21295104->21299199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760692.245327:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac38b0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512522240:12345-192.168.203.56@tcp:4:dd.0 Request processed in 29736us (30176us total) trans 4294972518 rc 0/0 00000100:00100000:2.0:1789760692.245337:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5229 00000100:00100000:2.0:1789760692.245347:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 671752119329 time 671779873277 state 0x2 ref 0 00000100:00100000:2.0:1789760692.249981:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512522368 00000100:00100000:2.0:1789760692.249990:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512522368 00000100:00100000:2.0:1789760692.250045:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 671752119329 time 671779873277 state 0x0 ref 1 00000100:00100000:2.0:1789760692.250061:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 671780133693 time 671779873277 state 0x1 ref 1 00000100:00100000:2.0:1789760692.250081:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 671780133693 time 671814381168 state 0x0 ref 1 00000100:00100000:2.0:1789760692.250093:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac38b3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512522368:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760692.250143:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21299200, mtime: 0x6aad94b3, atime: 0x0, ctime: 0x6aad94b3, blocks: 0xa807 00010000:00010000:2.0:1789760692.250152:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2cf5a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.250169:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=671s v=(0 0) 00010000:00010000:2.0:1789760692.250173:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760692.250180:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2cf5a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21299200->21303295] (req 21299200->21303295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.250194:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2cf5a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21299200->21303295] (req 21299200->21303295) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760692.280869:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21299200 -> 21303296 00010000:00010000:0.0:1789760692.280885:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2cf5a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21299200->21303295] (req 21299200->21303295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.280897:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2cf5a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21299200->21303295] (req 21299200->21303295) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.280905:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2cf5a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21299200->21303295] (req 21299200->21303295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.280912:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760692.280918:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2cf5a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21299200->21303295] (req 21299200->21303295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.280927:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2cf5a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21299200->21303295] (req 21299200->21303295) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.280947:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2cf5a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21299200->21303295] (req 21299200->21303295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.280954:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2cf5a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21299200->21303295] (req 21299200->21303295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760692.281229:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac38b3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512522368:12345-192.168.203.56@tcp:4:dd.0 Request processed in 31132us (31537us total) trans 4294972519 rc 0/0 00000100:00100000:0.0:1789760692.281239:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5230 00000100:00100000:0.0:1789760692.281248:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 671780133693 time 671814381168 state 0x2 ref 0 00000100:00100000:0.0:1789760692.284438:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512522496 00000100:00100000:0.0:1789760692.284449:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512522496 00000100:00100000:0.0:1789760692.284504:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 671780133693 time 671814381168 state 0x0 ref 1 00000100:00100000:0.0:1789760692.284520:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 671814641584 time 671814381168 state 0x1 ref 1 00000100:00100000:0.0:1789760692.284587:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 671814641584 time 671848886055 state 0x0 ref 1 00000100:00100000:0.0:1789760692.284601:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac38b2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512522496:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760692.284649:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21303296, mtime: 0x6aad94b3, atime: 0x0, ctime: 0x6aad94b3, blocks: 0xa807 00010000:00010000:0.0:1789760692.284658:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2cf61 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.284674:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=671s v=(0 0) 00010000:00010000:0.0:1789760692.284679:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760692.284685:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2cf61 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21303296->21307391] (req 21303296->21307391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.284751:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2cf61 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21303296->21307391] (req 21303296->21307391) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760692.313303:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972520 is committed 00000100:00100000:2.0:1789760692.316822:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512522624 00000100:00100000:2.0:1789760692.316832:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512522624 00000100:00100000:2.0:1789760692.316887:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 671814641584 time 671848886055 state 0x0 ref 1 00000100:00100000:2.0:1789760692.316907:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 671849146471 time 671848886055 state 0x1 ref 1 00000100:00100000:2.0:1789760692.316938:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 671849146471 time 671881237921 state 0x0 ref 1 00000100:00100000:2.0:1789760692.316951:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac38b1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512522624:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760692.316997:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21307392, mtime: 0x6aad94b3, atime: 0x0, ctime: 0x6aad94b3, blocks: 0xa807 00010000:00010000:2.0:1789760692.317005:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2cf68 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.317019:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=671s v=(0 0) 00010000:00010000:2.0:1789760692.317023:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760692.317029:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2cf68 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21307392->21311487] (req 21307392->21311487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.317048:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2cf68 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21307392->21311487] (req 21307392->21311487) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760692.373012:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972521 is committed 00002000:00010000:3.0:1789760692.373043:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21307392 -> 21311488 00010000:00010000:3.0:1789760692.373059:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2cf68 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21307392->21311487] (req 21307392->21311487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760692.373072:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2cf68 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21307392->21311487] (req 21307392->21311487) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760692.373081:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2cf68 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21307392->21311487] (req 21307392->21311487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760692.373165:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760692.373170:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2cf68 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21307392->21311487] (req 21307392->21311487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760692.373182:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2cf68 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21307392->21311487] (req 21307392->21311487) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760692.373201:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2cf68 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21307392->21311487] (req 21307392->21311487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760692.373208:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2cf68 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21307392->21311487] (req 21307392->21311487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760692.373305:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac38b1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512522624:12345-192.168.203.56@tcp:4:dd.0 Request processed in 56353us (56560us total) trans 4294972521 rc 0/0 00000100:00100000:3.0:1789760692.373317:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5232 00000100:00100000:3.0:1789760692.373328:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 671849146471 time 671881237921 state 0x2 ref 0 00000100:00100000:3.0:1789760692.376472:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760692.376559:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512522752 00000100:00100000:0.0:1789760692.376569:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512522752 00000100:00100000:0.0:1789760692.376627:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 671849146471 time 671881237921 state 0x0 ref 1 00000100:00100000:0.0:1789760692.376643:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 671881498337 time 671881237921 state 0x1 ref 1 00000100:00100000:0.0:1789760692.377642:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 671881498337 time 671941937198 state 0x0 ref 1 00000100:00100000:0.0:1789760692.377660:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac38b6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512522752:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760692.377738:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21311488, mtime: 0x6aad94b3, atime: 0x0, ctime: 0x6aad94b3, blocks: 0xa807 00010000:00010000:0.0:1789760692.377748:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2cf6f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.377764:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=671s v=(0 0) 00010000:00010000:0.0:1789760692.377769:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760692.377776:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2cf6f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21311488->21315583] (req 21311488->21315583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.377789:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2cf6f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21311488->21315583] (req 21311488->21315583) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760692.407722:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972522 is committed 00000100:00100000:0.0:1789760692.411551:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512522880 00000100:00100000:0.0:1789760692.411561:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512522880 00000100:00100000:0.0:1789760692.411616:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 671881498337 time 671941937198 state 0x0 ref 1 00000100:00100000:0.0:1789760692.411630:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 671942197614 time 671941937198 state 0x1 ref 1 00000100:00100000:0.0:1789760692.411661:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 671942197614 time 671975961088 state 0x0 ref 1 00000100:00100000:0.0:1789760692.411673:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac38b1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512522880:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760692.411763:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21315584, mtime: 0x6aad94b3, atime: 0x0, ctime: 0x6aad94b3, blocks: 0xa807 00010000:00010000:0.0:1789760692.411772:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2cf76 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.411789:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=671s v=(0 0) 00010000:00010000:0.0:1789760692.411793:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760692.411800:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2cf76 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21315584->21319679] (req 21315584->21319679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.411813:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2cf76 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21315584->21319679] (req 21315584->21319679) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760692.447771:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972523 is committed 00002000:00010000:0.0:1789760692.447833:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21315584 -> 21319680 00010000:00010000:0.0:1789760692.447849:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2cf76 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21315584->21319679] (req 21315584->21319679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.447863:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2cf76 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21315584->21319679] (req 21315584->21319679) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.447872:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2cf76 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21315584->21319679] (req 21315584->21319679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.447879:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760692.447885:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2cf76 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21315584->21319679] (req 21315584->21319679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.447895:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2cf76 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21315584->21319679] (req 21315584->21319679) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.447918:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2cf76 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21315584->21319679] (req 21315584->21319679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.447926:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2cf76 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21315584->21319679] (req 21315584->21319679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760692.448011:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac38b1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512522880:12345-192.168.203.56@tcp:4:dd.0 Request processed in 36334us (36779us total) trans 4294972523 rc 0/0 00000100:00100000:0.0:1789760692.450259:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5234 00000100:00100000:0.0:1789760692.450272:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 671942197614 time 671975961088 state 0x2 ref 0 00000100:00100000:0.0:1789760692.453401:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760692.453554:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512523008 00000100:00100000:3.0:1789760692.453564:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512523008 00000100:00100000:3.0:1789760692.453617:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 671942197614 time 671975961088 state 0x0 ref 1 00000100:00100000:3.0:1789760692.453632:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 671976221504 time 671975961088 state 0x1 ref 1 00000100:00100000:3.0:1789760692.453663:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 671976221504 time 672017963331 state 0x0 ref 1 00000100:00100000:3.0:1789760692.453676:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febea650380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512523008:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760692.453771:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21319680, mtime: 0x6aad94b3, atime: 0x0, ctime: 0x6aad94b3, blocks: 0xa807 00010000:00010000:3.0:1789760692.453780:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2cf7d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760692.453795:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=672s v=(0 0) 00010000:00010000:3.0:1789760692.453799:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760692.453806:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2cf7d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21319680->21323775] (req 21319680->21323775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760692.453819:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2cf7d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21319680->21323775] (req 21319680->21323775) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760692.506468:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21319680 -> 21323776 00010000:00010000:0.0:1789760692.506483:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2cf7d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21319680->21323775] (req 21319680->21323775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.506496:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2cf7d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21319680->21323775] (req 21319680->21323775) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.506506:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2cf7d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21319680->21323775] (req 21319680->21323775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.506514:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760692.506519:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2cf7d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21319680->21323775] (req 21319680->21323775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.506528:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2cf7d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21319680->21323775] (req 21319680->21323775) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.506547:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2cf7d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21319680->21323775] (req 21319680->21323775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.506555:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2cf7d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21319680->21323775] (req 21319680->21323775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760692.506848:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febea650380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512523008:12345-192.168.203.56@tcp:4:dd.0 Request processed in 53169us (53463us total) trans 4294972524 rc 0/0 00000100:00100000:0.0:1789760692.506859:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5235 00000100:00100000:0.0:1789760692.506869:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 671976221504 time 672017963331 state 0x2 ref 0 00000100:00100000:3.0:1789760692.509365:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00002000:00010000:0.0:1789760692.542362:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21323776 -> 21327872 00002000:00010000:0.0:1789760692.542368:0:6708:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760691 -> 1789760692 00002000:00010000:0.0:1789760692.542371:0:6708:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760691 -> 1789760692 00010000:00010000:0.0:1789760692.542385:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2cf84 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21323776->21327871] (req 21323776->21327871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.542399:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2cf84 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21323776->21327871] (req 21323776->21327871) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.542408:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2cf84 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21323776->21327871] (req 21323776->21327871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.542415:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760692.542420:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2cf84 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21323776->21327871] (req 21323776->21327871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.542430:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2cf84 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21323776->21327871] (req 21323776->21327871) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.542449:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2cf84 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21323776->21327871] (req 21323776->21327871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.542458:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2cf84 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21323776->21327871] (req 21323776->21327871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760692.542556:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac38b5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512523136:12345-192.168.203.56@tcp:4:dd.0 Request processed in 32635us (33197us total) trans 4294972525 rc 0/0 00000100:00100000:0.0:1789760692.542566:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5236 00000100:00100000:0.0:1789760692.542576:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 672018223747 time 672074204273 state 0x2 ref 0 00000100:00100000:0.0:1789760692.554406:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512523264 00000100:00100000:0.0:1789760692.554417:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512523264 00000100:00100000:0.0:1789760692.554473:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 672018223747 time 672074204273 state 0x0 ref 1 00000100:00100000:0.0:1789760692.554489:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 672074464689 time 672074204273 state 0x1 ref 1 00000100:00100000:0.0:1789760692.554519:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 672074464689 time 672118818550 state 0x0 ref 1 00000100:00100000:0.0:1789760692.554532:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febff933b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512523264:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760692.554581:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21327872, mtime: 0x6aad94b4, atime: 0x0, ctime: 0x6aad94b4, blocks: 0xa807 00010000:00010000:0.0:1789760692.554591:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2cf8b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.554608:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=672s v=(0 0) 00010000:00010000:0.0:1789760692.554612:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760692.554619:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2cf8b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21327872->21331967] (req 21327872->21331967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.554638:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2cf8b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21327872->21331967] (req 21327872->21331967) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.594843:0:5836:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:0.0:1789760692.652833:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21327872 -> 21331968 00010000:00010000:0.0:1789760692.653081:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2cf8b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21327872->21331967] (req 21327872->21331967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.653164:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2cf8b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21327872->21331967] (req 21327872->21331967) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.653175:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2cf8b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21327872->21331967] (req 21327872->21331967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.653183:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760692.653188:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2cf8b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21327872->21331967] (req 21327872->21331967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.653203:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2cf8b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21327872->21331967] (req 21327872->21331967) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.653225:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2cf8b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21327872->21331967] (req 21327872->21331967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.653232:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2cf8b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21327872->21331967] (req 21327872->21331967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760692.653366:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febff933b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512523264:12345-192.168.203.56@tcp:4:dd.0 Request processed in 98832us (99061us total) trans 4294972526 rc 0/0 00000100:00100000:0.0:1789760692.653385:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5237 00000100:00100000:0.0:1789760692.653394:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 672074464689 time 672118818550 state 0x2 ref 0 00000100:00100000:0.0:1789760692.667237:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512523392 00000100:00100000:0.0:1789760692.667247:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512523392 00000100:00100000:0.0:1789760692.667300:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 672074464689 time 672118818550 state 0x0 ref 1 00000100:00100000:0.0:1789760692.667315:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 672119078966 time 672118818550 state 0x1 ref 1 00000100:00100000:0.0:1789760692.667346:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 672119078966 time 672231645290 state 0x0 ref 1 00000100:00100000:0.0:1789760692.667358:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febff930380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512523392:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760692.667417:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21331968, mtime: 0x6aad94b4, atime: 0x0, ctime: 0x6aad94b4, blocks: 0xa807 00010000:00010000:0.0:1789760692.667428:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2cf92 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.667445:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=672s v=(0 0) 00010000:00010000:0.0:1789760692.667449:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760692.667456:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2cf92 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21331968->21336063] (req 21331968->21336063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.667470:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2cf92 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21331968->21336063] (req 21331968->21336063) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760692.745677:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21331968 -> 21336064 00010000:00010000:0.0:1789760692.745692:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2cf92 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21331968->21336063] (req 21331968->21336063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.745748:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2cf92 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21331968->21336063] (req 21331968->21336063) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.745757:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2cf92 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21331968->21336063] (req 21331968->21336063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.745764:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760692.745769:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2cf92 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21331968->21336063] (req 21331968->21336063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.745777:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2cf92 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21331968->21336063] (req 21331968->21336063) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.745795:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2cf92 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21331968->21336063] (req 21331968->21336063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.745803:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2cf92 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21331968->21336063] (req 21331968->21336063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760692.745884:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febff930380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512523392:12345-192.168.203.56@tcp:4:dd.0 Request processed in 78523us (79235us total) trans 4294972527 rc 0/0 00000100:00100000:0.0:1789760692.745899:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5238 00000100:00100000:0.0:1789760692.745909:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 672119078966 time 672231645290 state 0x2 ref 0 00000100:00100000:2.0:1789760692.753934:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760692.754004:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512523520 00000100:00100000:0.0:1789760692.754014:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512523520 00000100:00100000:0.0:1789760692.754063:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 672119078966 time 672231645290 state 0x0 ref 1 00000100:00100000:0.0:1789760692.754079:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 672231905706 time 672231645290 state 0x1 ref 1 00000100:00100000:0.0:1789760692.754109:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 672231905706 time 672318408443 state 0x0 ref 1 00000100:00100000:0.0:1789760692.754122:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac3352a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512523520:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760692.754172:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21336064, mtime: 0x6aad94b4, atime: 0x0, ctime: 0x6aad94b4, blocks: 0xa807 00010000:00010000:0.0:1789760692.754181:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2cf99 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.754198:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=672s v=(0 0) 00010000:00010000:0.0:1789760692.754203:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760692.754209:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2cf99 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21336064->21340159] (req 21336064->21340159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.754221:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2cf99 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21336064->21340159] (req 21336064->21340159) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760692.835224:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21336064 -> 21340160 00010000:00010000:0.0:1789760692.835242:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2cf99 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21336064->21340159] (req 21336064->21340159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.835255:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2cf99 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21336064->21340159] (req 21336064->21340159) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.835264:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2cf99 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21336064->21340159] (req 21336064->21340159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.835271:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760692.835275:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2cf99 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21336064->21340159] (req 21336064->21340159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.835284:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2cf99 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21336064->21340159] (req 21336064->21340159) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.835303:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2cf99 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21336064->21340159] (req 21336064->21340159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.835309:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2cf99 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21336064->21340159] (req 21336064->21340159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760692.835408:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac3352a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2358:x1876699512523520:12345-192.168.203.56@tcp:4:dd.0 Request processed in 81284us (81479us total) trans 4294972528 rc 0/0 00000100:00100000:0.0:1789760692.835418:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5239 00000100:00100000:0.0:1789760692.835427:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 672231905706 time 672318408443 state 0x2 ref 0 00000100:00100000:0.0:1789760692.838940:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512523648 00000100:00100000:0.0:1789760692.838950:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512523648 00000100:00100000:0.0:1789760692.839002:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 672231905706 time 672318408443 state 0x0 ref 1 00000100:00100000:0.0:1789760692.839017:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 672318668859 time 672318408443 state 0x1 ref 1 00000100:00100000:0.0:1789760692.839045:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 672318668859 time 672403344824 state 0x0 ref 1 00000100:00100000:0.0:1789760692.839056:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febff932a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512523648:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760692.839105:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21340160, mtime: 0x6aad94b4, atime: 0x0, ctime: 0x6aad94b4, blocks: 0xa807 00010000:00010000:0.0:1789760692.839114:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2cfa0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.839129:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=672s v=(0 0) 00010000:00010000:0.0:1789760692.839133:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760692.839139:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2cfa0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21340160->21344255] (req 21340160->21344255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.839152:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2cfa0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21340160->21344255] (req 21340160->21344255) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760692.874618:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972529 is committed 00002000:00010000:0.0:1789760692.874913:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21340160 -> 21344256 00010000:00010000:0.0:1789760692.874928:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2cfa0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21340160->21344255] (req 21340160->21344255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.874941:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2cfa0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21340160->21344255] (req 21340160->21344255) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.874954:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2cfa0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21340160->21344255] (req 21340160->21344255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.874960:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760692.874965:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2cfa0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21340160->21344255] (req 21340160->21344255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.874974:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2cfa0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21340160->21344255] (req 21340160->21344255) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.874993:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2cfa0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21340160->21344255] (req 21340160->21344255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.875001:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2cfa0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21340160->21344255] (req 21340160->21344255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760692.878833:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760692.879937:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512523776 00000100:00100000:0.0:1789760692.879949:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512523776 00000100:00100000:0.0:1789760692.880002:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 672318668859 time 672403344824 state 0x0 ref 1 00000100:00100000:0.0:1789760692.880022:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 672403605240 time 672403344824 state 0x1 ref 1 00000100:00100000:0.0:1789760692.880052:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 672403605240 time 672444351378 state 0x0 ref 1 00000100:00100000:0.0:1789760692.880067:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac3353480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512523776:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760692.880115:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21344256, mtime: 0x6aad94b4, atime: 0x0, ctime: 0x6aad94b4, blocks: 0xa807 00010000:00010000:0.0:1789760692.880124:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2cfa7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.880140:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=672s v=(0 0) 00010000:00010000:0.0:1789760692.880144:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760692.880151:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2cfa7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21344256->21348351] (req 21344256->21348351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.880163:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2cfa7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21344256->21348351] (req 21344256->21348351) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760692.904084:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972530 is committed 00002000:00010000:3.0:1789760692.904170:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21344256 -> 21348352 00010000:00010000:3.0:1789760692.904186:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2cfa7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21344256->21348351] (req 21344256->21348351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760692.904199:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2cfa7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21344256->21348351] (req 21344256->21348351) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760692.904209:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2cfa7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21344256->21348351] (req 21344256->21348351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760692.904216:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760692.904221:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2cfa7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21344256->21348351] (req 21344256->21348351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760692.904239:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2cfa7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21344256->21348351] (req 21344256->21348351) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760692.904362:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2cfa7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21344256->21348351] (req 21344256->21348351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760692.904379:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2cfa7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21344256->21348351] (req 21344256->21348351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760692.904466:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac3353480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512523776:12345-192.168.203.56@tcp:4:dd.0 Request processed in 24399us (25637us total) trans 4294972530 rc 0/0 00000100:00100000:3.0:1789760692.904477:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5241 00000100:00100000:3.0:1789760692.904487:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 672403605240 time 672444351378 state 0x2 ref 0 00000100:00100000:2.0:1789760692.909370:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760692.909439:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512523904 00000100:00100000:0.0:1789760692.909450:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512523904 00000100:00100000:0.0:1789760692.909506:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 672403605240 time 672444351378 state 0x0 ref 1 00000100:00100000:0.0:1789760692.909521:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 672444611794 time 672444351378 state 0x1 ref 1 00000100:00100000:0.0:1789760692.909555:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 672444611794 time 672473855027 state 0x0 ref 1 00000100:00100000:0.0:1789760692.909568:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac3351180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512523904:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760692.909617:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21348352, mtime: 0x6aad94b4, atime: 0x0, ctime: 0x6aad94b4, blocks: 0xa807 00010000:00010000:0.0:1789760692.909627:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2cfae lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.909661:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=672s v=(0 0) 00010000:00010000:0.0:1789760692.909666:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760692.909673:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2cfae lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21348352->21352447] (req 21348352->21352447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.909732:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2cfae lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21348352->21352447] (req 21348352->21352447) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760692.943474:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972531 is committed 00002000:00010000:0.0:1789760692.943784:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21348352 -> 21352448 00010000:00010000:0.0:1789760692.943799:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2cfae lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21348352->21352447] (req 21348352->21352447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.943811:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2cfae lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21348352->21352447] (req 21348352->21352447) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.943819:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2cfae lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21348352->21352447] (req 21348352->21352447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.943826:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760692.943830:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2cfae lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21348352->21352447] (req 21348352->21352447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.943849:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2cfae lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21348352->21352447] (req 21348352->21352447) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.943868:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2cfae lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21348352->21352447] (req 21348352->21352447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.943876:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2cfae lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21348352->21352447] (req 21348352->21352447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760692.943967:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac3351180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512523904:12345-192.168.203.56@tcp:4:dd.0 Request processed in 34397us (34602us total) trans 4294972531 rc 0/0 00000100:00100000:0.0:1789760692.943977:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5242 00000100:00100000:0.0:1789760692.943987:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 672444611794 time 672473855027 state 0x2 ref 0 00000100:00100000:0.0:1789760692.951481:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512524032 00000100:00100000:0.0:1789760692.951491:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512524032 00000100:00100000:0.0:1789760692.951546:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 672444611794 time 672473855027 state 0x0 ref 1 00000100:00100000:0.0:1789760692.951561:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 672474115443 time 672473855027 state 0x1 ref 1 00000100:00100000:0.0:1789760692.951594:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 672474115443 time 672515893616 state 0x0 ref 1 00000100:00100000:0.0:1789760692.951607:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febff930e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512524032:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760692.951655:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21352448, mtime: 0x6aad94b4, atime: 0x0, ctime: 0x6aad94b4, blocks: 0xa807 00010000:00010000:0.0:1789760692.951665:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2cfb5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.951682:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=672s v=(0 0) 00010000:00010000:0.0:1789760692.951686:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760692.951735:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2cfb5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21352448->21356543] (req 21352448->21356543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.951757:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2cfb5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21352448->21356543] (req 21352448->21356543) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760693.018294:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972532 is committed 00002000:00010000:0.0:1789760693.018532:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21352448 -> 21356544 00010000:00010000:0.0:1789760693.018548:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2cfb5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21352448->21356543] (req 21352448->21356543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.018562:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2cfb5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21352448->21356543] (req 21352448->21356543) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.018572:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2cfb5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21352448->21356543] (req 21352448->21356543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.018580:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760693.018585:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2cfb5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21352448->21356543] (req 21352448->21356543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.018595:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2cfb5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21352448->21356543] (req 21352448->21356543) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.018614:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2cfb5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21352448->21356543] (req 21352448->21356543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.018623:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2cfb5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21352448->21356543] (req 21352448->21356543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760693.018778:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febff930e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512524032:12345-192.168.203.56@tcp:4:dd.0 Request processed in 67169us (67714us total) trans 4294972532 rc 0/0 00000100:00100000:0.0:1789760693.018789:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5243 00000100:00100000:0.0:1789760693.018798:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 672474115443 time 672515893616 state 0x2 ref 0 00000100:00100000:2.0:1789760693.024755:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760693.024903:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512524160 00000100:00100000:3.0:1789760693.024914:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512524160 00000100:00100000:3.0:1789760693.024977:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 672474115443 time 672515893616 state 0x0 ref 1 00000100:00100000:3.0:1789760693.024991:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 672516154032 time 672515893616 state 0x1 ref 1 00000100:00100000:3.0:1789760693.025022:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 672516154032 time 672589321666 state 0x0 ref 1 00000100:00100000:3.0:1789760693.025035:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febfc975180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512524160:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760693.025081:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21356544, mtime: 0x6aad94b4, atime: 0x0, ctime: 0x6aad94b4, blocks: 0xa807 00010000:00010000:3.0:1789760693.025091:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2cfbc lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760693.025107:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=672s v=(0 0) 00010000:00010000:3.0:1789760693.025111:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760693.025118:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2cfbc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21356544->21360639] (req 21356544->21360639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760693.025131:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2cfbc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21356544->21360639] (req 21356544->21360639) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760693.099961:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972533 is committed 00002000:00010000:0.0:1789760693.100061:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21356544 -> 21360640 00010000:00010000:0.0:1789760693.100076:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2cfbc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21356544->21360639] (req 21356544->21360639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.100088:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2cfbc lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21356544->21360639] (req 21356544->21360639) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.100097:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2cfbc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21356544->21360639] (req 21356544->21360639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.100105:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760693.100109:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2cfbc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21356544->21360639] (req 21356544->21360639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.100118:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2cfbc lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21356544->21360639] (req 21356544->21360639) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.100135:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2cfbc lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21356544->21360639] (req 21356544->21360639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.100149:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2cfbc lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21356544->21360639] (req 21356544->21360639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760693.100239:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febfc975180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512524160:12345-192.168.203.56@tcp:4:dd.0 Request processed in 75202us (75488us total) trans 4294972533 rc 0/0 00000100:00100000:0.0:1789760693.100250:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5244 00000100:00100000:0.0:1789760693.100261:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 672516154032 time 672589321666 state 0x2 ref 0 00000100:00100000:0.0:1789760693.110031:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000001:00080000:2.0:1789760693.187027:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972534 is committed 00000100:00100000:2.0:1789760693.191265:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760693.191858:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512524416 00000100:00100000:0.0:1789760693.191869:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512524416 00000100:00100000:0.0:1789760693.191928:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 672589582082 time 672676220627 state 0x0 ref 1 00000100:00100000:0.0:1789760693.191943:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 672676481043 time 672676220627 state 0x1 ref 1 00000100:00100000:0.0:1789760693.192867:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 672676481043 time 672757159654 state 0x0 ref 1 00000100:00100000:0.0:1789760693.192887:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac3302680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512524416:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760693.192941:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21364736, mtime: 0x6aad94b4, atime: 0x0, ctime: 0x6aad94b4, blocks: 0xa807 00010000:00010000:0.0:1789760693.192950:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2cfca lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.192967:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=672s v=(0 0) 00010000:00010000:0.0:1789760693.192977:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760693.192984:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2cfca lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21364736->21368831] (req 21364736->21368831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.192999:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2cfca lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21364736->21368831] (req 21364736->21368831) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760693.235242:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21364736 -> 21368832 00010000:00010000:0.0:1789760693.235257:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2cfca lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21364736->21368831] (req 21364736->21368831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.235270:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2cfca lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21364736->21368831] (req 21364736->21368831) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.235279:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2cfca lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21364736->21368831] (req 21364736->21368831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.235287:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760693.235291:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2cfca lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21364736->21368831] (req 21364736->21368831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.235301:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2cfca lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21364736->21368831] (req 21364736->21368831) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.235319:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2cfca lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21364736->21368831] (req 21364736->21368831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.235327:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2cfca lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21364736->21368831] (req 21364736->21368831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760693.235443:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac3302680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512524416:12345-192.168.203.56@tcp:4:dd.0 Request processed in 42559us (44184us total) trans 4294972535 rc 0/0 00000100:00100000:0.0:1789760693.235454:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5246 00000100:00100000:0.0:1789760693.235464:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 672676481043 time 672757159654 state 0x2 ref 0 00000100:00100000:0.0:1789760693.239259:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512524544 00000100:00100000:0.0:1789760693.239269:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512524544 00000100:00100000:0.0:1789760693.239322:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 672676481043 time 672757159654 state 0x0 ref 1 00000100:00100000:0.0:1789760693.239337:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 672757420070 time 672757159654 state 0x1 ref 1 00000100:00100000:0.0:1789760693.239465:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 672757420070 time 672803763790 state 0x0 ref 1 00000100:00100000:0.0:1789760693.239478:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4133480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512524544:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760693.239525:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21368832, mtime: 0x6aad94b4, atime: 0x0, ctime: 0x6aad94b4, blocks: 0xa807 00010000:00010000:0.0:1789760693.239535:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2cfd1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.239551:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=672s v=(0 0) 00010000:00010000:0.0:1789760693.239556:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760693.239562:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2cfd1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21368832->21372927] (req 21368832->21372927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.239575:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2cfd1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21368832->21372927] (req 21368832->21372927) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760693.280935:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21368832 -> 21372928 00010000:00010000:0.0:1789760693.280950:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2cfd1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21368832->21372927] (req 21368832->21372927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.280963:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2cfd1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21368832->21372927] (req 21368832->21372927) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.280972:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2cfd1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21368832->21372927] (req 21368832->21372927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.280979:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760693.280983:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2cfd1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21368832->21372927] (req 21368832->21372927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.280992:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2cfd1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21368832->21372927] (req 21368832->21372927) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.281011:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2cfd1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21368832->21372927] (req 21368832->21372927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.281019:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2cfd1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21368832->21372927] (req 21368832->21372927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760693.281107:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4133480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512524544:12345-192.168.203.56@tcp:4:dd.0 Request processed in 41627us (42009us total) trans 4294972536 rc 0/0 00000100:00100000:0.0:1789760693.281120:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5247 00000100:00100000:0.0:1789760693.281129:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 672757420070 time 672803763790 state 0x2 ref 0 00000100:00100000:0.0:1789760693.284354:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512524672 00000100:00100000:0.0:1789760693.284365:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512524672 00000100:00100000:0.0:1789760693.284425:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 672757420070 time 672803763790 state 0x0 ref 1 00000100:00100000:0.0:1789760693.284441:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 672804024206 time 672803763790 state 0x1 ref 1 00000100:00100000:0.0:1789760693.284475:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 672804024206 time 672848774662 state 0x0 ref 1 00000100:00100000:0.0:1789760693.284496:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4130700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512524672:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760693.284554:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21372928, mtime: 0x6aad94b4, atime: 0x0, ctime: 0x6aad94b4, blocks: 0xa807 00010000:00010000:0.0:1789760693.284564:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2cfd8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.284582:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=672s v=(0 0) 00010000:00010000:0.0:1789760693.284587:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760693.284595:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2cfd8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21372928->21377023] (req 21372928->21377023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.284610:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2cfd8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21372928->21377023] (req 21372928->21377023) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760693.342919:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972537 is committed 00000100:00100000:2.0:1789760693.349164:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760693.349435:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512524800 00000100:00100000:0.0:1789760693.349446:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512524800 00000100:00100000:0.0:1789760693.349504:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 672804024206 time 672848774662 state 0x0 ref 1 00000100:00100000:0.0:1789760693.349520:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 672849035078 time 672848774662 state 0x1 ref 1 00000100:00100000:0.0:1789760693.349601:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 672849035078 time 672913899701 state 0x0 ref 1 00000100:00100000:0.0:1789760693.349615:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac3300e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512524800:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760693.349668:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21377024, mtime: 0x6aad94b4, atime: 0x0, ctime: 0x6aad94b4, blocks: 0xa807 00010000:00010000:0.0:1789760693.349678:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2cfdf lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.349753:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=672s v=(0 0) 00010000:00010000:0.0:1789760693.349758:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760693.349765:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2cfdf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21377024->21381119] (req 21377024->21381119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.349778:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2cfdf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21377024->21381119] (req 21377024->21381119) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760693.396631:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972538 is committed 00000100:00100000:3.0:1789760693.401679:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000001:00080000:2.0:1789760693.459608:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972539 is committed 00000100:00100000:3.0:1789760693.461610:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000001:00080000:2.0:1789760693.499987:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972540 is committed 00000100:00100000:2.0:1789760693.504261:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760693.515590:0:5848:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512525440 00000100:00100000:3.0:1789760693.515599:0:5848:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512525440 00000100:00100000:2.0:1789760693.515635:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512525312 00000100:00100000:2.0:1789760693.515640:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512525312 00000100:00100000:3.0:1789760693.515660:0:5848:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512525568 00000100:00100000:3.0:1789760693.515663:0:5848:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512525568 00000100:00100000:2.0:1789760693.515666:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 378 00000100:00100000:2.0:1789760693.515674:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febe00d4450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:e6cf7267-9b19-462a-a263-14115df15dab+13:2359:x1876699512525312:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:3.0:1789760693.515683:0:5848:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 424 00000100:00100000:3.0:1789760693.515689:0:5848:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4132a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+8:2359:x1876699512525440:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:2.0:1789760693.515769:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febe00d4450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:e6cf7267-9b19-462a-a263-14115df15dab+13:2359:x1876699512525312:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 94us (396us total) trans 0 rc 0/0 00000100:00080000:0.0:1789760693.515776:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789760693.515777:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 378 00000100:00100000:3.0:1789760693.516222:0:5848:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4132a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+8:2359:x1876699512525440:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 529us (902us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760693.516228:0:5848:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 424 00000100:00100000:3.0:1789760693.516244:0:5848:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 425 00000100:00100000:3.0:1789760693.516251:0:5848:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4131880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:d2c40150-e97b-40da-952b-cb9f18bf7e96+5:2359:x1876699512525568:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:3.0:1789760693.516278:0:5848:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4131880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:d2c40150-e97b-40da-952b-cb9f18bf7e96+5:2359:x1876699512525568:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 28us (875us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760693.516288:0:5848:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 425 00002000:00010000:0.0:1789760693.536685:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21389312 -> 21393408 00002000:00010000:0.0:1789760693.536732:0:13062:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760692 -> 1789760693 00002000:00010000:0.0:1789760693.536735:0:13062:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760692 -> 1789760693 00010000:00010000:0.0:1789760693.536749:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2cff4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21389312->21393407] (req 21389312->21393407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.536762:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2cff4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21389312->21393407] (req 21389312->21393407) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.536771:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2cff4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21389312->21393407] (req 21389312->21393407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.536778:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760693.536783:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2cff4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21389312->21393407] (req 21389312->21393407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.536791:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2cff4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21389312->21393407] (req 21389312->21393407) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.536810:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2cff4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21389312->21393407] (req 21389312->21393407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.536819:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2cff4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21389312->21393407] (req 21389312->21393407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760693.536907:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac3302d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2361:x1876699512525184:12345-192.168.203.56@tcp:4:dd.0 Request processed in 32283us (32715us total) trans 4294972541 rc 0/0 00000100:00100000:0.0:1789760693.536917:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5252 00000100:00100000:0.0:1789760693.536927:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 673026453082 time 673068906884 state 0x2 ref 0 00000001:00080000:2.0:1789760693.537004:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972541 is committed 00000100:00100000:0.0:1789760693.540435:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512525696 00000100:00100000:0.0:1789760693.540444:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512525696 00000100:00100000:0.0:1789760693.540496:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 673026453082 time 673068906884 state 0x0 ref 1 00000100:00100000:0.0:1789760693.540512:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 673069167300 time 673068906884 state 0x1 ref 1 00000100:00100000:0.0:1789760693.540544:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 673069167300 time 673104843170 state 0x0 ref 1 00000100:00100000:0.0:1789760693.540557:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4131180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512525696:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760693.540608:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21393408, mtime: 0x6aad94b5, atime: 0x0, ctime: 0x6aad94b5, blocks: 0xa807 00010000:00010000:0.0:1789760693.540617:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2cffb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.540631:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=673s v=(0 0) 00010000:00010000:0.0:1789760693.540636:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760693.540643:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2cffb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21393408->21397503] (req 21393408->21397503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.540657:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2cffb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21393408->21397503] (req 21393408->21397503) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760693.588107:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972542 is committed 00002000:00010000:0.0:1789760693.588311:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21393408 -> 21397504 00010000:00010000:0.0:1789760693.588329:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2cffb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21393408->21397503] (req 21393408->21397503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.588473:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2cffb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21393408->21397503] (req 21393408->21397503) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.588483:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2cffb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21393408->21397503] (req 21393408->21397503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.588491:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760693.588496:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2cffb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21393408->21397503] (req 21393408->21397503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.588506:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2cffb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21393408->21397503] (req 21393408->21397503) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.588525:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2cffb lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21393408->21397503] (req 21393408->21397503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.588532:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2cffb lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21393408->21397503] (req 21393408->21397503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760693.588671:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4131180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512525696:12345-192.168.203.56@tcp:4:dd.0 Request processed in 48112us (48332us total) trans 4294972542 rc 0/0 00000100:00100000:0.0:1789760693.588681:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5253 00000100:00100000:0.0:1789760693.588690:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 673069167300 time 673104843170 state 0x2 ref 0 00000100:00100000:2.0:1789760693.596496:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512525824 00000100:00100000:2.0:1789760693.596510:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512525824 00000100:00100000:2.0:1789760693.596565:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 673069167300 time 673104843170 state 0x0 ref 1 00000100:00100000:2.0:1789760693.596580:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 673105103586 time 673104843170 state 0x1 ref 1 00000100:00100000:2.0:1789760693.596608:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 673105103586 time 673160908132 state 0x0 ref 1 00000100:00100000:2.0:1789760693.596620:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4131500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512525824:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760693.596664:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21397504, mtime: 0x6aad94b5, atime: 0x0, ctime: 0x6aad94b5, blocks: 0xa807 00010000:00010000:2.0:1789760693.596673:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2d002 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760693.596689:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=673s v=(0 0) 00010000:00010000:2.0:1789760693.596716:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760693.596723:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2d002 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21397504->21401599] (req 21397504->21401599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760693.596736:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2d002 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21397504->21401599] (req 21397504->21401599) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.622893:0:5835:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:3.0:1789760693.644534:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972543 is committed 00000100:00100000:2.0:1789760693.649417:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760693.649782:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512525952 00000100:00100000:0.0:1789760693.649792:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512525952 00000100:00100000:0.0:1789760693.649848:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 673105103586 time 673160908132 state 0x0 ref 1 00000100:00100000:0.0:1789760693.649863:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 673161168548 time 673160908132 state 0x1 ref 1 00000100:00100000:0.0:1789760693.649889:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 673161168548 time 673214189018 state 0x0 ref 1 00000100:00100000:0.0:1789760693.649901:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac3303100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512525952:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760693.649955:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21401600, mtime: 0x6aad94b5, atime: 0x0, ctime: 0x6aad94b5, blocks: 0xa807 00010000:00010000:0.0:1789760693.649964:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2d009 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.649979:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=673s v=(0 0) 00010000:00010000:0.0:1789760693.649983:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760693.649989:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2d009 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21401600->21405695] (req 21401600->21405695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.650003:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2d009 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21401600->21405695] (req 21401600->21405695) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760693.699311:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972544 is committed 00000100:00100000:3.0:1789760693.701190:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760693.701371:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512526080 00000100:00100000:0.0:1789760693.701380:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512526080 00000100:00100000:0.0:1789760693.701498:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 673161168548 time 673214189018 state 0x0 ref 1 00000100:00100000:0.0:1789760693.701513:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 673214449434 time 673214189018 state 0x1 ref 1 00000100:00100000:0.0:1789760693.701544:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 673214449434 time 673265843691 state 0x0 ref 1 00000100:00100000:0.0:1789760693.701558:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac38b5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512526080:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760693.701606:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21405696, mtime: 0x6aad94b5, atime: 0x0, ctime: 0x6aad94b5, blocks: 0xa807 00010000:00010000:0.0:1789760693.701614:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2d010 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.701630:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=673s v=(0 0) 00010000:00010000:0.0:1789760693.701635:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760693.701641:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2d010 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21405696->21409791] (req 21405696->21409791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.701654:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2d010 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21405696->21409791] (req 21405696->21409791) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760693.745093:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972545 is committed 00000100:00100000:3.0:1789760693.748676:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512526208 00000100:00100000:3.0:1789760693.748686:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512526208 00000100:00100000:3.0:1789760693.748872:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 673214449434 time 673265843691 state 0x0 ref 1 00000100:00100000:3.0:1789760693.749146:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 673266104107 time 673265843691 state 0x1 ref 1 00000100:00100000:3.0:1789760693.749212:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 673266104107 time 673313510697 state 0x0 ref 1 00000100:00100000:3.0:1789760693.749226:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4130a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512526208:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760693.749274:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21409792, mtime: 0x6aad94b5, atime: 0x0, ctime: 0x6aad94b5, blocks: 0xa807 00010000:00010000:3.0:1789760693.749283:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2d017 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760693.749299:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=673s v=(0 0) 00010000:00010000:3.0:1789760693.749304:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760693.749310:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2d017 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21409792->21413887] (req 21409792->21413887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760693.749324:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2d017 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21409792->21413887] (req 21409792->21413887) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760693.796203:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972546 is committed 00000100:00100000:2.0:1789760693.800305:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00002000:00010000:3.0:1789760693.861739:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21413888 -> 21417984 00000001:00080000:0.0:1789760693.861754:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972547 is committed 00010000:00010000:3.0:1789760693.861759:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2d01e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21413888->21417983] (req 21413888->21417983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760693.861772:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2d01e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21413888->21417983] (req 21413888->21417983) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760693.861782:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2d01e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21413888->21417983] (req 21413888->21417983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760693.861789:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760693.861793:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2d01e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21413888->21417983] (req 21413888->21417983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760693.861801:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2d01e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21413888->21417983] (req 21413888->21417983) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760693.861821:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2d01e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21413888->21417983] (req 21413888->21417983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760693.861829:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2d01e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21413888->21417983] (req 21413888->21417983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760693.861922:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac3301c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512526336:12345-192.168.203.56@tcp:4:dd.0 Request processed in 61243us (61621us total) trans 4294972547 rc 0/0 00000100:00100000:3.0:1789760693.861936:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5258 00000100:00100000:3.0:1789760693.861946:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 673313771113 time 673364961291 state 0x2 ref 0 00000100:00100000:2.0:1789760693.864158:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000001:00080000:0.0:1789760693.916530:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972548 is committed 00002000:00010000:3.0:1789760693.919109:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21417984 -> 21422080 00010000:00010000:3.0:1789760693.919129:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2d025 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21417984->21422079] (req 21417984->21422079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760693.919141:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2d025 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21417984->21422079] (req 21417984->21422079) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760693.919150:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2d025 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21417984->21422079] (req 21417984->21422079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760693.919157:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760693.919163:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2d025 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21417984->21422079] (req 21417984->21422079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760693.919182:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2d025 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21417984->21422079] (req 21417984->21422079) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760693.919202:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2d025 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21417984->21422079] (req 21417984->21422079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760693.919211:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2d025 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21417984->21422079] (req 21417984->21422079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760693.919313:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4362d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512526464:12345-192.168.203.56@tcp:4:dd.0 Request processed in 54868us (55159us total) trans 4294972548 rc 0/0 00000100:00100000:3.0:1789760693.919323:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5259 00000100:00100000:3.0:1789760693.919333:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 673365221707 time 673428730423 state 0x2 ref 0 00000100:00100000:3.0:1789760693.922684:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512526592 00000100:00100000:3.0:1789760693.922722:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512526592 00000100:00100000:3.0:1789760693.922783:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 673365221707 time 673428730423 state 0x0 ref 1 00000100:00100000:3.0:1789760693.922797:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 673428990839 time 673428730423 state 0x1 ref 1 00000100:00100000:3.0:1789760693.922828:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 673428990839 time 673487128068 state 0x0 ref 1 00000100:00100000:3.0:1789760693.922840:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4132d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512526592:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760693.922885:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21422080, mtime: 0x6aad94b5, atime: 0x0, ctime: 0x6aad94b5, blocks: 0xa807 00010000:00010000:3.0:1789760693.922895:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2d02c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760693.922911:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=673s v=(0 0) 00010000:00010000:3.0:1789760693.922915:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760693.922921:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2d02c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21422080->21426175] (req 21422080->21426175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760693.922935:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2d02c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21422080->21426175] (req 21422080->21426175) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760693.973091:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972549 is committed 00002000:00010000:2.0:1789760693.973239:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21422080 -> 21426176 00010000:00010000:2.0:1789760693.973254:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2d02c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21422080->21426175] (req 21422080->21426175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760693.973272:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2d02c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21422080->21426175] (req 21422080->21426175) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760693.973280:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2d02c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21422080->21426175] (req 21422080->21426175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760693.973288:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760693.973292:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2d02c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21422080->21426175] (req 21422080->21426175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760693.973301:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2d02c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21422080->21426175] (req 21422080->21426175) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760693.973320:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2d02c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21422080->21426175] (req 21422080->21426175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760693.973327:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2d02c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21422080->21426175] (req 21422080->21426175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760693.973428:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4132d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512526592:12345-192.168.203.56@tcp:4:dd.0 Request processed in 50584us (50831us total) trans 4294972549 rc 0/0 00000100:00100000:2.0:1789760693.973438:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5260 00000100:00100000:2.0:1789760693.973448:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 673428990839 time 673487128068 state 0x2 ref 0 00000100:00100000:2.0:1789760693.976483:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760693.976639:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512526720 00000100:00100000:3.0:1789760693.976648:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512526720 00000100:00100000:3.0:1789760693.976733:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 673428990839 time 673487128068 state 0x0 ref 1 00000100:00100000:3.0:1789760693.976757:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 673487388484 time 673487128068 state 0x1 ref 1 00000100:00100000:3.0:1789760693.976785:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 673487388484 time 673541085032 state 0x0 ref 1 00000100:00100000:3.0:1789760693.976798:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4361180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512526720:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760693.976845:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21426176, mtime: 0x6aad94b5, atime: 0x0, ctime: 0x6aad94b5, blocks: 0xa807 00010000:00010000:3.0:1789760693.976853:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2d033 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760693.976869:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=673s v=(0 0) 00010000:00010000:3.0:1789760693.976874:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760693.976880:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2d033 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21426176->21430271] (req 21426176->21430271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760693.976893:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2d033 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21426176->21430271] (req 21426176->21430271) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760694.032719:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972550 is committed 00002000:00010000:2.0:1789760694.033939:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21426176 -> 21430272 00010000:00010000:2.0:1789760694.033957:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2d033 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21426176->21430271] (req 21426176->21430271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760694.033971:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2d033 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21426176->21430271] (req 21426176->21430271) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760694.033979:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2d033 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21426176->21430271] (req 21426176->21430271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760694.033986:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760694.033991:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2d033 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21426176->21430271] (req 21426176->21430271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760694.034000:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2d033 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21426176->21430271] (req 21426176->21430271) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760694.034018:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2d033 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21426176->21430271] (req 21426176->21430271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760694.034026:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2d033 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21426176->21430271] (req 21426176->21430271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760694.034141:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4361180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512526720:12345-192.168.203.56@tcp:4:dd.0 Request processed in 57341us (57664us total) trans 4294972550 rc 0/0 00000100:00100000:2.0:1789760694.034152:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5261 00000100:00100000:2.0:1789760694.034161:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 673487388484 time 673541085032 state 0x2 ref 0 00000100:00100000:3.0:1789760694.038482:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00002000:00010000:3.0:1789760694.118945:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21430272 -> 21434368 00010000:00010000:3.0:1789760694.118968:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2d03a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21430272->21434367] (req 21430272->21434367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760694.118982:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2d03a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21430272->21434367] (req 21430272->21434367) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760694.118991:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2d03a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21430272->21434367] (req 21430272->21434367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760694.118998:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760694.119003:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2d03a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21430272->21434367] (req 21430272->21434367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760694.119013:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2d03a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21430272->21434367] (req 21430272->21434367) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760694.119032:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2d03a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21430272->21434367] (req 21430272->21434367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760694.119039:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2d03a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21430272->21434367] (req 21430272->21434367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760694.119129:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac38b6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2358:x1876699512526848:12345-192.168.203.56@tcp:4:dd.0 Request processed in 80297us (80652us total) trans 4294972551 rc 0/0 00000100:00100000:3.0:1789760694.119139:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5262 00000100:00100000:3.0:1789760694.119150:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 673541345448 time 673603115481 state 0x2 ref 0 00000001:00080000:3.0:1789760694.119191:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972551 is committed 00000100:00100000:3.0:1789760694.126339:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760694.126481:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512526976 00000100:00100000:2.0:1789760694.126491:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512526976 00000100:00100000:2.0:1789760694.126543:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 673541345448 time 673603115481 state 0x0 ref 1 00000100:00100000:2.0:1789760694.126558:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 673603375897 time 673603115481 state 0x1 ref 1 00000100:00100000:2.0:1789760694.126590:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 673603375897 time 673690889438 state 0x0 ref 1 00000100:00100000:2.0:1789760694.126602:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac38b4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512526976:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760694.126648:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21434368, mtime: 0x6aad94b5, atime: 0x0, ctime: 0x6aad94b5, blocks: 0xa807 00010000:00010000:2.0:1789760694.126657:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2d041 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760694.126673:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=673s v=(0 0) 00010000:00010000:2.0:1789760694.126677:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760694.126684:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2d041 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21434368->21438463] (req 21434368->21438463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760694.126738:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2d041 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21434368->21438463] (req 21434368->21438463) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760694.223802:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972552 is committed 00002000:00010000:2.0:1789760694.233278:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21434368 -> 21438464 00010000:00010000:2.0:1789760694.233299:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2d041 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21434368->21438463] (req 21434368->21438463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760694.233318:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2d041 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21434368->21438463] (req 21434368->21438463) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760694.233327:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2d041 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21434368->21438463] (req 21434368->21438463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760694.233334:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760694.233338:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2d041 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21434368->21438463] (req 21434368->21438463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760694.233347:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2d041 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21434368->21438463] (req 21434368->21438463) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760694.233372:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2d041 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21434368->21438463] (req 21434368->21438463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760694.233380:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2d041 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21434368->21438463] (req 21434368->21438463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760694.233472:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac38b4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512526976:12345-192.168.203.56@tcp:4:dd.0 Request processed in 106868us (107139us total) trans 4294972552 rc 0/0 00000100:00100000:2.0:1789760694.233481:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5263 00000100:00100000:2.0:1789760694.233491:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 673603375897 time 673690889438 state 0x2 ref 0 00000100:00100000:2.0:1789760694.237682:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760694.238363:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512527104 00000100:00100000:3.0:1789760694.238373:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512527104 00000100:00100000:3.0:1789760694.238424:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 673603375897 time 673690889438 state 0x0 ref 1 00000100:00100000:3.0:1789760694.238438:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 673691149854 time 673690889438 state 0x1 ref 1 00000100:00100000:3.0:1789760694.238457:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 673691149854 time 673802757914 state 0x0 ref 1 00000100:00100000:3.0:1789760694.238467:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4363100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512527104:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760694.238658:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21438464, mtime: 0x6aad94b5, atime: 0x0, ctime: 0x6aad94b5, blocks: 0xa807 00010000:00010000:3.0:1789760694.238667:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2d048 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760694.238683:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=673s v=(0 0) 00010000:00010000:3.0:1789760694.238723:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760694.238898:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2d048 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21438464->21442559] (req 21438464->21442559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760694.238914:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2d048 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21438464->21442559] (req 21438464->21442559) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760694.340864:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972553 is committed 00002000:00010000:2.0:1789760694.340948:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21438464 -> 21442560 00010000:00010000:2.0:1789760694.340963:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2d048 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21438464->21442559] (req 21438464->21442559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760694.340975:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2d048 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21438464->21442559] (req 21438464->21442559) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760694.340983:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2d048 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21438464->21442559] (req 21438464->21442559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760694.340990:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760694.340994:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2d048 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21438464->21442559] (req 21438464->21442559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760694.341001:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2d048 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21438464->21442559] (req 21438464->21442559) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760694.341019:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2d048 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21438464->21442559] (req 21438464->21442559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760694.341026:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2d048 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21438464->21442559] (req 21438464->21442559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760694.341125:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4363100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512527104:12345-192.168.203.56@tcp:4:dd.0 Request processed in 102654us (103449us total) trans 4294972553 rc 0/0 00000100:00100000:2.0:1789760694.341135:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5264 00000100:00100000:2.0:1789760694.341145:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 673691149854 time 673802757914 state 0x2 ref 0 00000100:00100000:3.0:1789760694.347908:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000001:00080000:3.0:1789760694.442316:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972554 is committed 00000100:00100000:3.0:1789760694.445919:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760694.446072:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512527360 00000100:00100000:2.0:1789760694.446082:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512527360 00000100:00100000:2.0:1789760694.446138:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 673803018330 time 673912408699 state 0x0 ref 1 00000100:00100000:2.0:1789760694.446154:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 673912669115 time 673912408699 state 0x1 ref 1 00000100:00100000:2.0:1789760694.446188:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 673912669115 time 674010486913 state 0x0 ref 1 00000100:00100000:2.0:1789760694.446200:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac38b5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512527360:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760694.446247:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21446656, mtime: 0x6aad94b5, atime: 0x0, ctime: 0x6aad94b5, blocks: 0xa807 00010000:00010000:2.0:1789760694.446256:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2d056 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760694.446272:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=674s v=(0 0) 00010000:00010000:2.0:1789760694.446278:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760694.446284:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2d056 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21446656->21450751] (req 21446656->21450751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760694.446297:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2d056 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21446656->21450751] (req 21446656->21450751) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760694.517891:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972555 is committed 00000100:00100000:0.0:1789760694.533961:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760694.534066:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512527488 00000100:00100000:2.0:1789760694.534076:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512527488 00000100:00100000:2.0:1789760694.534141:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 673912669115 time 674010486913 state 0x0 ref 1 00000100:00100000:2.0:1789760694.534156:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 674010747329 time 674010486913 state 0x1 ref 1 00000100:00100000:2.0:1789760694.534195:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 674010747329 time 674098494442 state 0x0 ref 1 00000100:00100000:2.0:1789760694.534207:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febff929f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512527488:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760694.534254:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21450752, mtime: 0x6aad94b5, atime: 0x0, ctime: 0x6aad94b5, blocks: 0xa807 00010000:00010000:2.0:1789760694.534263:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2d05d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760694.534279:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=674s v=(0 0) 00010000:00010000:2.0:1789760694.534283:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760694.534289:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2d05d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21450752->21454847] (req 21450752->21454847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760694.534303:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2d05d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21450752->21454847] (req 21450752->21454847) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760694.592051:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972556 is committed 00000100:00100000:2.0:1789760694.597755:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00010000:00010000:0.0:1789760694.644486:0:7647:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:2.0:1789760694.658422:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21454848 -> 21458944 00010000:00010000:2.0:1789760694.658444:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2d064 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21454848->21458943] (req 21454848->21458943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760694.658456:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2d064 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21454848->21458943] (req 21454848->21458943) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760694.658464:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2d064 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21454848->21458943] (req 21454848->21458943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760694.658489:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760694.658494:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2d064 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21454848->21458943] (req 21454848->21458943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760694.658503:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2d064 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21454848->21458943] (req 21454848->21458943) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760694.658522:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2d064 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21454848->21458943] (req 21454848->21458943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760694.658529:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2d064 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21454848->21458943] (req 21454848->21458943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760694.658622:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4361500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2360:x1876699512527616:12345-192.168.203.56@tcp:4:dd.0 Request processed in 60702us (60880us total) trans 4294972557 rc 0/0 00000100:00100000:2.0:1789760694.658633:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5268 00000100:00100000:2.0:1789760694.658643:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 674098754858 time 674162204283 state 0x2 ref 0 00000001:00080000:0.0:1789760694.662747:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972557 is committed 00000100:00100000:3.0:1789760694.672830:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00002000:00010000:0.0:1789760694.746824:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21458944 -> 21463040 00010000:00010000:0.0:1789760694.746840:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2d06b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21458944->21463039] (req 21458944->21463039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.746854:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2d06b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21458944->21463039] (req 21458944->21463039) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.746863:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2d06b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21458944->21463039] (req 21458944->21463039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.746871:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760694.746876:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2d06b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21458944->21463039] (req 21458944->21463039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.746890:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2d06b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21458944->21463039] (req 21458944->21463039) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.746909:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2d06b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21458944->21463039] (req 21458944->21463039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.746917:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2d06b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21458944->21463039] (req 21458944->21463039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760694.747009:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febffb9f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512527744:12345-192.168.203.56@tcp:4:dd.0 Request processed in 73909us (74182us total) trans 4294972558 rc 0/0 00000100:00100000:0.0:1789760694.747020:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5269 00000100:00100000:0.0:1789760694.747031:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 674162464699 time 674237379542 state 0x2 ref 0 00000100:00100000:2.0:1789760694.752143:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512527872 00000100:00100000:2.0:1789760694.752153:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512527872 00000100:00100000:2.0:1789760694.752214:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 674162464699 time 674237379542 state 0x0 ref 1 00000100:00100000:2.0:1789760694.752229:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 674237639958 time 674237379542 state 0x1 ref 1 00000100:00100000:2.0:1789760694.752260:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 674237639958 time 674316559260 state 0x0 ref 1 00000100:00100000:2.0:1789760694.752276:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febea4b8850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512527872:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760694.752600:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21463040, mtime: 0x6aad94b6, atime: 0x0, ctime: 0x6aad94b6, blocks: 0xa807 00010000:00010000:2.0:1789760694.752612:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2d072 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760694.752628:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=674s v=(0 0) 00010000:00010000:2.0:1789760694.752633:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760694.752639:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2d072 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21463040->21467135] (req 21463040->21467135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760694.752654:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2d072 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21463040->21467135] (req 21463040->21467135) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760694.874123:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21463040 -> 21467136 00010000:00010000:2.0:1789760694.874146:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2d072 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21463040->21467135] (req 21463040->21467135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760694.874161:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2d072 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21463040->21467135] (req 21463040->21467135) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760694.874172:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2d072 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21463040->21467135] (req 21463040->21467135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760694.874180:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760694.874185:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2d072 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21463040->21467135] (req 21463040->21467135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760694.874193:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2d072 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21463040->21467135] (req 21463040->21467135) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760694.874213:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2d072 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21463040->21467135] (req 21463040->21467135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760694.874221:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2d072 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21463040->21467135] (req 21463040->21467135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760694.874314:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febea4b8850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2359:x1876699512527872:12345-192.168.203.56@tcp:4:dd.0 Request processed in 122036us (122271us total) trans 4294972559 rc 0/0 00000100:00100000:2.0:1789760694.874324:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5270 00000100:00100000:2.0:1789760694.874335:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 674237639958 time 674316559260 state 0x2 ref 0 00000001:00080000:0.0:1789760694.875251:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972559 is committed 00000100:00100000:2.0:1789760694.884346:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00002000:00010000:2.0:1789760694.885068:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21467136, mtime: 0x6aad94b6, atime: 0x0, ctime: 0x6aad94b6, blocks: 0xa807 00010000:00010000:2.0:1789760694.885079:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2d079 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760694.885096:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=674s v=(0 0) 00010000:00010000:2.0:1789760694.885101:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760694.885108:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2d079 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21467136->21471231] (req 21467136->21471231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760694.885121:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2d079 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21467136->21471231] (req 21467136->21471231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760694.994448:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21467136 -> 21471232 00010000:00010000:2.0:1789760694.994468:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2d079 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21467136->21471231] (req 21467136->21471231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760694.994577:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2d079 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21467136->21471231] (req 21467136->21471231) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760694.994587:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2d079 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21467136->21471231] (req 21467136->21471231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760694.994595:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760694.994600:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2d079 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21467136->21471231] (req 21467136->21471231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760694.994610:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2d079 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21467136->21471231] (req 21467136->21471231) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760694.994629:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2d079 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21467136->21471231] (req 21467136->21471231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760694.994637:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2d079 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21467136->21471231] (req 21467136->21471231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760694.994768:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4362d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2359:x1876699512528000:12345-192.168.203.56@tcp:4:dd.0 Request processed in 110008us (110428us total) trans 4294972560 rc 0/0 00000100:00100000:2.0:1789760694.994782:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5271 00000100:00100000:2.0:1789760694.994791:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 674316819676 time 674449045918 state 0x2 ref 0 00000001:00080000:0.0:1789760694.994870:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972560 is committed 00000100:00100000:0.0:1789760694.999508:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512528128 00000100:00100000:0.0:1789760694.999518:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512528128 00000100:00100000:0.0:1789760694.999568:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 674316819676 time 674449045918 state 0x0 ref 1 00000100:00100000:0.0:1789760694.999585:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 674449306334 time 674449045918 state 0x1 ref 1 00000100:00100000:0.0:1789760694.999614:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 674449306334 time 674563914094 state 0x0 ref 1 00000100:00100000:0.0:1789760694.999627:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4132300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512528128:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760694.999677:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21471232, mtime: 0x6aad94b6, atime: 0x0, ctime: 0x6aad94b6, blocks: 0xa807 00010000:00010000:0.0:1789760694.999724:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2d080 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.999741:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=674s v=(0 0) 00010000:00010000:0.0:1789760694.999745:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760694.999752:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2d080 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21471232->21475327] (req 21471232->21475327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.999765:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2d080 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21471232->21475327] (req 21471232->21475327) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760695.031252:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972561 is committed 00000100:00100000:3.0:1789760695.039774:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512528256 00000100:00100000:3.0:1789760695.039795:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512528256 00000100:00100000:3.0:1789760695.039862:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 674449306334 time 674563914094 state 0x0 ref 1 00000100:00100000:3.0:1789760695.039878:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 674564174510 time 674563914094 state 0x1 ref 1 00000100:00100000:3.0:1789760695.039904:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 674564174510 time 674604203915 state 0x0 ref 1 00000100:00100000:3.0:1789760695.039917:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4131c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512528256:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760695.039964:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21475328, mtime: 0x6aad94b6, atime: 0x0, ctime: 0x6aad94b6, blocks: 0xa807 00010000:00010000:3.0:1789760695.039973:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2d087 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760695.039989:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=674s v=(0 0) 00010000:00010000:3.0:1789760695.039994:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760695.040001:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2d087 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21475328->21479423] (req 21475328->21479423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760695.040016:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2d087 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21475328->21479423] (req 21475328->21479423) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760695.082338:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972562 is committed 00002000:00010000:2.0:1789760695.082489:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21475328 -> 21479424 00010000:00010000:2.0:1789760695.082504:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2d087 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21475328->21479423] (req 21475328->21479423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760695.082515:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2d087 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21475328->21479423] (req 21475328->21479423) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760695.082525:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2d087 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21475328->21479423] (req 21475328->21479423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760695.082534:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760695.082538:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2d087 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21475328->21479423] (req 21475328->21479423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760695.082547:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2d087 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21475328->21479423] (req 21475328->21479423) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760695.082564:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2d087 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21475328->21479423] (req 21475328->21479423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760695.082571:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2d087 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21475328->21479423] (req 21475328->21479423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760695.082668:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4131c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512528256:12345-192.168.203.56@tcp:4:dd.0 Request processed in 42749us (43030us total) trans 4294972562 rc 0/0 00000100:00100000:2.0:1789760695.082679:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5273 00000100:00100000:2.0:1789760695.082735:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 674564174510 time 674604203915 state 0x2 ref 0 00000100:00100000:3.0:1789760695.085514:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760695.086466:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512528384 00000100:00100000:0.0:1789760695.086477:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512528384 00000100:00100000:0.0:1789760695.086534:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 674564174510 time 674604203915 state 0x0 ref 1 00000100:00100000:0.0:1789760695.086547:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 674604464331 time 674604203915 state 0x1 ref 1 00000100:00100000:0.0:1789760695.086579:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 674604464331 time 674650878262 state 0x0 ref 1 00000100:00100000:0.0:1789760695.086590:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febffb9f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512528384:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760695.086632:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21479424, mtime: 0x6aad94b6, atime: 0x0, ctime: 0x6aad94b6, blocks: 0xa807 00010000:00010000:0.0:1789760695.086639:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2d08e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760695.086659:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=674s v=(0 0) 00010000:00010000:0.0:1789760695.086663:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760695.086669:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2d08e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21479424->21483519] (req 21479424->21483519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760695.086723:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2d08e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21479424->21483519] (req 21479424->21483519) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760695.150627:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21479424 -> 21483520 00010000:00010000:0.0:1789760695.150646:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2d08e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21479424->21483519] (req 21479424->21483519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760695.150658:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2d08e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21479424->21483519] (req 21479424->21483519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760695.150667:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2d08e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21479424->21483519] (req 21479424->21483519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760695.150675:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760695.150679:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2d08e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21479424->21483519] (req 21479424->21483519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760695.150727:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2d08e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21479424->21483519] (req 21479424->21483519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760695.150746:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2d08e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21479424->21483519] (req 21479424->21483519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760695.150755:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2d08e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21479424->21483519] (req 21479424->21483519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760695.150847:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febffb9f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2360:x1876699512528384:12345-192.168.203.56@tcp:4:dd.0 Request processed in 64255us (65338us total) trans 4294972563 rc 0/0 00000100:00100000:0.0:1789760695.150858:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5274 00000100:00100000:0.0:1789760695.150866:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 674604464331 time 674650878262 state 0x2 ref 0 00000001:00080000:2.0:1789760695.152565:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972563 is committed 00000100:00100000:3.0:1789760695.154256:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760695.154593:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512528512 00000100:00100000:0.0:1789760695.154603:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512528512 00000100:00100000:0.0:1789760695.154655:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 674604464331 time 674650878262 state 0x0 ref 1 00000100:00100000:0.0:1789760695.154669:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 674651138678 time 674650878262 state 0x1 ref 1 00000100:00100000:0.0:1789760695.154740:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 674651138678 time 674719039367 state 0x0 ref 1 00000100:00100000:0.0:1789760695.154751:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febffb9e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512528512:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760695.154806:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21483520, mtime: 0x6aad94b6, atime: 0x0, ctime: 0x6aad94b6, blocks: 0xa807 00010000:00010000:0.0:1789760695.154826:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2d095 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760695.154842:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=674s v=(0 0) 00010000:00010000:0.0:1789760695.154846:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760695.154853:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2d095 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21483520->21487615] (req 21483520->21487615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760695.154871:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2d095 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21483520->21487615] (req 21483520->21487615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760695.219062:0:3311:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8feac4361180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3311:1876699540214528:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1789760695.219062:0:3312:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8feac4363100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:6df98d54-eb52-4c5c-8fc9-ee9d74f90f98:3312:1876699540214656:0@lo:400:kworker.0 00000100:00100000:2.0:1789760695.219144:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760695.219199:0:3311:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8feac28a0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3311:1876699540215040:0@lo:400:kworker.0 00000100:00100000:2.0:1789760695.219226:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760695.220115:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760695.220151:0:3312:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8feac28a3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3312:1876699540215168:0@lo:400:kworker.0 00000100:00100000:3.0:1789760695.220178:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760695.220216:0:6707:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540214528 00000100:00100000:3.0:1789760695.220222:0:6707:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540214528 00000100:00100000:3.0:1789760695.220283:0:6707:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 178 00000100:00100000:3.0:1789760695.220293:0:6707:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febfbfe7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3311:x1876699540214528:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1789760695.220317:0:6707:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 528023552 pending 1703936 free 3836739584 avail 3825205248 00000100:00100000:3.0:1789760695.220483:0:6707:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febfbfe7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3311:x1876699540214528:12345-0@lo:13:osp-pre-0-0.0 Request processed in 190us (1347us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760695.220491:0:6707:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 178 00000100:00100000:3.0:1789760695.220516:0:6705:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540215040 00000100:00100000:3.0:1789760695.220520:0:6705:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540215040 00000100:00100000:3.0:1789760695.220557:0:6705:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 164 00000100:00100000:3.0:1789760695.220564:0:6705:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac38b2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3311:x1876699540215040:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760695.220596:0:6705:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac38b2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3311:x1876699540215040:12345-0@lo:400:kworker.0 Request processed in 33us (1369us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760695.220603:0:6705:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 164 00000100:00100000:3.0:1789760695.220621:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540214656 00000100:00100000:3.0:1789760695.220624:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540214656 00000100:00100000:3.0:1789760695.220648:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 379 00000100:00100000:3.0:1789760695.220654:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febe00d7850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:6df98d54-eb52-4c5c-8fc9-ee9d74f90f98+19:3312:x1876699540214656:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760695.220723:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febe00d7850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:6df98d54-eb52-4c5c-8fc9-ee9d74f90f98+19:3312:x1876699540214656:12345-0@lo:400:kworker.0 Request processed in 68us (609us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760695.220729:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 379 00000100:00100000:3.0:1789760695.220758:0:5848:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540215168 00000100:00100000:3.0:1789760695.220762:0:5848:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540215168 00000100:00100000:3.0:1789760695.220802:0:5848:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 426 00000100:00100000:3.0:1789760695.220809:0:5848:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febffb9d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3312:x1876699540215168:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760695.220840:0:3312:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8feac4363100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:6df98d54-eb52-4c5c-8fc9-ee9d74f90f98:3312:1876699540214656:0@lo:400:kworker.0 00000100:00100000:3.0:1789760695.220840:0:5848:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febffb9d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3312:x1876699540215168:12345-0@lo:400:kworker.0 Request processed in 32us (661us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760695.220846:0:5848:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 426 00000100:00100000:2.0:1789760695.220853:0:3312:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8feac28a3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3312:1876699540215168:0@lo:400:kworker.0 00000100:00080000:3.0:1789760695.220861:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:3.0:1789760695.220865:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1789760695.220888:0:3311:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8feac4361180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3311:1876699540214528:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1789760695.220898:0:3311:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8feac28a0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3311:1876699540215040:0@lo:400:kworker.0 00000100:00100000:0.0:1789760695.226389:0:3309:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8feac4361500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3309:1876699540214784:0@lo:400:kworker.0 00000100:00100000:0.0:1789760695.226464:0:3309:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760695.226514:0:3309:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8feac28a1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3309:1876699540215296:0@lo:400:kworker.0 00000100:00100000:0.0:1789760695.226541:0:3309:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760695.226567:0:3309:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1789760695.226573:0:3309:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8feac28a2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3309:1876699540214912:0@lo:400:kworker.0 00000100:00100000:0.0:1789760695.226595:0:3309:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760695.231309:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540214784 00000100:00100000:3.0:1789760695.231326:0:5847:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540214784 00000100:00100000:3.0:1789760695.231398:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540214912 00000100:00100000:3.0:1789760695.231402:0:5847:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540214912 00000100:00100000:3.0:1789760695.231441:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 427 00000100:00100000:3.0:1789760695.231453:0:5847:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febff92b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3309:x1876699540214784:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760695.231560:0:5847:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febff92b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3309:x1876699540214784:12345-0@lo:400:kworker.0 Request processed in 110us (5096us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760695.231568:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 427 00000100:00100000:3.0:1789760695.231586:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 428 00000100:00100000:3.0:1789760695.231593:0:5847:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febff929f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3309:x1876699540214912:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760695.231625:0:5847:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febff929f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3309:x1876699540214912:12345-0@lo:400:kworker.0 Request processed in 32us (5028us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760695.231631:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 428 00000100:00100000:3.0:1789760695.231652:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540215296 00000100:00100000:3.0:1789760695.231657:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540215296 00000100:00100000:3.0:1789760695.231715:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 165 00000100:00100000:3.0:1789760695.231724:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febff92aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3309:x1876699540215296:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760695.231756:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febff92aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3309:x1876699540215296:12345-0@lo:400:kworker.0 Request processed in 34us (5215us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760695.231762:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 165 00000100:00100000:3.0:1789760695.231823:0:3309:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8feac28a2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3309:1876699540214912:0@lo:400:kworker.0 00000100:00100000:3.0:1789760695.231837:0:3309:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8feac4361500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3309:1876699540214784:0@lo:400:kworker.0 00000100:00100000:3.0:1789760695.231847:0:3309:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8feac28a1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3309:1876699540215296:0@lo:400:kworker.0 00000100:00080000:3.0:1789760695.231887:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:3.0:1789760695.231891:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000001:00080000:3.0:1789760695.232003:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972564 is committed 00002000:00010000:3.0:1789760695.233615:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21483520 -> 21487616 00010000:00010000:3.0:1789760695.233632:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2d095 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21483520->21487615] (req 21483520->21487615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760695.233645:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2d095 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21483520->21487615] (req 21483520->21487615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760695.233655:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2d095 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21483520->21487615] (req 21483520->21487615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760695.233662:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760695.233667:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2d095 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21483520->21487615] (req 21483520->21487615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760695.233676:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2d095 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21483520->21487615] (req 21483520->21487615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760695.233743:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2d095 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21483520->21487615] (req 21483520->21487615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760695.233752:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2d095 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21483520->21487615] (req 21483520->21487615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760695.233883:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febffb9e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512528512:12345-192.168.203.56@tcp:4:dd.0 Request processed in 79129us (79631us total) trans 4294972564 rc 0/0 00000100:00100000:3.0:1789760695.233894:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5275 00000100:00100000:3.0:1789760695.233904:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 674651138678 time 674719039367 state 0x2 ref 0 00000100:00100000:3.0:1789760695.247839:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760695.248453:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512528640 00000100:00100000:2.0:1789760695.248464:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512528640 00000100:00100000:2.0:1789760695.248519:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 674651138678 time 674719039367 state 0x0 ref 1 00000100:00100000:2.0:1789760695.248534:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 674719299783 time 674719039367 state 0x1 ref 1 00000100:00100000:2.0:1789760695.248564:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 674719299783 time 674812864407 state 0x0 ref 1 00000100:00100000:2.0:1789760695.248576:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febffb9ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512528640:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760695.248620:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21487616, mtime: 0x6aad94b6, atime: 0x0, ctime: 0x6aad94b6, blocks: 0xa807 00010000:00010000:2.0:1789760695.248630:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2d09c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760695.248645:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=674s v=(0 0) 00010000:00010000:2.0:1789760695.248649:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760695.248655:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2d09c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21487616->21491711] (req 21487616->21491711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760695.248666:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2d09c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21487616->21491711] (req 21487616->21491711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760695.319056:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972565 is committed 00000100:00100000:3.0:1789760695.319149:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512528768 00000100:00100000:3.0:1789760695.319159:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512528768 00000100:00100000:3.0:1789760695.319208:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 674719299783 time 674812864407 state 0x0 ref 1 00000100:00100000:3.0:1789760695.319224:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 674813124823 time 674812864407 state 0x1 ref 1 00000100:00100000:3.0:1789760695.319255:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 674813124823 time 674883554242 state 0x0 ref 1 00000100:00100000:3.0:1789760695.319268:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4130e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512528768:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760695.319316:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21491712, mtime: 0x6aad94b6, atime: 0x0, ctime: 0x6aad94b6, blocks: 0xa807 00010000:00010000:3.0:1789760695.319326:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2d0a3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760695.319346:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=674s v=(0 0) 00010000:00010000:3.0:1789760695.319362:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760695.319370:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2d0a3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21491712->21495807] (req 21491712->21495807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760695.319389:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2d0a3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21491712->21495807] (req 21491712->21495807) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760695.361670:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972566 is committed 00002000:00010000:3.0:1789760695.361788:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21491712 -> 21495808 00010000:00010000:3.0:1789760695.361803:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2d0a3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21491712->21495807] (req 21491712->21495807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760695.361816:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2d0a3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21491712->21495807] (req 21491712->21495807) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760695.361825:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2d0a3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21491712->21495807] (req 21491712->21495807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760695.361832:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760695.361837:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2d0a3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21491712->21495807] (req 21491712->21495807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760695.361846:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2d0a3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21491712->21495807] (req 21491712->21495807) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760695.361864:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2d0a3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21491712->21495807] (req 21491712->21495807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760695.361872:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2d0a3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21491712->21495807] (req 21491712->21495807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760695.361962:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4130e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512528768:12345-192.168.203.56@tcp:4:dd.0 Request processed in 42693us (43011us total) trans 4294972566 rc 0/0 00000100:00100000:3.0:1789760695.361972:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5277 00000100:00100000:3.0:1789760695.361982:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 674813124823 time 674883554242 state 0x2 ref 0 00000100:00100000:3.0:1789760695.364217:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760695.364407:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512528896 00000100:00100000:0.0:1789760695.364417:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512528896 00000100:00100000:0.0:1789760695.364470:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 674813124823 time 674883554242 state 0x0 ref 1 00000100:00100000:0.0:1789760695.364484:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 674883814658 time 674883554242 state 0x1 ref 1 00000100:00100000:0.0:1789760695.364512:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 674883814658 time 674928811926 state 0x0 ref 1 00000100:00100000:0.0:1789760695.364525:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febffb9c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512528896:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760695.364572:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21495808, mtime: 0x6aad94b6, atime: 0x0, ctime: 0x6aad94b6, blocks: 0xa807 00010000:00010000:0.0:1789760695.364581:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2d0aa lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760695.364595:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=674s v=(0 0) 00010000:00010000:0.0:1789760695.364601:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760695.364607:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2d0aa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21495808->21499903] (req 21495808->21499903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760695.364619:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2d0aa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21495808->21499903] (req 21495808->21499903) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760695.411743:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972567 is committed 00002000:00010000:2.0:1789760695.411905:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21495808 -> 21499904 00010000:00010000:2.0:1789760695.411920:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2d0aa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21495808->21499903] (req 21495808->21499903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760695.411933:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2d0aa lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21495808->21499903] (req 21495808->21499903) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760695.411941:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2d0aa lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21495808->21499903] (req 21495808->21499903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760695.411948:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760695.411952:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2d0aa lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21495808->21499903] (req 21495808->21499903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760695.411961:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2d0aa lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21495808->21499903] (req 21495808->21499903) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760695.411985:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2d0aa lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21495808->21499903] (req 21495808->21499903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760695.411994:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2d0aa lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21495808->21499903] (req 21495808->21499903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760695.412181:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febffb9c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512528896:12345-192.168.203.56@tcp:4:dd.0 Request processed in 47653us (47966us total) trans 4294972567 rc 0/0 00000100:00100000:2.0:1789760695.412193:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5278 00000100:00100000:2.0:1789760695.412203:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 674883814658 time 674928811926 state 0x2 ref 0 00000100:00100000:3.0:1789760695.415045:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000001:00080000:2.0:1789760695.493122:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972568 is committed 00002000:00010000:3.0:1789760695.494218:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21499904 -> 21504000 00010000:00010000:3.0:1789760695.494238:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2d0b1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21499904->21503999] (req 21499904->21503999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760695.494251:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2d0b1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21499904->21503999] (req 21499904->21503999) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760695.494260:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2d0b1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21499904->21503999] (req 21499904->21503999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760695.494268:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760695.494273:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2d0b1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21499904->21503999] (req 21499904->21503999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760695.494281:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2d0b1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21499904->21503999] (req 21499904->21503999) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760695.494302:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2d0b1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21499904->21503999] (req 21499904->21503999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760695.494311:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2d0b1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21499904->21503999] (req 21499904->21503999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760695.494992:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febffb9c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512529024:12345-192.168.203.56@tcp:4:dd.0 Request processed in 79763us (79948us total) trans 4294972568 rc 0/0 00000100:00100000:3.0:1789760695.495002:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5279 00000100:00100000:3.0:1789760695.495011:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 674929072342 time 674979512623 state 0x2 ref 0 00000100:00100000:0.0:1789760695.498103:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760695.498248:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512529152 00000100:00100000:3.0:1789760695.498258:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512529152 00000100:00100000:3.0:1789760695.498308:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 674929072342 time 674979512623 state 0x0 ref 1 00000100:00100000:3.0:1789760695.498440:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 674979773039 time 674979512623 state 0x1 ref 1 00000100:00100000:3.0:1789760695.498967:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 674979773039 time 675062775556 state 0x0 ref 1 00000100:00100000:3.0:1789760695.498984:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febea771880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512529152:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760695.499032:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21504000, mtime: 0x6aad94b6, atime: 0x0, ctime: 0x6aad94b6, blocks: 0xa807 00010000:00010000:3.0:1789760695.499041:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2d0b8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760695.499056:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=675s v=(0 0) 00010000:00010000:3.0:1789760695.499062:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760695.499078:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2d0b8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21504000->21508095] (req 21504000->21508095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760695.499094:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2d0b8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21504000->21508095] (req 21504000->21508095) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760695.550140:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972569 is committed 00002000:00010000:3.0:1789760695.550963:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21504000 -> 21508096 00002000:00010000:3.0:1789760695.550968:0:6710:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760694 -> 1789760695 00002000:00010000:3.0:1789760695.550971:0:6710:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760694 -> 1789760695 00010000:00010000:3.0:1789760695.550984:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2d0b8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21504000->21508095] (req 21504000->21508095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760695.550995:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2d0b8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21504000->21508095] (req 21504000->21508095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760695.551005:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2d0b8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21504000->21508095] (req 21504000->21508095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760695.551012:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760695.551016:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2d0b8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21504000->21508095] (req 21504000->21508095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760695.551024:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2d0b8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21504000->21508095] (req 21504000->21508095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760695.551050:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2d0b8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21504000->21508095] (req 21504000->21508095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760695.551057:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2d0b8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21504000->21508095] (req 21504000->21508095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760695.551152:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febea771880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512529152:12345-192.168.203.56@tcp:4:dd.0 Request processed in 52168us (53054us total) trans 4294972569 rc 0/0 00000100:00100000:3.0:1789760695.551170:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5280 00000100:00100000:3.0:1789760695.551179:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 674979773039 time 675062775556 state 0x2 ref 0 00000100:00100000:2.0:1789760695.555017:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512529280 00000100:00100000:2.0:1789760695.555030:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512529280 00000100:00100000:2.0:1789760695.555088:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 674979773039 time 675062775556 state 0x0 ref 1 00000100:00100000:2.0:1789760695.555104:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 675063035972 time 675062775556 state 0x1 ref 1 00000100:00100000:2.0:1789760695.555138:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 675063035972 time 675119436980 state 0x0 ref 1 00000100:00100000:2.0:1789760695.555151:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4133100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512529280:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760695.555200:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21508096, mtime: 0x6aad94b7, atime: 0x0, ctime: 0x6aad94b7, blocks: 0xa807 00010000:00010000:2.0:1789760695.555210:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2d0bf lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760695.555231:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=675s v=(0 0) 00010000:00010000:2.0:1789760695.555235:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760695.555241:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2d0bf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21508096->21512191] (req 21508096->21512191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760695.555254:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2d0bf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21508096->21512191] (req 21508096->21512191) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760695.603734:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972570 is committed 00002000:00010000:3.0:1789760695.603973:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21508096 -> 21512192 00010000:00010000:3.0:1789760695.603987:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2d0bf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21508096->21512191] (req 21508096->21512191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760695.603999:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2d0bf lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21508096->21512191] (req 21508096->21512191) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760695.604007:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2d0bf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21508096->21512191] (req 21508096->21512191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760695.604013:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760695.604017:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2d0bf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21508096->21512191] (req 21508096->21512191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760695.604025:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2d0bf lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21508096->21512191] (req 21508096->21512191) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760695.604046:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2d0bf lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21508096->21512191] (req 21508096->21512191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760695.604053:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2d0bf lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21508096->21512191] (req 21508096->21512191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760695.604216:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4133100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512529280:12345-192.168.203.56@tcp:4:dd.0 Request processed in 49061us (49676us total) trans 4294972570 rc 0/0 00000100:00100000:3.0:1789760695.604227:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5281 00000100:00100000:3.0:1789760695.604236:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 675063035972 time 675119436980 state 0x2 ref 0 00000100:00100000:3.0:1789760695.609558:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512529408 00000100:00100000:3.0:1789760695.609567:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512529408 00000100:00100000:3.0:1789760695.609619:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 675063035972 time 675119436980 state 0x0 ref 1 00000100:00100000:3.0:1789760695.609634:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 675119697396 time 675119436980 state 0x1 ref 1 00000100:00100000:3.0:1789760695.609662:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 675119697396 time 675173961694 state 0x0 ref 1 00000100:00100000:3.0:1789760695.609675:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4131f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512529408:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760695.609768:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21512192, mtime: 0x6aad94b7, atime: 0x0, ctime: 0x6aad94b7, blocks: 0xa807 00010000:00010000:3.0:1789760695.609778:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2d0c6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760695.609794:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=675s v=(0 0) 00010000:00010000:3.0:1789760695.609799:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760695.609804:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2d0c6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21512192->21516287] (req 21512192->21516287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760695.609818:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2d0c6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21512192->21516287] (req 21512192->21516287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760695.669292:0:5836:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:2.0:1789760695.676882:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972571 is committed 00002000:00010000:2.0:1789760695.688064:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21512192 -> 21516288 00010000:00010000:2.0:1789760695.688092:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2d0c6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21512192->21516287] (req 21512192->21516287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760695.688105:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2d0c6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21512192->21516287] (req 21512192->21516287) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760695.688113:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2d0c6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21512192->21516287] (req 21512192->21516287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760695.688120:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760695.688125:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2d0c6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21512192->21516287] (req 21512192->21516287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760695.688133:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2d0c6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21512192->21516287] (req 21512192->21516287) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760695.688156:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2d0c6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21512192->21516287] (req 21512192->21516287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760695.688164:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2d0c6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21512192->21516287] (req 21512192->21516287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760695.688254:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4131f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512529408:12345-192.168.203.56@tcp:4:dd.0 Request processed in 78577us (78777us total) trans 4294972571 rc 0/0 00000100:00100000:2.0:1789760695.688264:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5282 00000100:00100000:2.0:1789760695.688273:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 675119697396 time 675173961694 state 0x2 ref 0 00000100:00100000:0.0:1789760695.692110:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760695.692199:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512529536 00000100:00100000:3.0:1789760695.692208:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512529536 00000100:00100000:3.0:1789760695.692261:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 675119697396 time 675173961694 state 0x0 ref 1 00000100:00100000:3.0:1789760695.692277:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 675174222110 time 675173961694 state 0x1 ref 1 00000100:00100000:3.0:1789760695.692309:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 675174222110 time 675256608655 state 0x0 ref 1 00000100:00100000:3.0:1789760695.692323:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febea772680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512529536:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760695.692372:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21516288, mtime: 0x6aad94b7, atime: 0x0, ctime: 0x6aad94b7, blocks: 0xa807 00010000:00010000:3.0:1789760695.692380:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2d0cd lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760695.692395:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=675s v=(0 0) 00010000:00010000:3.0:1789760695.692399:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760695.692406:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2d0cd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21516288->21520383] (req 21516288->21520383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760695.692419:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2d0cd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21516288->21520383] (req 21516288->21520383) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760695.730271:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972572 is committed 00002000:00010000:3.0:1789760695.730483:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21516288 -> 21520384 00010000:00010000:3.0:1789760695.730498:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2d0cd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21516288->21520383] (req 21516288->21520383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760695.730510:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2d0cd lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21516288->21520383] (req 21516288->21520383) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760695.730519:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2d0cd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21516288->21520383] (req 21516288->21520383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760695.730527:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760695.730532:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2d0cd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21516288->21520383] (req 21516288->21520383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760695.730541:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2d0cd lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21516288->21520383] (req 21516288->21520383) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760695.730559:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2d0cd lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21516288->21520383] (req 21516288->21520383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760695.730568:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2d0cd lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21516288->21520383] (req 21516288->21520383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760695.730742:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febea772680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512529536:12345-192.168.203.56@tcp:4:dd.0 Request processed in 38373us (38595us total) trans 4294972572 rc 0/0 00000100:00100000:3.0:1789760695.730756:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5283 00000100:00100000:3.0:1789760695.730765:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 675174222110 time 675256608655 state 0x2 ref 0 00000100:00100000:3.0:1789760695.745942:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512529664 00000100:00100000:3.0:1789760695.745952:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512529664 00000100:00100000:3.0:1789760695.746006:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 675174222110 time 675256608655 state 0x0 ref 1 00000100:00100000:3.0:1789760695.746022:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 675256869071 time 675256608655 state 0x1 ref 1 00000100:00100000:3.0:1789760695.746055:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 675256869071 time 675310354627 state 0x0 ref 1 00000100:00100000:3.0:1789760695.746067:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4130380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512529664:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760695.746113:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21520384, mtime: 0x6aad94b7, atime: 0x0, ctime: 0x6aad94b7, blocks: 0xa807 00010000:00010000:3.0:1789760695.746122:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2d0d4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760695.746137:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=675s v=(0 0) 00010000:00010000:3.0:1789760695.746142:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760695.746148:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2d0d4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21520384->21524479] (req 21520384->21524479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760695.746160:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2d0d4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21520384->21524479] (req 21520384->21524479) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760695.825023:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972573 is committed 00002000:00010000:3.0:1789760695.825764:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21520384 -> 21524480 00010000:00010000:3.0:1789760695.825781:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2d0d4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21520384->21524479] (req 21520384->21524479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760695.825793:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2d0d4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21520384->21524479] (req 21520384->21524479) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760695.825802:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2d0d4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21520384->21524479] (req 21520384->21524479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760695.825810:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760695.825816:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2d0d4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21520384->21524479] (req 21520384->21524479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760695.825826:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2d0d4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21520384->21524479] (req 21520384->21524479) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760695.825846:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2d0d4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21520384->21524479] (req 21520384->21524479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760695.825853:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2d0d4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21520384->21524479] (req 21520384->21524479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760695.826035:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4130380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512529664:12345-192.168.203.56@tcp:4:dd.0 Request processed in 79965us (80257us total) trans 4294972573 rc 0/0 00000100:00100000:3.0:1789760695.826045:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5284 00000100:00100000:3.0:1789760695.826053:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 675256869071 time 675310354627 state 0x2 ref 0 00000100:00100000:2.0:1789760695.829631:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000001:00080000:2.0:1789760695.899362:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972574 is committed 00000100:00100000:0.0:1789760695.903505:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760695.903588:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512529920 00000100:00100000:3.0:1789760695.903598:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512529920 00000100:00100000:3.0:1789760695.903645:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 675310615043 time 675394392312 state 0x0 ref 1 00000100:00100000:3.0:1789760695.903660:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 675394652728 time 675394392312 state 0x1 ref 1 00000100:00100000:3.0:1789760695.903790:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 675394652728 time 675468081893 state 0x0 ref 1 00000100:00100000:3.0:1789760695.903803:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febea770380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512529920:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760695.903847:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21528576, mtime: 0x6aad94b7, atime: 0x0, ctime: 0x6aad94b7, blocks: 0xa807 00010000:00010000:3.0:1789760695.903856:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2d0e2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760695.903871:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=675s v=(0 0) 00010000:00010000:3.0:1789760695.903876:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760695.903882:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2d0e2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21528576->21532671] (req 21528576->21532671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760695.903896:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2d0e2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21528576->21532671] (req 21528576->21532671) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760695.969097:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972575 is committed 00002000:00010000:0.0:1789760695.970649:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21528576 -> 21532672 00010000:00010000:0.0:1789760695.970667:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2d0e2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21528576->21532671] (req 21528576->21532671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760695.970692:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2d0e2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21528576->21532671] (req 21528576->21532671) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760695.970717:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2d0e2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21528576->21532671] (req 21528576->21532671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760695.970725:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760695.970730:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2d0e2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21528576->21532671] (req 21528576->21532671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760695.970738:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2d0e2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21528576->21532671] (req 21528576->21532671) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760695.970758:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2d0e2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21528576->21532671] (req 21528576->21532671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760695.970766:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2d0e2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21528576->21532671] (req 21528576->21532671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760695.970989:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febea770380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512529920:12345-192.168.203.56@tcp:4:dd.0 Request processed in 67182us (67487us total) trans 4294972575 rc 0/0 00000100:00100000:0.0:1789760695.971000:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5286 00000100:00100000:0.0:1789760695.971010:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 675394652728 time 675468081893 state 0x2 ref 0 00000100:00100000:2.0:1789760695.978545:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760695.978628:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512530048 00000100:00100000:3.0:1789760695.978638:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512530048 00000100:00100000:3.0:1789760695.978727:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 675394652728 time 675468081893 state 0x0 ref 1 00000100:00100000:3.0:1789760695.978743:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 675468342309 time 675468081893 state 0x1 ref 1 00000100:00100000:3.0:1789760695.978769:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 675468342309 time 675543069341 state 0x0 ref 1 00000100:00100000:3.0:1789760695.978791:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac38b3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512530048:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760695.978837:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21532672, mtime: 0x6aad94b7, atime: 0x0, ctime: 0x6aad94b7, blocks: 0xa807 00010000:00010000:3.0:1789760695.978846:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2d0e9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760695.978860:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=675s v=(0 0) 00010000:00010000:3.0:1789760695.978865:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760695.978870:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2d0e9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21532672->21536767] (req 21532672->21536767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760695.978882:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2d0e9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21532672->21536767] (req 21532672->21536767) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760696.095722:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972576 is committed 00000100:00100000:0.0:1789760696.099640:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000001:00080000:3.0:1789760696.143188:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972577 is committed 00000100:00100000:2.0:1789760696.146461:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760696.146541:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512530304 00000100:00100000:0.0:1789760696.146550:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512530304 00000100:00100000:0.0:1789760696.146597:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 675543329757 time 675664149791 state 0x0 ref 1 00000100:00100000:0.0:1789760696.146611:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 675664410207 time 675664149791 state 0x1 ref 1 00000100:00100000:0.0:1789760696.146664:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 675664410207 time 675710963846 state 0x0 ref 1 00000100:00100000:0.0:1789760696.146685:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac38b0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512530304:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760696.146803:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21540864, mtime: 0x6aad94b7, atime: 0x0, ctime: 0x6aad94b7, blocks: 0xa807 00010000:00010000:0.0:1789760696.146813:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2d0f7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.146833:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=675s v=(0 0) 00010000:00010000:0.0:1789760696.146837:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760696.146843:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2d0f7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21540864->21544959] (req 21540864->21544959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.146856:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2d0f7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21540864->21544959] (req 21540864->21544959) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760696.180299:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972578 is committed 00002000:00010000:0.0:1789760696.180502:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21540864 -> 21544960 00010000:00010000:0.0:1789760696.180517:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2d0f7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21540864->21544959] (req 21540864->21544959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.180530:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2d0f7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21540864->21544959] (req 21540864->21544959) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.180539:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2d0f7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21540864->21544959] (req 21540864->21544959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.180547:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760696.180551:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2d0f7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21540864->21544959] (req 21540864->21544959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.180561:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2d0f7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21540864->21544959] (req 21540864->21544959) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.180581:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2d0f7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21540864->21544959] (req 21540864->21544959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.180589:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2d0f7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21540864->21544959] (req 21540864->21544959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760696.180840:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac38b0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512530304:12345-192.168.203.56@tcp:4:dd.0 Request processed in 34160us (34381us total) trans 4294972578 rc 0/0 00000100:00100000:0.0:1789760696.180851:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5289 00000100:00100000:0.0:1789760696.180861:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 675664410207 time 675710963846 state 0x2 ref 0 00000100:00100000:0.0:1789760696.185614:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000001:00080000:0.0:1789760696.233186:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972579 is committed 00002000:00010000:3.0:1789760696.233266:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21544960 -> 21549056 00010000:00010000:3.0:1789760696.233283:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2d0fe lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21544960->21549055] (req 21544960->21549055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760696.233297:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2d0fe lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21544960->21549055] (req 21544960->21549055) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760696.233307:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2d0fe lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21544960->21549055] (req 21544960->21549055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760696.233453:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760696.233459:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2d0fe lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21544960->21549055] (req 21544960->21549055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760696.233470:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2d0fe lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21544960->21549055] (req 21544960->21549055) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760696.233490:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2d0fe lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21544960->21549055] (req 21544960->21549055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760696.233498:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2d0fe lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21544960->21549055] (req 21544960->21549055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760696.233686:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4217100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512530432:12345-192.168.203.56@tcp:4:dd.0 Request processed in 47658us (48069us total) trans 4294972579 rc 0/0 00000100:00100000:3.0:1789760696.233739:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5290 00000100:00100000:3.0:1789760696.233748:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 675711224262 time 675750302657 state 0x2 ref 0 00000100:00100000:2.0:1789760696.237251:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512530560 00000100:00100000:2.0:1789760696.237260:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512530560 00000100:00100000:2.0:1789760696.237310:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 675711224262 time 675750302657 state 0x0 ref 1 00000100:00100000:2.0:1789760696.237450:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 675750563073 time 675750302657 state 0x1 ref 1 00000100:00100000:2.0:1789760696.237578:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 675750563073 time 675801875387 state 0x0 ref 1 00000100:00100000:2.0:1789760696.237596:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4133b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512530560:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760696.237641:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21549056, mtime: 0x6aad94b7, atime: 0x0, ctime: 0x6aad94b7, blocks: 0xa807 00010000:00010000:2.0:1789760696.237650:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2d105 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760696.237666:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=675s v=(0 0) 00010000:00010000:2.0:1789760696.237670:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760696.237688:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2d105 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21549056->21553151] (req 21549056->21553151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760696.237730:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2d105 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21549056->21553151] (req 21549056->21553151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760696.279563:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000001:00080000:0.0:1789760696.279962:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972580 is committed 00000001:00080000:0.0:1789760696.317245:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972581 is committed 00000100:00100000:3.0:1789760696.321948:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512530816 00000100:00100000:3.0:1789760696.321958:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512530816 00000100:00100000:3.0:1789760696.322012:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 675802135803 time 675844735804 state 0x0 ref 1 00000100:00100000:3.0:1789760696.322027:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 675844996220 time 675844735804 state 0x1 ref 1 00000100:00100000:3.0:1789760696.322188:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 675844996220 time 675886486287 state 0x0 ref 1 00000100:00100000:3.0:1789760696.322202:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4133800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512530816:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760696.322252:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21557248, mtime: 0x6aad94b7, atime: 0x0, ctime: 0x6aad94b7, blocks: 0xa807 00010000:00010000:3.0:1789760696.322262:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2d113 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760696.322277:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=675s v=(0 0) 00010000:00010000:3.0:1789760696.322282:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760696.322288:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2d113 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21557248->21561343] (req 21557248->21561343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760696.322302:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2d113 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21557248->21561343] (req 21557248->21561343) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760696.364348:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972582 is committed 00002000:00010000:0.0:1789760696.364503:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21557248 -> 21561344 00010000:00010000:0.0:1789760696.364522:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2d113 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21557248->21561343] (req 21557248->21561343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.364536:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2d113 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21557248->21561343] (req 21557248->21561343) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.364544:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2d113 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21557248->21561343] (req 21557248->21561343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.364551:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760696.364556:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2d113 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21557248->21561343] (req 21557248->21561343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.364571:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2d113 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21557248->21561343] (req 21557248->21561343) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.364590:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2d113 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21557248->21561343] (req 21557248->21561343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.364598:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2d113 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21557248->21561343] (req 21557248->21561343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760696.364738:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4133800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512530816:12345-192.168.203.56@tcp:4:dd.0 Request processed in 42497us (42923us total) trans 4294972582 rc 0/0 00000100:00100000:0.0:1789760696.364749:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5293 00000100:00100000:0.0:1789760696.364758:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 675844996220 time 675886486287 state 0x2 ref 0 00000100:00100000:0.0:1789760696.368140:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000001:00080000:2.0:1789760696.407032:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972583 is committed 00002000:00010000:0.0:1789760696.407455:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21561344 -> 21565440 00010000:00010000:0.0:1789760696.407471:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2d11a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21561344->21565439] (req 21561344->21565439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.407484:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2d11a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21561344->21565439] (req 21561344->21565439) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.407494:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2d11a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21561344->21565439] (req 21561344->21565439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.407507:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760696.407512:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2d11a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21561344->21565439] (req 21561344->21565439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.407520:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2d11a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21561344->21565439] (req 21561344->21565439) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.407540:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2d11a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21561344->21565439] (req 21561344->21565439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.407548:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2d11a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21561344->21565439] (req 21561344->21565439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760696.407686:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac437e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512530944:12345-192.168.203.56@tcp:4:dd.0 Request processed in 39199us (39543us total) trans 4294972583 rc 0/0 00000100:00100000:0.0:1789760696.407743:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5294 00000100:00100000:0.0:1789760696.407753:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 675886746703 time 675932760596 state 0x2 ref 0 00000100:00100000:2.0:1789760696.411553:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000001:00080000:3.0:1789760696.450600:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972584 is committed 00000100:00100000:0.0:1789760696.455522:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512531200 00000100:00100000:0.0:1789760696.455533:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512531200 00000100:00100000:0.0:1789760696.455586:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 675933021012 time 675976309886 state 0x0 ref 1 00000100:00100000:0.0:1789760696.455602:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 675976570302 time 675976309886 state 0x1 ref 1 00000100:00100000:0.0:1789760696.455627:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 675976570302 time 676019926543 state 0x0 ref 1 00000100:00100000:0.0:1789760696.455643:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4130000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512531200:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760696.455722:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21569536, mtime: 0x6aad94b7, atime: 0x0, ctime: 0x6aad94b7, blocks: 0xa807 00010000:00010000:0.0:1789760696.455731:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2d128 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.455747:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=676s v=(0 0) 00010000:00010000:0.0:1789760696.455752:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760696.455758:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2d128 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21569536->21573631] (req 21569536->21573631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.455782:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2d128 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21569536->21573631] (req 21569536->21573631) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760696.498276:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972585 is committed 00002000:00010000:0.0:1789760696.498435:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21569536 -> 21573632 00010000:00010000:0.0:1789760696.498453:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2d128 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21569536->21573631] (req 21569536->21573631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.498467:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2d128 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21569536->21573631] (req 21569536->21573631) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.498476:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2d128 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21569536->21573631] (req 21569536->21573631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.498484:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760696.498489:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2d128 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21569536->21573631] (req 21569536->21573631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.498498:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2d128 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21569536->21573631] (req 21569536->21573631) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.498518:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2d128 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21569536->21573631] (req 21569536->21573631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.498526:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2d128 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21569536->21573631] (req 21569536->21573631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760696.498654:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4130000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512531200:12345-192.168.203.56@tcp:4:dd.0 Request processed in 43008us (43364us total) trans 4294972585 rc 0/0 00000100:00100000:0.0:1789760696.498665:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5296 00000100:00100000:0.0:1789760696.498683:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 675976570302 time 676019926543 state 0x2 ref 0 00000100:00100000:0.0:1789760696.502239:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000001:00080000:0.0:1789760696.535967:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972586 is committed 00002000:00010000:2.0:1789760696.536074:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21573632 -> 21577728 00002000:00010000:2.0:1789760696.536079:0:13062:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760695 -> 1789760696 00002000:00010000:2.0:1789760696.536081:0:13062:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760695 -> 1789760696 00010000:00010000:2.0:1789760696.536095:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2d12f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21573632->21577727] (req 21573632->21577727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760696.536107:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2d12f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21573632->21577727] (req 21573632->21577727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760696.536122:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2d12f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21573632->21577727] (req 21573632->21577727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760696.536130:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760696.536135:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2d12f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21573632->21577727] (req 21573632->21577727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760696.536145:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2d12f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21573632->21577727] (req 21573632->21577727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760696.536164:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2d12f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21573632->21577727] (req 21573632->21577727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760696.536171:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2d12f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21573632->21577727] (req 21573632->21577727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760696.536260:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac437e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512531328:12345-192.168.203.56@tcp:4:dd.0 Request processed in 33583us (34026us total) trans 4294972586 rc 0/0 00000100:00100000:2.0:1789760696.536270:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5297 00000100:00100000:2.0:1789760696.536279:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 676020186959 time 676066958280 state 0x2 ref 0 00000100:00100000:2.0:1789760696.539448:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000001:00080000:2.0:1789760696.577656:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972587 is committed 00002000:00010000:0.0:1789760696.577796:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21577728 -> 21581824 00010000:00010000:0.0:1789760696.577812:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2d136 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21577728->21581823] (req 21577728->21581823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.577826:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2d136 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21577728->21581823] (req 21577728->21581823) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.577835:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2d136 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21577728->21581823] (req 21577728->21581823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.577848:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760696.577854:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2d136 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21577728->21581823] (req 21577728->21581823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.577863:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2d136 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21577728->21581823] (req 21577728->21581823) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.577882:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2d136 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21577728->21581823] (req 21577728->21581823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.577890:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2d136 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21577728->21581823] (req 21577728->21581823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760696.577981:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac38b2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512531456:12345-192.168.203.56@tcp:4:dd.0 Request processed in 38320us (38536us total) trans 4294972587 rc 0/0 00000100:00100000:0.0:1789760696.577992:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5298 00000100:00100000:0.0:1789760696.578002:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 676067218696 time 676103944884 state 0x2 ref 0 00000100:00100000:0.0:1789760696.581048:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760696.581181:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512531584 00000100:00100000:3.0:1789760696.581189:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512531584 00000100:00100000:3.0:1789760696.581249:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 676067218696 time 676103944884 state 0x0 ref 1 00000100:00100000:3.0:1789760696.581264:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 676104205300 time 676103944884 state 0x1 ref 1 00000100:00100000:3.0:1789760696.581297:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 676104205300 time 676145595937 state 0x0 ref 1 00000100:00100000:3.0:1789760696.581309:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac437ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512531584:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760696.581364:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21581824, mtime: 0x6aad94b8, atime: 0x0, ctime: 0x6aad94b8, blocks: 0xa807 00010000:00010000:3.0:1789760696.581373:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2d13d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760696.581393:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=676s v=(0 0) 00010000:00010000:3.0:1789760696.581397:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760696.581403:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2d13d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21581824->21585919] (req 21581824->21585919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760696.581417:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2d13d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21581824->21585919] (req 21581824->21585919) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760696.611124:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21581824 -> 21585920 00010000:00010000:3.0:1789760696.611143:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2d13d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21581824->21585919] (req 21581824->21585919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760696.611157:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2d13d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21581824->21585919] (req 21581824->21585919) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760696.611166:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2d13d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21581824->21585919] (req 21581824->21585919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760696.611174:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760696.611179:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2d13d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21581824->21585919] (req 21581824->21585919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760696.611188:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2d13d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21581824->21585919] (req 21581824->21585919) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760696.611207:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2d13d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21581824->21585919] (req 21581824->21585919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760696.611215:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2d13d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21581824->21585919] (req 21581824->21585919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760696.611310:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac437ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2361:x1876699512531584:12345-192.168.203.56@tcp:4:dd.0 Request processed in 29998us (30266us total) trans 4294972588 rc 0/0 00000100:00100000:3.0:1789760696.611321:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5299 00000100:00100000:3.0:1789760696.611331:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 676104205300 time 676145595937 state 0x2 ref 0 00000100:00100000:2.0:1789760696.614593:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760696.614747:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512531712 00000100:00100000:3.0:1789760696.614756:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512531712 00000100:00100000:3.0:1789760696.614809:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 676104205300 time 676145595937 state 0x0 ref 1 00000100:00100000:3.0:1789760696.614825:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 676145856353 time 676145595937 state 0x1 ref 1 00000100:00100000:3.0:1789760696.614849:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 676145856353 time 676179148494 state 0x0 ref 1 00000100:00100000:3.0:1789760696.614862:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac38b1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2361:x1876699512531712:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760696.614907:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21585920, mtime: 0x6aad94b8, atime: 0x0, ctime: 0x6aad94b8, blocks: 0xa807 00010000:00010000:3.0:1789760696.614915:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2d144 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760696.614930:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=676s v=(0 0) 00010000:00010000:3.0:1789760696.614934:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760696.614941:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2d144 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21585920->21590015] (req 21585920->21590015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760696.614959:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2d144 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21585920->21590015] (req 21585920->21590015) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760696.615715:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972588 is committed 00002000:00010000:0.0:1789760696.646018:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21585920 -> 21590016 00010000:00010000:0.0:1789760696.646033:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2d144 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21585920->21590015] (req 21585920->21590015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.646046:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2d144 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21585920->21590015] (req 21585920->21590015) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.646056:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2d144 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21585920->21590015] (req 21585920->21590015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.646063:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760696.646068:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2d144 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21585920->21590015] (req 21585920->21590015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.646077:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2d144 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21585920->21590015] (req 21585920->21590015) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.646096:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2d144 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21585920->21590015] (req 21585920->21590015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.646104:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2d144 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21585920->21590015] (req 21585920->21590015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760696.646501:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac38b1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512531712:12345-192.168.203.56@tcp:4:dd.0 Request processed in 31637us (31912us total) trans 4294972589 rc 0/0 00000100:00100000:0.0:1789760696.646513:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5300 00000100:00100000:0.0:1789760696.646523:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 676145856353 time 676179148494 state 0x2 ref 0 00000100:00100000:2.0:1789760696.649631:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760696.649755:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512531840 00000100:00100000:0.0:1789760696.649765:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512531840 00000100:00100000:0.0:1789760696.649830:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 676145856353 time 676179148494 state 0x0 ref 1 00000100:00100000:0.0:1789760696.649846:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 676179408910 time 676179148494 state 0x1 ref 1 00000100:00100000:0.0:1789760696.649877:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 676179408910 time 676214177055 state 0x0 ref 1 00000100:00100000:0.0:1789760696.649890:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac38b1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512531840:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760696.649943:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21590016, mtime: 0x6aad94b8, atime: 0x0, ctime: 0x6aad94b8, blocks: 0xa807 00010000:00010000:0.0:1789760696.649952:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2d14b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.649968:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=676s v=(0 0) 00010000:00010000:0.0:1789760696.649972:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760696.649978:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2d14b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21590016->21594111] (req 21590016->21594111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.649992:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2d14b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21590016->21594111] (req 21590016->21594111) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760696.694211:0:5835:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:0.0:1789760696.698666:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972590 is committed 00002000:00010000:0.0:1789760696.698758:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21590016 -> 21594112 00010000:00010000:0.0:1789760696.698842:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2d14b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21590016->21594111] (req 21590016->21594111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.698856:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2d14b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21590016->21594111] (req 21590016->21594111) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.698865:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2d14b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21590016->21594111] (req 21590016->21594111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.698878:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760696.698883:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2d14b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21590016->21594111] (req 21590016->21594111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.698891:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2d14b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21590016->21594111] (req 21590016->21594111) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.698911:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2d14b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21590016->21594111] (req 21590016->21594111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.698918:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2d14b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21590016->21594111] (req 21590016->21594111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760696.698997:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac38b1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512531840:12345-192.168.203.56@tcp:4:dd.0 Request processed in 49105us (49373us total) trans 4294972590 rc 0/0 00000100:00100000:0.0:1789760696.699007:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5301 00000100:00100000:0.0:1789760696.699016:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 676179408910 time 676214177055 state 0x2 ref 0 00000100:00100000:0.0:1789760696.702513:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760696.702606:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512531968 00000100:00100000:3.0:1789760696.702615:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512531968 00000100:00100000:3.0:1789760696.702667:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 676179408910 time 676214177055 state 0x0 ref 1 00000100:00100000:3.0:1789760696.702846:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 676214437471 time 676214177055 state 0x1 ref 1 00000100:00100000:3.0:1789760696.702957:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 676214437471 time 676267255380 state 0x0 ref 1 00000100:00100000:3.0:1789760696.702971:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac437df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512531968:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760696.703019:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21594112, mtime: 0x6aad94b8, atime: 0x0, ctime: 0x6aad94b8, blocks: 0xa807 00010000:00010000:3.0:1789760696.703028:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2d152 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760696.703043:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=676s v=(0 0) 00010000:00010000:3.0:1789760696.703047:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760696.703053:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2d152 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21594112->21598207] (req 21594112->21598207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760696.703080:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2d152 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21594112->21598207] (req 21594112->21598207) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760696.745619:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972591 is committed 00002000:00010000:3.0:1789760696.745829:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21594112 -> 21598208 00010000:00010000:3.0:1789760696.745845:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2d152 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21594112->21598207] (req 21594112->21598207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760696.745858:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2d152 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21594112->21598207] (req 21594112->21598207) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760696.745868:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2d152 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21594112->21598207] (req 21594112->21598207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760696.745876:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760696.745881:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2d152 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21594112->21598207] (req 21594112->21598207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760696.745890:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2d152 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21594112->21598207] (req 21594112->21598207) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760696.745908:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2d152 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21594112->21598207] (req 21594112->21598207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760696.745916:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2d152 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21594112->21598207] (req 21594112->21598207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760696.747075:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac437df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512531968:12345-192.168.203.56@tcp:4:dd.0 Request processed in 44091us (44554us total) trans 4294972591 rc 0/0 00000100:00100000:3.0:1789760696.747091:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5302 00000100:00100000:3.0:1789760696.747102:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 676214437471 time 676267255380 state 0x2 ref 0 00000100:00100000:0.0:1789760696.750960:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760696.751059:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512532096 00000100:00100000:3.0:1789760696.751069:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512532096 00000100:00100000:3.0:1789760696.751122:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 676214437471 time 676267255380 state 0x0 ref 1 00000100:00100000:3.0:1789760696.751136:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 676267515796 time 676267255380 state 0x1 ref 1 00000100:00100000:3.0:1789760696.751365:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 676267515796 time 676315663063 state 0x0 ref 1 00000100:00100000:3.0:1789760696.751380:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac437fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512532096:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760696.751433:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21598208, mtime: 0x6aad94b8, atime: 0x0, ctime: 0x6aad94b8, blocks: 0xa807 00010000:00010000:3.0:1789760696.751442:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2d159 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760696.751458:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=676s v=(0 0) 00010000:00010000:3.0:1789760696.751463:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760696.751470:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2d159 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21598208->21602303] (req 21598208->21602303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760696.751484:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2d159 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21598208->21602303] (req 21598208->21602303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760696.812221:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972592 is committed 00002000:00010000:3.0:1789760696.812385:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21598208 -> 21602304 00010000:00010000:3.0:1789760696.812400:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2d159 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21598208->21602303] (req 21598208->21602303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760696.812415:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2d159 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21598208->21602303] (req 21598208->21602303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760696.812427:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2d159 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21598208->21602303] (req 21598208->21602303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760696.812433:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760696.812437:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2d159 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21598208->21602303] (req 21598208->21602303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760696.812446:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2d159 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21598208->21602303] (req 21598208->21602303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760696.812464:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2d159 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21598208->21602303] (req 21598208->21602303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760696.812470:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2d159 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21598208->21602303] (req 21598208->21602303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760696.812560:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac437fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512532096:12345-192.168.203.56@tcp:4:dd.0 Request processed in 61178us (61604us total) trans 4294972592 rc 0/0 00000100:00100000:3.0:1789760696.812569:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5303 00000100:00100000:3.0:1789760696.812578:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 676267515796 time 676315663063 state 0x2 ref 0 00000100:00100000:0.0:1789760696.816656:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00002000:00010000:2.0:1789760696.869441:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21602304 -> 21606400 00010000:00010000:2.0:1789760696.869470:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2d160 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21602304->21606399] (req 21602304->21606399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760696.869484:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2d160 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21602304->21606399] (req 21602304->21606399) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760696.869493:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2d160 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21602304->21606399] (req 21602304->21606399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760696.869500:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760696.869504:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2d160 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21602304->21606399] (req 21602304->21606399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760696.869513:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2d160 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21602304->21606399] (req 21602304->21606399) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760696.869533:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2d160 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21602304->21606399] (req 21602304->21606399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760696.869540:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2d160 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21602304->21606399] (req 21602304->21606399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760696.869684:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac437f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2360:x1876699512532224:12345-192.168.203.56@tcp:4:dd.0 Request processed in 52385us (53033us total) trans 4294972593 rc 0/0 00000100:00100000:2.0:1789760696.869733:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5304 00000100:00100000:2.0:1789760696.869744:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 676315923479 time 676381582820 state 0x2 ref 0 00000001:00080000:0.0:1789760696.872220:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972593 is committed 00000100:00100000:0.0:1789760696.876885:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760696.876969:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512532352 00000100:00100000:3.0:1789760696.876980:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512532352 00000100:00100000:3.0:1789760696.877036:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 676315923479 time 676381582820 state 0x0 ref 1 00000100:00100000:3.0:1789760696.877051:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 676381843236 time 676381582820 state 0x1 ref 1 00000100:00100000:3.0:1789760696.877097:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 676381843236 time 676441392566 state 0x0 ref 1 00000100:00100000:3.0:1789760696.877111:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac437d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512532352:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760696.877160:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21606400, mtime: 0x6aad94b8, atime: 0x0, ctime: 0x6aad94b8, blocks: 0xa807 00010000:00010000:3.0:1789760696.877170:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2d167 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760696.877184:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=676s v=(0 0) 00010000:00010000:3.0:1789760696.877188:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760696.877195:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2d167 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21606400->21610495] (req 21606400->21610495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760696.877210:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2d167 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21606400->21610495] (req 21606400->21610495) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760696.919880:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21606400 -> 21610496 00010000:00010000:0.0:1789760696.919896:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2d167 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21606400->21610495] (req 21606400->21610495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.919909:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2d167 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21606400->21610495] (req 21606400->21610495) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.919919:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2d167 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21606400->21610495] (req 21606400->21610495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.919926:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760696.919930:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2d167 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21606400->21610495] (req 21606400->21610495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.919939:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2d167 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21606400->21610495] (req 21606400->21610495) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.919958:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2d167 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21606400->21610495] (req 21606400->21610495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.919966:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2d167 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21606400->21610495] (req 21606400->21610495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760696.922006:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac437d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512532352:12345-192.168.203.56@tcp:4:dd.0 Request processed in 42945us (43177us total) trans 4294972594 rc 0/0 00000100:00100000:0.0:1789760696.922028:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5305 00000100:00100000:0.0:1789760696.922038:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 676381843236 time 676441392566 state 0x2 ref 0 00000100:00100000:3.0:1789760696.923943:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760696.924180:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512532480 00000100:00100000:0.0:1789760696.924190:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512532480 00000100:00100000:0.0:1789760696.924241:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 676381843236 time 676441392566 state 0x0 ref 1 00000100:00100000:0.0:1789760696.924258:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 676441652982 time 676441392566 state 0x1 ref 1 00000100:00100000:0.0:1789760696.924332:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 676441652982 time 676488631165 state 0x0 ref 1 00000100:00100000:0.0:1789760696.924447:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febffb9d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512532480:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760696.924494:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21610496, mtime: 0x6aad94b8, atime: 0x0, ctime: 0x6aad94b8, blocks: 0xa807 00010000:00010000:0.0:1789760696.924503:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2d16e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.924518:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=676s v=(0 0) 00010000:00010000:0.0:1789760696.924523:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760696.924529:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2d16e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21610496->21614591] (req 21610496->21614591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.924542:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2d16e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21610496->21614591] (req 21610496->21614591) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760696.971631:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21610496 -> 21614592 00010000:00010000:3.0:1789760696.971645:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2d16e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21610496->21614591] (req 21610496->21614591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760696.971658:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2d16e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21610496->21614591] (req 21610496->21614591) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760696.971683:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2d16e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21610496->21614591] (req 21610496->21614591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760696.971691:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760696.971720:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2d16e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21610496->21614591] (req 21610496->21614591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760696.971730:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2d16e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21610496->21614591] (req 21610496->21614591) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760696.971749:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2d16e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21610496->21614591] (req 21610496->21614591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760696.971756:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2d16e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21610496->21614591] (req 21610496->21614591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760696.971854:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febffb9d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512532480:12345-192.168.203.56@tcp:4:dd.0 Request processed in 47408us (47913us total) trans 4294972595 rc 0/0 00000100:00100000:3.0:1789760696.971865:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5306 00000100:00100000:3.0:1789760696.971875:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 676441652982 time 676488631165 state 0x2 ref 0 00000100:00100000:2.0:1789760696.975465:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512532608 00000100:00100000:2.0:1789760696.975475:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512532608 00000100:00100000:2.0:1789760696.975525:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 676441652982 time 676488631165 state 0x0 ref 1 00000100:00100000:2.0:1789760696.975540:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 676488891581 time 676488631165 state 0x1 ref 1 00000100:00100000:2.0:1789760696.975574:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 676488891581 time 676539872984 state 0x0 ref 1 00000100:00100000:2.0:1789760696.975587:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4133480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512532608:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760696.975630:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21614592, mtime: 0x6aad94b8, atime: 0x0, ctime: 0x6aad94b8, blocks: 0xa807 00010000:00010000:2.0:1789760696.975638:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2d175 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760696.975652:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=676s v=(0 0) 00010000:00010000:2.0:1789760696.975656:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760696.975662:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2d175 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21614592->21618687] (req 21614592->21618687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760696.975685:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2d175 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21614592->21618687] (req 21614592->21618687) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760697.018206:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972596 is committed 00002000:00010000:3.0:1789760697.019323:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21614592 -> 21618688 00010000:00010000:3.0:1789760697.019345:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2d175 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21614592->21618687] (req 21614592->21618687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760697.019364:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2d175 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21614592->21618687] (req 21614592->21618687) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760697.019373:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2d175 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21614592->21618687] (req 21614592->21618687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760697.019380:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760697.019385:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2d175 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21614592->21618687] (req 21614592->21618687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760697.019394:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2d175 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21614592->21618687] (req 21614592->21618687) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760697.019412:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2d175 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21614592->21618687] (req 21614592->21618687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760697.019420:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2d175 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21614592->21618687] (req 21614592->21618687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760697.019511:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4133480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512532608:12345-192.168.203.56@tcp:4:dd.0 Request processed in 43922us (44240us total) trans 4294972596 rc 0/0 00000100:00100000:3.0:1789760697.019520:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5307 00000100:00100000:3.0:1789760697.019529:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 676488891581 time 676539872984 state 0x2 ref 0 00000100:00100000:0.0:1789760697.023649:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512532736 00000100:00100000:0.0:1789760697.023660:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512532736 00000100:00100000:0.0:1789760697.023777:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 676488891581 time 676539872984 state 0x0 ref 1 00000100:00100000:0.0:1789760697.023793:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 676540133400 time 676539872984 state 0x1 ref 1 00000100:00100000:0.0:1789760697.023829:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 676540133400 time 676588128486 state 0x0 ref 1 00000100:00100000:0.0:1789760697.023841:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4130700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512532736:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760697.023888:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21618688, mtime: 0x6aad94b8, atime: 0x0, ctime: 0x6aad94b8, blocks: 0xa807 00010000:00010000:0.0:1789760697.023897:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2d17c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.023913:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=676s v=(0 0) 00010000:00010000:0.0:1789760697.023918:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760697.023923:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2d17c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21618688->21622783] (req 21618688->21622783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.023937:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2d17c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21618688->21622783] (req 21618688->21622783) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1789760697.024644:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:2.0:1789760697.059766:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972597 is committed 00002000:00010000:2.0:1789760697.059827:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21618688 -> 21622784 00010000:00010000:2.0:1789760697.059841:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2d17c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21618688->21622783] (req 21618688->21622783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760697.059861:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2d17c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21618688->21622783] (req 21618688->21622783) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760697.059871:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2d17c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21618688->21622783] (req 21618688->21622783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760697.059880:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760697.059885:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2d17c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21618688->21622783] (req 21618688->21622783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760697.059895:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2d17c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21618688->21622783] (req 21618688->21622783) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760697.059915:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2d17c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21618688->21622783] (req 21618688->21622783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760697.059924:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2d17c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21618688->21622783] (req 21618688->21622783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760697.060018:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4130700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512532736:12345-192.168.203.56@tcp:4:dd.0 Request processed in 36174us (36625us total) trans 4294972597 rc 0/0 00000100:00100000:2.0:1789760697.060029:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5308 00000100:00100000:2.0:1789760697.060039:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 676540133400 time 676588128486 state 0x2 ref 0 00000100:00100000:3.0:1789760697.063154:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000001:00080000:2.0:1789760697.095278:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972598 is committed 00002000:00010000:3.0:1789760697.095527:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21622784 -> 21626880 00010000:00010000:3.0:1789760697.095544:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2d183 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21622784->21626879] (req 21622784->21626879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760697.095557:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2d183 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21622784->21626879] (req 21622784->21626879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760697.095566:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2d183 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21622784->21626879] (req 21622784->21626879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760697.095573:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760697.095578:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2d183 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21622784->21626879] (req 21622784->21626879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760697.095587:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2d183 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21622784->21626879] (req 21622784->21626879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760697.095605:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2d183 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21622784->21626879] (req 21622784->21626879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760697.095614:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2d183 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21622784->21626879] (req 21622784->21626879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760697.095788:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febffb9f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512532864:12345-192.168.203.56@tcp:4:dd.0 Request processed in 32316us (32638us total) trans 4294972598 rc 0/0 00000100:00100000:3.0:1789760697.095799:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5309 00000100:00100000:3.0:1789760697.095808:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 676588388902 time 676627757876 state 0x2 ref 0 00000100:00100000:3.0:1789760697.102162:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512532992 00000100:00100000:3.0:1789760697.102172:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512532992 00000100:00100000:3.0:1789760697.102224:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 676588388902 time 676627757876 state 0x0 ref 1 00000100:00100000:3.0:1789760697.102240:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 676628018292 time 676627757876 state 0x1 ref 1 00000100:00100000:3.0:1789760697.102326:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 676628018292 time 676666624167 state 0x0 ref 1 00000100:00100000:3.0:1789760697.102457:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4131180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512532992:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760697.102506:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21626880, mtime: 0x6aad94b8, atime: 0x0, ctime: 0x6aad94b8, blocks: 0xa807 00010000:00010000:3.0:1789760697.102515:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2d18a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760697.102530:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=676s v=(0 0) 00010000:00010000:3.0:1789760697.102534:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760697.102540:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2d18a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21626880->21630975] (req 21626880->21630975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760697.102552:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2d18a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21626880->21630975] (req 21626880->21630975) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760697.148109:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972599 is committed 00002000:00010000:3.0:1789760697.148324:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21626880 -> 21630976 00010000:00010000:3.0:1789760697.148348:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2d18a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21626880->21630975] (req 21626880->21630975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760697.148367:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2d18a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21626880->21630975] (req 21626880->21630975) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760697.148376:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2d18a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21626880->21630975] (req 21626880->21630975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760697.148384:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760697.148389:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2d18a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21626880->21630975] (req 21626880->21630975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760697.148398:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2d18a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21626880->21630975] (req 21626880->21630975) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760697.148416:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2d18a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21626880->21630975] (req 21626880->21630975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760697.148424:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2d18a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21626880->21630975] (req 21626880->21630975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760697.148517:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4131180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512532992:12345-192.168.203.56@tcp:4:dd.0 Request processed in 46175us (46515us total) trans 4294972599 rc 0/0 00000100:00100000:3.0:1789760697.148528:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5310 00000100:00100000:3.0:1789760697.148537:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 676628018292 time 676666624167 state 0x2 ref 0 00000100:00100000:2.0:1789760697.151823:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512533120 00000100:00100000:2.0:1789760697.151832:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512533120 00000100:00100000:2.0:1789760697.151884:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 676628018292 time 676666624167 state 0x0 ref 1 00000100:00100000:2.0:1789760697.151900:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 676666884583 time 676666624167 state 0x1 ref 1 00000100:00100000:2.0:1789760697.151931:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 676666884583 time 676716229625 state 0x0 ref 1 00000100:00100000:2.0:1789760697.151944:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4131500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512533120:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760697.151990:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21630976, mtime: 0x6aad94b8, atime: 0x0, ctime: 0x6aad94b8, blocks: 0xa807 00010000:00010000:2.0:1789760697.151999:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2d191 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760697.152013:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=676s v=(0 0) 00010000:00010000:2.0:1789760697.152017:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760697.152023:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2d191 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21630976->21635071] (req 21630976->21635071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760697.152036:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2d191 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21630976->21635071] (req 21630976->21635071) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760697.206659:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972600 is committed 00002000:00010000:3.0:1789760697.206890:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21630976 -> 21635072 00010000:00010000:3.0:1789760697.206906:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2d191 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21630976->21635071] (req 21630976->21635071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760697.206918:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2d191 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21630976->21635071] (req 21630976->21635071) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760697.206927:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2d191 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21630976->21635071] (req 21630976->21635071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760697.206934:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760697.206939:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2d191 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21630976->21635071] (req 21630976->21635071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760697.206948:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2d191 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21630976->21635071] (req 21630976->21635071) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760697.206967:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2d191 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21630976->21635071] (req 21630976->21635071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760697.206975:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2d191 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21630976->21635071] (req 21630976->21635071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760697.207075:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4131500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512533120:12345-192.168.203.56@tcp:4:dd.0 Request processed in 55129us (55502us total) trans 4294972600 rc 0/0 00000100:00100000:3.0:1789760697.207084:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5311 00000100:00100000:3.0:1789760697.207093:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 676666884583 time 676716229625 state 0x2 ref 0 00000100:00100000:0.0:1789760697.212821:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760697.212893:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512533248 00000100:00100000:3.0:1789760697.212903:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512533248 00000100:00100000:3.0:1789760697.212956:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 676666884583 time 676716229625 state 0x0 ref 1 00000100:00100000:3.0:1789760697.212971:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 676716490041 time 676716229625 state 0x1 ref 1 00000100:00100000:3.0:1789760697.213002:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 676716490041 time 676777301890 state 0x0 ref 1 00000100:00100000:3.0:1789760697.213015:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2fbb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512533248:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760697.213122:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21635072, mtime: 0x6aad94b8, atime: 0x0, ctime: 0x6aad94b8, blocks: 0xa807 00010000:00010000:3.0:1789760697.213132:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2d198 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760697.213148:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=676s v=(0 0) 00010000:00010000:3.0:1789760697.213153:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760697.213160:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2d198 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21635072->21639167] (req 21635072->21639167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760697.213174:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2d198 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21635072->21639167] (req 21635072->21639167) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760697.256146:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972601 is committed 00002000:00010000:3.0:1789760697.256285:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21635072 -> 21639168 00010000:00010000:3.0:1789760697.256300:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2d198 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21635072->21639167] (req 21635072->21639167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760697.256324:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2d198 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21635072->21639167] (req 21635072->21639167) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760697.256333:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2d198 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21635072->21639167] (req 21635072->21639167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760697.256348:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760697.256353:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2d198 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21635072->21639167] (req 21635072->21639167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760697.256362:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2d198 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21635072->21639167] (req 21635072->21639167) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760697.256380:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2d198 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21635072->21639167] (req 21635072->21639167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760697.256387:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2d198 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21635072->21639167] (req 21635072->21639167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760697.256481:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2fbb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512533248:12345-192.168.203.56@tcp:4:dd.0 Request processed in 43463us (43665us total) trans 4294972601 rc 0/0 00000100:00100000:3.0:1789760697.256492:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5312 00000100:00100000:3.0:1789760697.256501:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 676716490041 time 676777301890 state 0x2 ref 0 00000100:00100000:0.0:1789760697.259986:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760697.260485:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512533376 00000100:00100000:3.0:1789760697.260496:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512533376 00000100:00100000:3.0:1789760697.260547:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 676716490041 time 676777301890 state 0x0 ref 1 00000100:00100000:3.0:1789760697.260562:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 676777562306 time 676777301890 state 0x1 ref 1 00000100:00100000:3.0:1789760697.260595:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 676777562306 time 676824894553 state 0x0 ref 1 00000100:00100000:3.0:1789760697.260606:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2fb8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512533376:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760697.260649:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21639168, mtime: 0x6aad94b8, atime: 0x0, ctime: 0x6aad94b8, blocks: 0xa807 00010000:00010000:3.0:1789760697.260657:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2d19f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760697.260682:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=676s v=(0 0) 00010000:00010000:3.0:1789760697.260686:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760697.260692:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2d19f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21639168->21643263] (req 21639168->21643263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760697.261210:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2d19f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21639168->21643263] (req 21639168->21643263) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760697.307350:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972602 is committed 00002000:00010000:2.0:1789760697.307426:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21639168 -> 21643264 00010000:00010000:2.0:1789760697.307440:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2d19f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21639168->21643263] (req 21639168->21643263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760697.307457:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2d19f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21639168->21643263] (req 21639168->21643263) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760697.307466:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2d19f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21639168->21643263] (req 21639168->21643263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760697.307473:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760697.307477:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2d19f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21639168->21643263] (req 21639168->21643263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760697.307487:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2d19f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21639168->21643263] (req 21639168->21643263) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760697.307506:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2d19f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21639168->21643263] (req 21639168->21643263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760697.307515:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2d19f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21639168->21643263] (req 21639168->21643263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760697.307603:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2fb8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512533376:12345-192.168.203.56@tcp:4:dd.0 Request processed in 46993us (47620us total) trans 4294972602 rc 0/0 00000100:00100000:2.0:1789760697.307613:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5313 00000100:00100000:2.0:1789760697.307623:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 676777562306 time 676824894553 state 0x2 ref 0 00000100:00100000:2.0:1789760697.316589:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760697.316685:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512533504 00000100:00100000:3.0:1789760697.316730:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512533504 00000100:00100000:3.0:1789760697.316781:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 676777562306 time 676824894553 state 0x0 ref 1 00000100:00100000:3.0:1789760697.316797:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 676825154969 time 676824894553 state 0x1 ref 1 00000100:00100000:3.0:1789760697.316827:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 676825154969 time 676881127369 state 0x0 ref 1 00000100:00100000:3.0:1789760697.316840:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac28a1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512533504:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760697.316888:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21643264, mtime: 0x6aad94b8, atime: 0x0, ctime: 0x6aad94b8, blocks: 0xa807 00010000:00010000:3.0:1789760697.316897:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2d1a6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760697.316912:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=676s v=(0 0) 00010000:00010000:3.0:1789760697.316917:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760697.316923:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2d1a6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21643264->21647359] (req 21643264->21647359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760697.316936:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2d1a6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21643264->21647359] (req 21643264->21647359) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760697.382071:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972603 is committed 00000100:00100000:0.0:1789760697.384608:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760697.384873:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512533632 00000100:00100000:0.0:1789760697.384882:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512533632 00000100:00100000:0.0:1789760697.384929:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 676825154969 time 676881127369 state 0x0 ref 1 00000100:00100000:0.0:1789760697.384943:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 676881387785 time 676881127369 state 0x1 ref 1 00000100:00100000:0.0:1789760697.384972:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 676881387785 time 676949271956 state 0x0 ref 1 00000100:00100000:0.0:1789760697.384984:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2fb9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512533632:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760697.385028:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21647360, mtime: 0x6aad94b8, atime: 0x0, ctime: 0x6aad94b8, blocks: 0xa807 00010000:00010000:0.0:1789760697.385036:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2d1ad lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.385050:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=676s v=(0 0) 00010000:00010000:0.0:1789760697.385054:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760697.385071:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2d1ad lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21647360->21651455] (req 21647360->21651455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.385085:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2d1ad lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21647360->21651455] (req 21647360->21651455) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760697.428633:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972604 is committed 00002000:00010000:0.0:1789760697.428763:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21647360 -> 21651456 00010000:00010000:0.0:1789760697.428779:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2d1ad lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21647360->21651455] (req 21647360->21651455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.428792:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2d1ad lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21647360->21651455] (req 21647360->21651455) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.428802:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2d1ad lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21647360->21651455] (req 21647360->21651455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.428808:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760697.428813:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2d1ad lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21647360->21651455] (req 21647360->21651455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.428937:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2d1ad lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21647360->21651455] (req 21647360->21651455) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.428956:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2d1ad lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21647360->21651455] (req 21647360->21651455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.428963:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2d1ad lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21647360->21651455] (req 21647360->21651455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760697.433887:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2fb9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512533632:12345-192.168.203.56@tcp:4:dd.0 Request processed in 48893us (49275us total) trans 4294972604 rc 0/0 00000100:00100000:0.0:1789760697.433904:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5315 00000100:00100000:0.0:1789760697.433914:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 676881387785 time 676949271956 state 0x2 ref 0 00000100:00100000:0.0:1789760697.440106:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760697.443466:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512533760 00000100:00100000:2.0:1789760697.443489:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512533760 00000100:00100000:2.0:1789760697.443552:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 676881387785 time 676949271956 state 0x0 ref 1 00000100:00100000:2.0:1789760697.443568:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 676949532372 time 676949271956 state 0x1 ref 1 00000100:00100000:2.0:1789760697.443605:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 676949532372 time 677007904516 state 0x0 ref 1 00000100:00100000:2.0:1789760697.443618:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2fbaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512533760:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760697.443683:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21651456, mtime: 0x6aad94b8, atime: 0x0, ctime: 0x6aad94b8, blocks: 0xa807 00010000:00010000:2.0:1789760697.443692:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2d1b4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760697.443739:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=677s v=(0 0) 00010000:00010000:2.0:1789760697.443743:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760697.443750:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2d1b4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21651456->21655551] (req 21651456->21655551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760697.443765:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2d1b4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21651456->21655551] (req 21651456->21655551) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760697.495547:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972605 is committed 00002000:00010000:0.0:1789760697.495908:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21651456 -> 21655552 00010000:00010000:0.0:1789760697.495930:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2d1b4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21651456->21655551] (req 21651456->21655551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.495957:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2d1b4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21651456->21655551] (req 21651456->21655551) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.495968:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2d1b4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21651456->21655551] (req 21651456->21655551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.495981:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760697.495987:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2d1b4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21651456->21655551] (req 21651456->21655551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.495997:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2d1b4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21651456->21655551] (req 21651456->21655551) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.496024:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2d1b4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21651456->21655551] (req 21651456->21655551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.496033:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2d1b4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21651456->21655551] (req 21651456->21655551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760697.496159:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2fbaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512533760:12345-192.168.203.56@tcp:4:dd.0 Request processed in 52535us (56053us total) trans 4294972605 rc 0/0 00000100:00100000:0.0:1789760697.496171:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5316 00000100:00100000:0.0:1789760697.496183:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 676949532372 time 677007904516 state 0x2 ref 0 00000100:00100000:2.0:1789760697.504491:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00002000:00010000:0.0:1789760697.539616:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21655552 -> 21659648 00002000:00010000:0.0:1789760697.539628:0:6708:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760696 -> 1789760697 00002000:00010000:0.0:1789760697.539631:0:6708:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760696 -> 1789760697 00010000:00010000:0.0:1789760697.539651:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2d1bb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21655552->21659647] (req 21655552->21659647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.539678:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2d1bb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21655552->21659647] (req 21655552->21659647) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760697.539688:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972606 is committed 00010000:00010000:0.0:1789760697.539690:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2d1bb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21655552->21659647] (req 21655552->21659647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.539719:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760697.539726:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2d1bb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21655552->21659647] (req 21655552->21659647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.539736:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2d1bb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21655552->21659647] (req 21655552->21659647) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.539761:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2d1bb lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21655552->21659647] (req 21655552->21659647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.539770:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2d1bb lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21655552->21659647] (req 21655552->21659647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760697.540065:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac28a2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512533888:12345-192.168.203.56@tcp:4:dd.0 Request processed in 35247us (35581us total) trans 4294972606 rc 0/0 00000100:00100000:0.0:1789760697.540078:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5317 00000100:00100000:0.0:1789760697.540090:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 677008164932 time 677069098657 state 0x2 ref 0 00000100:00100000:3.0:1789760697.543510:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760697.543589:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512534016 00000100:00100000:0.0:1789760697.543606:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512534016 00000100:00100000:0.0:1789760697.543693:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 677008164932 time 677069098657 state 0x0 ref 1 00000100:00100000:0.0:1789760697.543899:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 677069359073 time 677069098657 state 0x1 ref 1 00000100:00100000:0.0:1789760697.544051:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 677069359073 time 677108344188 state 0x0 ref 1 00000100:00100000:0.0:1789760697.544067:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febfb89ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512534016:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760697.544134:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21659648, mtime: 0x6aad94b9, atime: 0x0, ctime: 0x6aad94b9, blocks: 0xa807 00010000:00010000:0.0:1789760697.544145:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2d1c2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.544169:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=677s v=(0 0) 00010000:00010000:0.0:1789760697.544175:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760697.544182:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2d1c2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21659648->21663743] (req 21659648->21663743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.544198:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2d1c2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21659648->21663743] (req 21659648->21663743) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760697.595885:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21659648 -> 21663744 00010000:00010000:0.0:1789760697.595916:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2d1c2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21659648->21663743] (req 21659648->21663743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.595932:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2d1c2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21659648->21663743] (req 21659648->21663743) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.596151:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2d1c2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21659648->21663743] (req 21659648->21663743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.596159:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760697.596167:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2d1c2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21659648->21663743] (req 21659648->21663743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.596178:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2d1c2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21659648->21663743] (req 21659648->21663743) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.596202:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2d1c2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21659648->21663743] (req 21659648->21663743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.596210:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2d1c2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21659648->21663743] (req 21659648->21663743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760697.596459:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febfb89ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512534016:12345-192.168.203.56@tcp:4:dd.0 Request processed in 52387us (52950us total) trans 4294972607 rc 0/0 00000100:00100000:0.0:1789760697.596472:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5318 00000100:00100000:0.0:1789760697.596485:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 677069359073 time 677108344188 state 0x2 ref 0 00000100:00100000:0.0:1789760697.619304:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512534144 00000100:00100000:0.0:1789760697.619317:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512534144 00000100:00100000:0.0:1789760697.619460:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 677069359073 time 677108344188 state 0x0 ref 1 00000100:00100000:0.0:1789760697.619482:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 677108604604 time 677108344188 state 0x1 ref 1 00000100:00100000:0.0:1789760697.619517:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 677108604604 time 677183816804 state 0x0 ref 1 00000100:00100000:0.0:1789760697.619530:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4130a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512534144:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760697.621107:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21663744, mtime: 0x6aad94b9, atime: 0x0, ctime: 0x6aad94b9, blocks: 0xa807 00010000:00010000:0.0:1789760697.621128:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2d1c9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.621151:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=677s v=(0 0) 00010000:00010000:0.0:1789760697.621157:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760697.621165:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2d1c9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21663744->21667839] (req 21663744->21667839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.621181:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2d1c9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21663744->21667839] (req 21663744->21667839) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.715293:0:7647:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:2.0:1789760697.743331:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21663744 -> 21667840 00010000:00010000:2.0:1789760697.743417:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2d1c9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21663744->21667839] (req 21663744->21667839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760697.743433:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2d1c9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21663744->21667839] (req 21663744->21667839) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760697.743444:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2d1c9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21663744->21667839] (req 21663744->21667839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760697.743452:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760697.743458:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2d1c9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21663744->21667839] (req 21663744->21667839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760697.743468:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2d1c9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21663744->21667839] (req 21663744->21667839) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760697.743490:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2d1c9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21663744->21667839] (req 21663744->21667839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760697.743497:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2d1c9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21663744->21667839] (req 21663744->21667839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760697.743837:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4130a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512534144:12345-192.168.203.56@tcp:4:dd.0 Request processed in 124300us (124648us total) trans 4294972608 rc 0/0 00000100:00100000:2.0:1789760697.743850:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5319 00000100:00100000:2.0:1789760697.743862:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 677108604604 time 677183816804 state 0x2 ref 0 00000100:00100000:2.0:1789760697.750172:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000001:00080000:2.0:1789760697.841271:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972609 is committed 00002000:00010000:0.0:1789760697.841576:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21667840 -> 21671936 00010000:00010000:0.0:1789760697.841596:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2d1d0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21667840->21671935] (req 21667840->21671935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.841612:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2d1d0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21667840->21671935] (req 21667840->21671935) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.841623:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2d1d0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21667840->21671935] (req 21667840->21671935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.841631:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760697.841638:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2d1d0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21667840->21671935] (req 21667840->21671935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.841649:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2d1d0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21667840->21671935] (req 21667840->21671935) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.841681:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2d1d0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21667840->21671935] (req 21667840->21671935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.841689:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2d1d0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21667840->21671935] (req 21667840->21671935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760697.841828:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac28a0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512534272:12345-192.168.203.56@tcp:4:dd.0 Request processed in 91378us (91663us total) trans 4294972609 rc 0/0 00000100:00100000:0.0:1789760697.841839:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5320 00000100:00100000:0.0:1789760697.841851:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 677184077220 time 677314729417 state 0x2 ref 0 00000100:00100000:2.0:1789760697.845540:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760697.845633:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512534400 00000100:00100000:0.0:1789760697.845647:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512534400 00000100:00100000:0.0:1789760697.845759:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 677184077220 time 677314729417 state 0x0 ref 1 00000100:00100000:0.0:1789760697.845776:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 677314989833 time 677314729417 state 0x1 ref 1 00000100:00100000:0.0:1789760697.845834:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 677314989833 time 677410133539 state 0x0 ref 1 00000100:00100000:0.0:1789760697.845848:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac28a3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512534400:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760697.845908:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21671936, mtime: 0x6aad94b9, atime: 0x0, ctime: 0x6aad94b9, blocks: 0xa807 00010000:00010000:0.0:1789760697.845919:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2d1d7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.845939:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=677s v=(0 0) 00010000:00010000:0.0:1789760697.845945:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760697.845952:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2d1d7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21671936->21676031] (req 21671936->21676031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.845967:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2d1d7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21671936->21676031] (req 21671936->21676031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760697.942755:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972610 is committed 00002000:00010000:0.0:1789760697.942919:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21671936 -> 21676032 00010000:00010000:0.0:1789760697.942938:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2d1d7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21671936->21676031] (req 21671936->21676031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.942954:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2d1d7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21671936->21676031] (req 21671936->21676031) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.942962:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2d1d7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21671936->21676031] (req 21671936->21676031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.942971:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760697.942976:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2d1d7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21671936->21676031] (req 21671936->21676031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.942985:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2d1d7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21671936->21676031] (req 21671936->21676031) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.943007:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2d1d7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21671936->21676031] (req 21671936->21676031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.943019:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2d1d7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21671936->21676031] (req 21671936->21676031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760697.943145:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac28a3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512534400:12345-192.168.203.56@tcp:4:dd.0 Request processed in 97292us (97611us total) trans 4294972610 rc 0/0 00000100:00100000:0.0:1789760697.943157:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5321 00000100:00100000:0.0:1789760697.943169:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 677314989833 time 677410133539 state 0x2 ref 0 00000100:00100000:2.0:1789760697.950221:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512534528 00000100:00100000:2.0:1789760697.950234:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512534528 00000100:00100000:2.0:1789760697.950306:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 677314989833 time 677410133539 state 0x0 ref 1 00000100:00100000:2.0:1789760697.950325:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 677410393955 time 677410133539 state 0x1 ref 1 00000100:00100000:2.0:1789760697.950359:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 677410393955 time 677514658427 state 0x0 ref 1 00000100:00100000:2.0:1789760697.950372:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4132d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512534528:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760697.950440:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21676032, mtime: 0x6aad94b9, atime: 0x0, ctime: 0x6aad94b9, blocks: 0xa807 00010000:00010000:2.0:1789760697.950452:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2d1de lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760697.950472:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=677s v=(0 0) 00010000:00010000:2.0:1789760697.950476:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760697.950484:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2d1de lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21676032->21680127] (req 21676032->21680127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760697.950499:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2d1de lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21676032->21680127] (req 21676032->21680127) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760698.019410:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972611 is committed 00002000:00010000:2.0:1789760698.019459:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21676032 -> 21680128 00010000:00010000:2.0:1789760698.019480:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2d1de lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21676032->21680127] (req 21676032->21680127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760698.019495:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2d1de lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21676032->21680127] (req 21676032->21680127) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760698.019510:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2d1de lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21676032->21680127] (req 21676032->21680127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760698.019519:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760698.019525:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2d1de lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21676032->21680127] (req 21676032->21680127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760698.019535:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2d1de lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21676032->21680127] (req 21676032->21680127) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760698.019559:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2d1de lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21676032->21680127] (req 21676032->21680127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760698.019633:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2d1de lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21676032->21680127] (req 21676032->21680127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760698.020228:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4132d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512534528:12345-192.168.203.56@tcp:4:dd.0 Request processed in 69849us (70146us total) trans 4294972611 rc 0/0 00000100:00100000:2.0:1789760698.020242:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5322 00000100:00100000:2.0:1789760698.020254:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 677410393955 time 677514658427 state 0x2 ref 0 00000100:00100000:0.0:1789760698.025957:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512534656 00000100:00100000:0.0:1789760698.025971:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512534656 00000100:00100000:0.0:1789760698.026048:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 677410393955 time 677514658427 state 0x0 ref 1 00000100:00100000:0.0:1789760698.026082:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 677514918843 time 677514658427 state 0x1 ref 1 00000100:00100000:0.0:1789760698.026213:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 677514918843 time 677590510447 state 0x0 ref 1 00000100:00100000:0.0:1789760698.026229:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5305c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512534656:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760698.026307:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21680128, mtime: 0x6aad94b9, atime: 0x0, ctime: 0x6aad94b9, blocks: 0xa807 00010000:00010000:0.0:1789760698.026318:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2d1e5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.026345:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=677s v=(0 0) 00010000:00010000:0.0:1789760698.026351:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760698.026358:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2d1e5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21680128->21684223] (req 21680128->21684223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.026376:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2d1e5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21680128->21684223] (req 21680128->21684223) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760698.077528:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972612 is committed 00002000:00010000:0.0:1789760698.077870:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21680128 -> 21684224 00010000:00010000:0.0:1789760698.077889:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2d1e5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21680128->21684223] (req 21680128->21684223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.077905:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2d1e5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21680128->21684223] (req 21680128->21684223) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.077915:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2d1e5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21680128->21684223] (req 21680128->21684223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.077923:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760698.077929:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2d1e5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21680128->21684223] (req 21680128->21684223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.077940:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2d1e5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21680128->21684223] (req 21680128->21684223) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.077962:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2d1e5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21680128->21684223] (req 21680128->21684223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.077970:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2d1e5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21680128->21684223] (req 21680128->21684223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760698.078670:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5305c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512534656:12345-192.168.203.56@tcp:4:dd.0 Request processed in 52426us (52993us total) trans 4294972612 rc 0/0 00000100:00100000:0.0:1789760698.078683:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5323 00000100:00100000:0.0:1789760698.078723:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 677514918843 time 677590510447 state 0x2 ref 0 00000100:00100000:3.0:1789760698.090837:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760698.092369:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512534784 00000100:00100000:0.0:1789760698.092388:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512534784 00000100:00100000:0.0:1789760698.092478:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 677514918843 time 677590510447 state 0x0 ref 1 00000100:00100000:0.0:1789760698.092499:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 677590770863 time 677590510447 state 0x1 ref 1 00000100:00100000:0.0:1789760698.092541:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 677590770863 time 677656840491 state 0x0 ref 1 00000100:00100000:0.0:1789760698.092557:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febfb89fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512534784:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760698.092645:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21684224, mtime: 0x6aad94b9, atime: 0x0, ctime: 0x6aad94b9, blocks: 0xa807 00010000:00010000:0.0:1789760698.092656:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2d1ec lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.092686:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=677s v=(0 0) 00010000:00010000:0.0:1789760698.092691:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760698.092738:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2d1ec lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21684224->21688319] (req 21684224->21688319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.092760:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2d1ec lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21684224->21688319] (req 21684224->21688319) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760698.201547:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972613 is committed 00002000:00010000:0.0:1789760698.201642:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21684224 -> 21688320 00010000:00010000:0.0:1789760698.201669:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2d1ec lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21684224->21688319] (req 21684224->21688319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.201686:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2d1ec lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21684224->21688319] (req 21684224->21688319) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.201780:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2d1ec lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21684224->21688319] (req 21684224->21688319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.201790:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760698.201796:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2d1ec lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21684224->21688319] (req 21684224->21688319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.201808:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2d1ec lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21684224->21688319] (req 21684224->21688319) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.201838:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2d1ec lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21684224->21688319] (req 21684224->21688319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.201847:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2d1ec lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21684224->21688319] (req 21684224->21688319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760698.218148:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512534912 00000100:00100000:2.0:1789760698.218160:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512534912 00000100:00100000:2.0:1789760698.218226:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 677590770863 time 677656840491 state 0x0 ref 1 00000100:00100000:2.0:1789760698.218244:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 677657100907 time 677656840491 state 0x1 ref 1 00000100:00100000:2.0:1789760698.218281:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 677657100907 time 677782580904 state 0x0 ref 1 00000100:00100000:2.0:1789760698.218302:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5304e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512534912:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760698.218367:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21688320, mtime: 0x6aad94b9, atime: 0x0, ctime: 0x6aad94b9, blocks: 0xa807 00010000:00010000:2.0:1789760698.218379:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2d1f3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760698.218405:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=677s v=(0 0) 00010000:00010000:2.0:1789760698.218410:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760698.218417:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2d1f3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21688320->21692415] (req 21688320->21692415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760698.218433:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2d1f3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21688320->21692415] (req 21688320->21692415) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760698.291928:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21688320 -> 21692416 00010000:00010000:0.0:1789760698.291951:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2d1f3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21688320->21692415] (req 21688320->21692415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.291967:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2d1f3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21688320->21692415] (req 21688320->21692415) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.291977:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2d1f3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21688320->21692415] (req 21688320->21692415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.291986:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760698.291992:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2d1f3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21688320->21692415] (req 21688320->21692415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.292003:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2d1f3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21688320->21692415] (req 21688320->21692415) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.292029:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2d1f3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21688320->21692415] (req 21688320->21692415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.292038:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2d1f3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21688320->21692415] (req 21688320->21692415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760698.292266:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5304e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512534912:12345-192.168.203.56@tcp:4:dd.0 Request processed in 73957us (74260us total) trans 4294972614 rc 0/0 00000100:00100000:0.0:1789760698.292278:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5325 00000100:00100000:0.0:1789760698.292290:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 677657100907 time 677782580904 state 0x2 ref 0 00000100:00100000:0.0:1789760698.295076:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512535040 00000100:00100000:0.0:1789760698.295089:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512535040 00000100:00100000:0.0:1789760698.295174:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 677657100907 time 677782580904 state 0x0 ref 1 00000100:00100000:0.0:1789760698.295194:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 677782841320 time 677782580904 state 0x1 ref 1 00000100:00100000:0.0:1789760698.295232:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 677782841320 time 677859530855 state 0x0 ref 1 00000100:00100000:0.0:1789760698.295246:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5307100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512535040:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760698.295309:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21692416, mtime: 0x6aad94b9, atime: 0x0, ctime: 0x6aad94b9, blocks: 0xa807 00010000:00010000:0.0:1789760698.295320:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2d1fa lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.295340:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=677s v=(0 0) 00010000:00010000:0.0:1789760698.295346:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760698.295354:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2d1fa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21692416->21696511] (req 21692416->21696511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.295369:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2d1fa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21692416->21696511] (req 21692416->21696511) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760698.351958:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972615 is committed 00000100:00100000:3.0:1789760698.352906:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000001:00080000:2.0:1789760698.397024:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972616 is committed 00000100:00100000:0.0:1789760698.407062:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760698.407224:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512535296 00000100:00100000:2.0:1789760698.407237:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512535296 00000100:00100000:2.0:1789760698.407308:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 677859791271 time 677917539627 state 0x0 ref 1 00000100:00100000:2.0:1789760698.407337:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 677917800043 time 677917539627 state 0x1 ref 1 00000100:00100000:2.0:1789760698.407372:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 677917800043 time 677971671140 state 0x0 ref 1 00000100:00100000:2.0:1789760698.407386:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2fb8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512535296:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760698.407455:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21700608, mtime: 0x6aad94b9, atime: 0x0, ctime: 0x6aad94b9, blocks: 0xa807 00010000:00010000:2.0:1789760698.407465:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2d208 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760698.407486:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=677s v=(0 0) 00010000:00010000:2.0:1789760698.407492:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760698.407501:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2d208 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21700608->21704703] (req 21700608->21704703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760698.407517:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2d208 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21700608->21704703] (req 21700608->21704703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760698.505286:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972617 is committed 00002000:00010000:2.0:1789760698.505500:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21700608 -> 21704704 00010000:00010000:2.0:1789760698.505523:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2d208 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21700608->21704703] (req 21700608->21704703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760698.505539:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2d208 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21700608->21704703] (req 21700608->21704703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760698.505551:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2d208 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21700608->21704703] (req 21700608->21704703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760698.505560:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760698.505567:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2d208 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21700608->21704703] (req 21700608->21704703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760698.505583:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2d208 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21700608->21704703] (req 21700608->21704703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760698.505615:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2d208 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21700608->21704703] (req 21700608->21704703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760698.505625:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2d208 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21700608->21704703] (req 21700608->21704703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760698.505793:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2fb8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512535296:12345-192.168.203.56@tcp:4:dd.0 Request processed in 98402us (98752us total) trans 4294972617 rc 0/0 00000100:00100000:2.0:1789760698.505804:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5328 00000100:00100000:2.0:1789760698.505816:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 677917800043 time 677971671140 state 0x2 ref 0 00000100:00100000:2.0:1789760698.516922:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000001:00080000:0.0:1789760698.625267:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972618 is committed 00002000:00010000:2.0:1789760698.633181:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21704704 -> 21708800 00002000:00010000:2.0:1789760698.633194:0:6708:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760697 -> 1789760698 00002000:00010000:2.0:1789760698.633197:0:6708:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760697 -> 1789760698 00010000:00010000:2.0:1789760698.633215:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2d20f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21704704->21708799] (req 21704704->21708799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760698.633231:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2d20f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21704704->21708799] (req 21704704->21708799) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760698.633242:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2d20f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21704704->21708799] (req 21704704->21708799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760698.633251:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760698.633257:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2d20f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21704704->21708799] (req 21704704->21708799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760698.633267:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2d20f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21704704->21708799] (req 21704704->21708799) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760698.633296:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2d20f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21704704->21708799] (req 21704704->21708799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760698.633305:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2d20f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21704704->21708799] (req 21704704->21708799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760698.633821:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac28a1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512535424:12345-192.168.203.56@tcp:4:dd.0 Request processed in 116620us (116898us total) trans 4294972618 rc 0/0 00000100:00100000:3.0:1789760698.633836:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5329 00000100:00100000:3.0:1789760698.633851:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 677971931556 time 678081469749 state 0x2 ref 0 00000100:00100000:2.0:1789760698.652323:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760698.652692:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512535936 00000100:00100000:3.0:1789760698.652725:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512535936 00000100:00100000:3.0:1789760698.652801:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 677971931556 time 678081469749 state 0x0 ref 1 00000100:00100000:3.0:1789760698.652823:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 678081730165 time 678081469749 state 0x1 ref 1 00000100:00100000:3.0:1789760698.652861:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 678081730165 time 678217159972 state 0x0 ref 1 00000100:00100000:3.0:1789760698.652876:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac28a2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512535936:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760698.652947:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21708800, mtime: 0x6aad94ba, atime: 0x0, ctime: 0x6aad94ba, blocks: 0xa807 00010000:00010000:3.0:1789760698.652959:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2d216 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760698.652980:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=678s v=(0 0) 00010000:00010000:3.0:1789760698.652986:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760698.652995:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2d216 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21708800->21712895] (req 21708800->21712895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760698.653013:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2d216 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21708800->21712895] (req 21708800->21712895) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760698.655729:0:5842:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512535552 00000100:00100000:3.0:1789760698.655740:0:5842:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512535552 00000100:00100000:3.0:1789760698.655803:0:5842:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 380 00000100:00100000:3.0:1789760698.655816:0:5842:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febe00d5850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:e6cf7267-9b19-462a-a263-14115df15dab+13:2360:x1876699512535552:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:3.0:1789760698.655905:0:5842:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febe00d5850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:e6cf7267-9b19-462a-a263-14115df15dab+13:2360:x1876699512535552:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 91us (548us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760698.655912:0:5842:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 380 00000100:00100000:0.0:1789760698.672832:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512535680 00000100:00100000:0.0:1789760698.672854:0:5847:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512535680 00000100:00100000:0.0:1789760698.672958:0:8229:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512535808 00000100:00100000:0.0:1789760698.672962:0:8229:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512535808 00000100:00100000:0.0:1789760698.673002:0:8229:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 429 00000100:00100000:0.0:1789760698.673013:0:8229:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5305c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+8:2361:x1876699512535680:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:0.0:1789760698.673253:0:8229:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5305c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+8:2361:x1876699512535680:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 240us (17699us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760698.673262:0:8229:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 429 00000100:00100000:0.0:1789760698.673284:0:8229:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 430 00000100:00100000:0.0:1789760698.673292:0:8229:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5304e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:d2c40150-e97b-40da-952b-cb9f18bf7e96+5:2360:x1876699512535808:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:0.0:1789760698.673318:0:8229:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5304e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:d2c40150-e97b-40da-952b-cb9f18bf7e96+5:2360:x1876699512535808:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 27us (17646us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760698.673392:0:8229:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 430 00000001:00080000:2.0:1789760698.723546:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972619 is committed 00002000:00010000:0.0:1789760698.724590:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21708800 -> 21712896 00010000:00010000:0.0:1789760698.724617:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2d216 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21708800->21712895] (req 21708800->21712895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.724632:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2d216 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21708800->21712895] (req 21708800->21712895) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.724641:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2d216 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21708800->21712895] (req 21708800->21712895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.724649:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760698.724664:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2d216 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21708800->21712895] (req 21708800->21712895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.724674:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2d216 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21708800->21712895] (req 21708800->21712895) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.724723:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2d216 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21708800->21712895] (req 21708800->21712895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.724733:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2d216 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21708800->21712895] (req 21708800->21712895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760698.724912:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac28a2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512535936:12345-192.168.203.56@tcp:4:dd.0 Request processed in 72030us (72597us total) trans 4294972619 rc 0/0 00000100:00100000:0.0:1789760698.724927:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5330 00000100:00100000:0.0:1789760698.724940:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 678081730165 time 678217159972 state 0x2 ref 0 00000100:00100000:2.0:1789760698.729954:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512536064 00000100:00100000:2.0:1789760698.729968:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512536064 00000100:00100000:2.0:1789760698.730045:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 678081730165 time 678217159972 state 0x0 ref 1 00000100:00100000:2.0:1789760698.730077:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 678217420388 time 678217159972 state 0x1 ref 1 00000100:00100000:2.0:1789760698.730114:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 678217420388 time 678294412222 state 0x0 ref 1 00000100:00100000:2.0:1789760698.730128:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5307100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512536064:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760698.730189:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21712896, mtime: 0x6aad94ba, atime: 0x0, ctime: 0x6aad94ba, blocks: 0xa807 00010000:00010000:2.0:1789760698.730201:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2d21d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760698.730222:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=678s v=(0 0) 00010000:00010000:2.0:1789760698.730227:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760698.730235:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2d21d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21712896->21716991] (req 21712896->21716991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760698.730253:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2d21d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21712896->21716991] (req 21712896->21716991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.744580:0:5836:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:3.0:1789760698.796095:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21712896 -> 21716992 00010000:00010000:3.0:1789760698.796115:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2d21d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21712896->21716991] (req 21712896->21716991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760698.796136:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2d21d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21712896->21716991] (req 21712896->21716991) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760698.796146:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2d21d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21712896->21716991] (req 21712896->21716991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760698.796154:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760698.796159:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2d21d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21712896->21716991] (req 21712896->21716991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760698.796168:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2d21d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21712896->21716991] (req 21712896->21716991) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760698.796188:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2d21d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21712896->21716991] (req 21712896->21716991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760698.796196:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2d21d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21712896->21716991] (req 21712896->21716991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760698.796317:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5307100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2359:x1876699512536064:12345-192.168.203.56@tcp:4:dd.0 Request processed in 66179us (66533us total) trans 4294972620 rc 0/0 00000100:00100000:3.0:1789760698.797169:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5331 00000100:00100000:3.0:1789760698.797180:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 678217420388 time 678294412222 state 0x2 ref 0 00000100:00100000:3.0:1789760698.805126:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000001:00080000:3.0:1789760698.888426:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972621 is committed 00002000:00010000:0.0:1789760698.888784:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21716992 -> 21721088 00010000:00010000:0.0:1789760698.888802:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2d224 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21716992->21721087] (req 21716992->21721087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.888813:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2d224 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21716992->21721087] (req 21716992->21721087) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.888820:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2d224 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21716992->21721087] (req 21716992->21721087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.888825:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760698.888830:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2d224 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21716992->21721087] (req 21716992->21721087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.888841:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2d224 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21716992->21721087] (req 21716992->21721087) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.888861:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2d224 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21716992->21721087] (req 21716992->21721087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.888865:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2d224 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21716992->21721087] (req 21716992->21721087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760698.888969:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febfb89ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512536192:12345-192.168.203.56@tcp:4:dd.0 Request processed in 83563us (83851us total) trans 4294972621 rc 0/0 00000100:00100000:0.0:1789760698.888982:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5332 00000100:00100000:0.0:1789760698.888994:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 678294672638 time 678369686743 state 0x2 ref 0 00000100:00100000:2.0:1789760698.894200:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760698.894279:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512536320 00000100:00100000:0.0:1789760698.894291:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512536320 00000100:00100000:0.0:1789760698.894527:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 678294672638 time 678369686743 state 0x0 ref 1 00000100:00100000:0.0:1789760698.894550:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 678369947159 time 678369686743 state 0x1 ref 1 00000100:00100000:0.0:1789760698.894586:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 678369947159 time 678458885321 state 0x0 ref 1 00000100:00100000:0.0:1789760698.894601:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac28a0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512536320:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760698.894684:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21721088, mtime: 0x6aad94ba, atime: 0x0, ctime: 0x6aad94ba, blocks: 0xa807 00010000:00010000:0.0:1789760698.894725:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2d22b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.894749:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=678s v=(0 0) 00010000:00010000:0.0:1789760698.894755:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760698.894763:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2d22b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21721088->21725183] (req 21721088->21725183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.894780:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2d22b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21721088->21725183] (req 21721088->21725183) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760698.949044:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972622 is committed 00002000:00010000:0.0:1789760698.949164:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21721088 -> 21725184 00010000:00010000:0.0:1789760698.949185:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2d22b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21721088->21725183] (req 21721088->21725183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.949201:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2d22b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21721088->21725183] (req 21721088->21725183) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.949212:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2d22b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21721088->21725183] (req 21721088->21725183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.949220:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760698.949226:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2d22b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21721088->21725183] (req 21721088->21725183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.949236:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2d22b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21721088->21725183] (req 21721088->21725183) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.949428:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2d22b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21721088->21725183] (req 21721088->21725183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.949440:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2d22b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21721088->21725183] (req 21721088->21725183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760698.949603:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac28a0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512536320:12345-192.168.203.56@tcp:4:dd.0 Request processed in 54997us (55411us total) trans 4294972622 rc 0/0 00000100:00100000:0.0:1789760698.949616:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5333 00000100:00100000:0.0:1789760698.949628:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 678369947159 time 678458885321 state 0x2 ref 0 00000100:00100000:3.0:1789760698.953907:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512536448 00000100:00100000:3.0:1789760698.953921:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512536448 00000100:00100000:3.0:1789760698.953997:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 678369947159 time 678458885321 state 0x0 ref 1 00000100:00100000:3.0:1789760698.954017:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 678459145737 time 678458885321 state 0x1 ref 1 00000100:00100000:3.0:1789760698.954085:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 678459145737 time 678518383911 state 0x0 ref 1 00000100:00100000:3.0:1789760698.954099:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5305f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512536448:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760698.954165:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21725184, mtime: 0x6aad94ba, atime: 0x0, ctime: 0x6aad94ba, blocks: 0xa807 00010000:00010000:3.0:1789760698.954176:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2d232 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760698.954196:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=678s v=(0 0) 00010000:00010000:3.0:1789760698.954202:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760698.954210:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2d232 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21725184->21729279] (req 21725184->21729279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760698.954226:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2d232 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21725184->21729279] (req 21725184->21729279) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760699.006516:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972623 is committed 00002000:00010000:3.0:1789760699.006622:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21725184 -> 21729280 00010000:00010000:3.0:1789760699.006642:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2d232 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21725184->21729279] (req 21725184->21729279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760699.006673:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2d232 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21725184->21729279] (req 21725184->21729279) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760699.006681:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2d232 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21725184->21729279] (req 21725184->21729279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760699.006686:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760699.006690:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2d232 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21725184->21729279] (req 21725184->21729279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760699.006727:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2d232 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21725184->21729279] (req 21725184->21729279) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760699.006756:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2d232 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21725184->21729279] (req 21725184->21729279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760699.006765:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2d232 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21725184->21729279] (req 21725184->21729279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760699.007034:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5305f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512536448:12345-192.168.203.56@tcp:4:dd.0 Request processed in 52928us (53340us total) trans 4294972623 rc 0/0 00000100:00100000:2.0:1789760699.007047:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5334 00000100:00100000:2.0:1789760699.007058:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 678459145737 time 678518383911 state 0x2 ref 0 00000100:00100000:3.0:1789760699.012692:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760699.012828:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512536576 00000100:00100000:0.0:1789760699.012841:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512536576 00000100:00100000:0.0:1789760699.012929:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 678459145737 time 678518383911 state 0x0 ref 1 00000100:00100000:0.0:1789760699.012949:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 678518644327 time 678518383911 state 0x1 ref 1 00000100:00100000:0.0:1789760699.013020:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 678518644327 time 678577318384 state 0x0 ref 1 00000100:00100000:0.0:1789760699.013034:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febfb89fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512536576:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760699.013260:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21729280, mtime: 0x6aad94ba, atime: 0x0, ctime: 0x6aad94ba, blocks: 0xa807 00010000:00010000:0.0:1789760699.013270:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2d239 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760699.013290:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=678s v=(0 0) 00010000:00010000:0.0:1789760699.013295:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760699.013304:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2d239 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21729280->21733375] (req 21729280->21733375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760699.013332:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2d239 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21729280->21733375] (req 21729280->21733375) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760699.080909:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760699.081012:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512536704 00000100:00100000:3.0:1789760699.081026:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512536704 00000100:00100000:3.0:1789760699.081116:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 678518644327 time 678577318384 state 0x0 ref 1 00000100:00100000:3.0:1789760699.081136:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 678577578800 time 678577318384 state 0x1 ref 1 00000100:00100000:3.0:1789760699.081220:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 678577578800 time 678645518214 state 0x0 ref 1 00000100:00100000:3.0:1789760699.081234:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2fb9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512536704:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760699.081308:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21733376, mtime: 0x6aad94ba, atime: 0x0, ctime: 0x6aad94ba, blocks: 0xa807 00010000:00010000:3.0:1789760699.081321:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2d240 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760699.081343:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=678s v=(0 0) 00010000:00010000:3.0:1789760699.081348:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760699.081356:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2d240 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21733376->21737471] (req 21733376->21737471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760699.081374:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2d240 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21733376->21737471] (req 21733376->21737471) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760699.126259:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972625 is committed 00000100:00100000:2.0:1789760699.131557:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760699.132050:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512536832 00000100:00100000:0.0:1789760699.132065:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512536832 00000100:00100000:0.0:1789760699.132149:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 678577578800 time 678645518214 state 0x0 ref 1 00000100:00100000:0.0:1789760699.132170:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 678645778630 time 678645518214 state 0x1 ref 1 00000100:00100000:0.0:1789760699.132212:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 678645778630 time 678696510867 state 0x0 ref 1 00000100:00100000:0.0:1789760699.132229:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac28a3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512536832:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760699.132297:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21737472, mtime: 0x6aad94ba, atime: 0x0, ctime: 0x6aad94ba, blocks: 0xa807 00010000:00010000:0.0:1789760699.132308:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2d247 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760699.132328:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=678s v=(0 0) 00010000:00010000:0.0:1789760699.132333:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760699.132342:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2d247 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21737472->21741567] (req 21737472->21741567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760699.132359:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2d247 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21737472->21741567] (req 21737472->21741567) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760699.179057:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972626 is committed 00002000:00010000:2.0:1789760699.179147:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21737472 -> 21741568 00010000:00010000:2.0:1789760699.179164:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2d247 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21737472->21741567] (req 21737472->21741567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.179176:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2d247 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21737472->21741567] (req 21737472->21741567) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.179184:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2d247 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21737472->21741567] (req 21737472->21741567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.179194:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760699.179198:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2d247 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21737472->21741567] (req 21737472->21741567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.179205:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2d247 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21737472->21741567] (req 21737472->21741567) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.179223:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2d247 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21737472->21741567] (req 21737472->21741567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.179228:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2d247 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21737472->21741567] (req 21737472->21741567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760699.179841:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac28a3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512536832:12345-192.168.203.56@tcp:4:dd.0 Request processed in 47606us (48288us total) trans 4294972626 rc 0/0 00000100:00100000:2.0:1789760699.179855:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5337 00000100:00100000:2.0:1789760699.179868:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 678645778630 time 678696510867 state 0x2 ref 0 00000100:00100000:3.0:1789760699.197157:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000001:00080000:3.0:1789760699.253746:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972627 is committed 00000100:00100000:0.0:1789760699.257626:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000001:00080000:0.0:1789760699.305642:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972628 is committed 00000100:00100000:2.0:1789760699.310573:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760699.311055:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 678762326598 time 678822151516 state 0x0 ref 1 00000100:00100000:0.0:1789760699.311076:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 678822411932 time 678822151516 state 0x1 ref 1 00000100:00100000:0.0:1789760699.311139:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 678822411932 time 678875437784 state 0x0 ref 1 00000100:00100000:0.0:1789760699.311152:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4362d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512537216:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760699.311220:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21749760, mtime: 0x6aad94ba, atime: 0x0, ctime: 0x6aad94ba, blocks: 0xa807 00010000:00010000:0.0:1789760699.311231:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2d25c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760699.311252:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=678s v=(0 0) 00010000:00010000:0.0:1789760699.311257:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760699.311264:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2d25c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21749760->21753855] (req 21749760->21753855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760699.311281:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2d25c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21749760->21753855] (req 21749760->21753855) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760699.353888:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21749760 -> 21753856 00010000:00010000:2.0:1789760699.353906:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2d25c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21749760->21753855] (req 21749760->21753855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.353917:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2d25c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21749760->21753855] (req 21749760->21753855) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.353929:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2d25c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21749760->21753855] (req 21749760->21753855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.353935:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760699.353939:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2d25c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21749760->21753855] (req 21749760->21753855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.353946:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2d25c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21749760->21753855] (req 21749760->21753855) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.353965:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2d25c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21749760->21753855] (req 21749760->21753855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.353969:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2d25c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21749760->21753855] (req 21749760->21753855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760699.354895:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4362d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512537216:12345-192.168.203.56@tcp:4:dd.0 Request processed in 43730us (44321us total) trans 4294972629 rc 0/0 00000100:00100000:2.0:1789760699.354912:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5340 00000100:00100000:2.0:1789760699.354926:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 678822411932 time 678875437784 state 0x2 ref 0 00000100:00100000:3.0:1789760699.358140:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000001:00080000:3.0:1789760699.400899:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972630 is committed 00000100:00100000:0.0:1789760699.404213:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000001:00080000:0.0:1789760699.431584:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972631 is committed 00000100:00100000:2.0:1789760699.435632:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760699.435834:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512537600 00000100:00100000:0.0:1789760699.435848:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512537600 00000100:00100000:0.0:1789760699.435923:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 678922969738 time 678968932173 state 0x0 ref 1 00000100:00100000:0.0:1789760699.435940:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 678969192589 time 678968932173 state 0x1 ref 1 00000100:00100000:0.0:1789760699.436019:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 678969192589 time 679000317076 state 0x0 ref 1 00000100:00100000:0.0:1789760699.436033:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4361500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512537600:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760699.436115:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21762048, mtime: 0x6aad94ba, atime: 0x0, ctime: 0x6aad94ba, blocks: 0xa807 00010000:00010000:0.0:1789760699.436126:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2d271 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760699.436146:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=678s v=(0 0) 00010000:00010000:0.0:1789760699.436150:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760699.436158:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2d271 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21762048->21766143] (req 21762048->21766143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760699.436174:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2d271 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21762048->21766143] (req 21762048->21766143) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760699.530899:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972632 is committed 00002000:00010000:3.0:1789760699.531212:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21762048 -> 21766144 00010000:00010000:3.0:1789760699.531232:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2d271 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21762048->21766143] (req 21762048->21766143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760699.531249:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2d271 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21762048->21766143] (req 21762048->21766143) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760699.531259:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2d271 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21762048->21766143] (req 21762048->21766143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760699.531268:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760699.531274:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2d271 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21762048->21766143] (req 21762048->21766143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760699.531289:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2d271 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21762048->21766143] (req 21762048->21766143) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760699.531315:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2d271 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21762048->21766143] (req 21762048->21766143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760699.531451:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2d271 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21762048->21766143] (req 21762048->21766143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760699.532044:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4361500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512537600:12345-192.168.203.56@tcp:4:dd.0 Request processed in 96005us (96418us total) trans 4294972632 rc 0/0 00000100:00100000:3.0:1789760699.532058:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5343 00000100:00100000:3.0:1789760699.532071:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 678969192589 time 679000317076 state 0x2 ref 0 00000100:00100000:3.0:1789760699.541838:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00002000:00010000:2.0:1789760699.577865:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21766144 -> 21770240 00002000:00010000:2.0:1789760699.577879:0:6708:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760698 -> 1789760699 00002000:00010000:2.0:1789760699.577882:0:6708:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760698 -> 1789760699 00010000:00010000:2.0:1789760699.577901:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2d278 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21766144->21770239] (req 21766144->21770239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.577918:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2d278 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21766144->21770239] (req 21766144->21770239) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.577927:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2d278 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21766144->21770239] (req 21766144->21770239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.577935:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760699.577941:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2d278 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21766144->21770239] (req 21766144->21770239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.577951:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2d278 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21766144->21770239] (req 21766144->21770239) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.577986:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2d278 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21766144->21770239] (req 21766144->21770239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.577994:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2d278 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21766144->21770239] (req 21766144->21770239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760699.578113:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4058a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2359:x1876699512537728:12345-192.168.203.56@tcp:4:dd.0 Request processed in 35922us (36283us total) trans 4294972633 rc 0/0 00000100:00100000:2.0:1789760699.578126:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5344 00000100:00100000:2.0:1789760699.578138:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 679000577492 time 679106460632 state 0x2 ref 0 00000001:00080000:3.0:1789760699.580236:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972633 is committed 00000100:00100000:0.0:1789760699.583446:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000001:00080000:2.0:1789760699.673260:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972634 is committed 00002000:00010000:3.0:1789760699.673364:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21770240 -> 21774336 00010000:00010000:3.0:1789760699.673380:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2d27f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21770240->21774335] (req 21770240->21774335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760699.673394:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2d27f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21770240->21774335] (req 21770240->21774335) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760699.673405:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2d27f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21770240->21774335] (req 21770240->21774335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760699.673413:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760699.673418:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2d27f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21770240->21774335] (req 21770240->21774335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760699.673428:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2d27f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21770240->21774335] (req 21770240->21774335) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760699.673626:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2d27f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21770240->21774335] (req 21770240->21774335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760699.673635:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2d27f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21770240->21774335] (req 21770240->21774335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760699.673818:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2fbb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512537856:12345-192.168.203.56@tcp:4:dd.0 Request processed in 90059us (90380us total) trans 4294972634 rc 0/0 00000100:00100000:3.0:1789760699.673829:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5345 00000100:00100000:3.0:1789760699.673838:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 679106721048 time 679148038298 state 0x2 ref 0 00000100:00100000:0.0:1789760699.679603:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760699.679823:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512537984 00000100:00100000:3.0:1789760699.680522:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512537984 00000100:00100000:3.0:1789760699.680595:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 679106721048 time 679148038298 state 0x0 ref 1 00000100:00100000:3.0:1789760699.680614:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 679148298714 time 679148038298 state 0x1 ref 1 00000100:00100000:3.0:1789760699.680658:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 679148298714 time 679244958012 state 0x0 ref 1 00000100:00100000:3.0:1789760699.680672:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2fb8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512537984:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760699.680819:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21774336, mtime: 0x6aad94bb, atime: 0x0, ctime: 0x6aad94bb, blocks: 0xa807 00010000:00010000:3.0:1789760699.681519:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2d286 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760699.681538:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=679s v=(0 0) 00010000:00010000:3.0:1789760699.681542:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760699.681550:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2d286 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21774336->21778431] (req 21774336->21778431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760699.681565:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2d286 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21774336->21778431] (req 21774336->21778431) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.763432:0:5835:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:0.0:1789760699.779548:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972635 is committed 00002000:00010000:3.0:1789760699.779629:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21774336 -> 21778432 00010000:00010000:3.0:1789760699.779644:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2d286 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21774336->21778431] (req 21774336->21778431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760699.779665:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2d286 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21774336->21778431] (req 21774336->21778431) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760699.779680:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2d286 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21774336->21778431] (req 21774336->21778431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760699.779688:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760699.779693:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2d286 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21774336->21778431] (req 21774336->21778431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760699.779796:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2d286 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21774336->21778431] (req 21774336->21778431) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760699.779816:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2d286 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21774336->21778431] (req 21774336->21778431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760699.779824:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2d286 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21774336->21778431] (req 21774336->21778431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760699.779956:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2fb8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512537984:12345-192.168.203.56@tcp:4:dd.0 Request processed in 99282us (100358us total) trans 4294972635 rc 0/0 00000100:00100000:3.0:1789760699.779965:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5346 00000100:00100000:3.0:1789760699.779975:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 679148298714 time 679244958012 state 0x2 ref 0 00000100:00100000:0.0:1789760699.782642:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760699.782895:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512538112 00000100:00100000:2.0:1789760699.782905:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512538112 00000100:00100000:2.0:1789760699.782965:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 679148298714 time 679244958012 state 0x0 ref 1 00000100:00100000:2.0:1789760699.782982:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 679245218428 time 679244958012 state 0x1 ref 1 00000100:00100000:2.0:1789760699.783011:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 679245218428 time 679347311390 state 0x0 ref 1 00000100:00100000:2.0:1789760699.783024:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2fb9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512538112:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760699.783072:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21778432, mtime: 0x6aad94bb, atime: 0x0, ctime: 0x6aad94bb, blocks: 0xa807 00010000:00010000:2.0:1789760699.783080:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2d28d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.783098:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=679s v=(0 0) 00010000:00010000:2.0:1789760699.783103:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760699.783110:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2d28d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21778432->21782527] (req 21778432->21782527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.783124:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2d28d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21778432->21782527] (req 21778432->21782527) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760699.838383:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972636 is committed 00000100:00100000:0.0:1789760699.849105:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760699.849259:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512538240 00000100:00100000:3.0:1789760699.849268:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512538240 00000100:00100000:3.0:1789760699.849337:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 679245218428 time 679347311390 state 0x0 ref 1 00000100:00100000:3.0:1789760699.849358:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 679347571806 time 679347311390 state 0x1 ref 1 00000100:00100000:3.0:1789760699.849394:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 679347571806 time 679413693939 state 0x0 ref 1 00000100:00100000:3.0:1789760699.849408:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2fbaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512538240:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760699.849466:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21782528, mtime: 0x6aad94bb, atime: 0x0, ctime: 0x6aad94bb, blocks: 0xa807 00010000:00010000:3.0:1789760699.849482:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2d294 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760699.849497:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=679s v=(0 0) 00010000:00010000:3.0:1789760699.849502:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760699.849508:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2d294 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21782528->21786623] (req 21782528->21786623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760699.849521:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2d294 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21782528->21786623] (req 21782528->21786623) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760699.892330:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972637 is committed 00002000:00010000:0.0:1789760699.892444:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21782528 -> 21786624 00010000:00010000:0.0:1789760699.892459:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2d294 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21782528->21786623] (req 21782528->21786623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760699.892472:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2d294 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21782528->21786623] (req 21782528->21786623) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760699.892481:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2d294 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21782528->21786623] (req 21782528->21786623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760699.892488:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760699.892493:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2d294 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21782528->21786623] (req 21782528->21786623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760699.892502:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2d294 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21782528->21786623] (req 21782528->21786623) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760699.892522:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2d294 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21782528->21786623] (req 21782528->21786623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760699.892529:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2d294 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21782528->21786623] (req 21782528->21786623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760699.892619:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2fbaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512538240:12345-192.168.203.56@tcp:4:dd.0 Request processed in 43210us (43518us total) trans 4294972637 rc 0/0 00000100:00100000:0.0:1789760699.892629:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5348 00000100:00100000:0.0:1789760699.892638:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 679347571806 time 679413693939 state 0x2 ref 0 00000100:00100000:0.0:1789760699.896007:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760699.896267:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512538368 00000100:00100000:3.0:1789760699.896278:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512538368 00000100:00100000:3.0:1789760699.896340:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 679347571806 time 679413693939 state 0x0 ref 1 00000100:00100000:3.0:1789760699.896356:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 679413954355 time 679413693939 state 0x1 ref 1 00000100:00100000:3.0:1789760699.896389:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 679413954355 time 679460688511 state 0x0 ref 1 00000100:00100000:3.0:1789760699.896406:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febc6bd6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512538368:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760699.896454:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21786624, mtime: 0x6aad94bb, atime: 0x0, ctime: 0x6aad94bb, blocks: 0xa807 00010000:00010000:3.0:1789760699.896463:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2d29b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760699.896480:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=679s v=(0 0) 00010000:00010000:3.0:1789760699.896485:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760699.896491:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2d29b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21786624->21790719] (req 21786624->21790719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760699.896505:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2d29b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21786624->21790719] (req 21786624->21790719) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760699.945642:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972638 is committed 00002000:00010000:3.0:1789760699.945936:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21786624 -> 21790720 00010000:00010000:3.0:1789760699.945951:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2d29b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21786624->21790719] (req 21786624->21790719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760699.945964:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2d29b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21786624->21790719] (req 21786624->21790719) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760699.945973:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2d29b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21786624->21790719] (req 21786624->21790719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760699.945980:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760699.945985:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2d29b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21786624->21790719] (req 21786624->21790719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760699.945995:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2d29b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21786624->21790719] (req 21786624->21790719) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760699.946013:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2d29b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21786624->21790719] (req 21786624->21790719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760699.946022:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2d29b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21786624->21790719] (req 21786624->21790719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760699.946916:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febc6bd6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512538368:12345-192.168.203.56@tcp:4:dd.0 Request processed in 50502us (50906us total) trans 4294972638 rc 0/0 00000100:00100000:3.0:1789760699.946936:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5349 00000100:00100000:3.0:1789760699.946946:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 679413954355 time 679460688511 state 0x2 ref 0 00000100:00100000:3.0:1789760699.950417:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760699.951845:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 679460948927 time 679516132706 state 0x0 ref 1 00000100:00100000:0.0:1789760699.951862:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac405ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512538496:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760699.951914:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21790720, mtime: 0x6aad94bb, atime: 0x0, ctime: 0x6aad94bb, blocks: 0xa807 00010000:00010000:0.0:1789760699.951924:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2d2a2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760699.951941:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=679s v=(0 0) 00010000:00010000:0.0:1789760699.951946:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760699.951952:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2d2a2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21790720->21794815] (req 21790720->21794815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760699.951967:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2d2a2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21790720->21794815] (req 21790720->21794815) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760699.997344:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972639 is committed 00000100:00100000:0.0:1789760700.000476:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000001:00080000:0.0:1789760700.038207:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972640 is committed 00000100:00100000:3.0:1789760700.042988:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000001:00080000:0.0:1789760700.091581:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972641 is committed 00000100:00100000:0.0:1789760700.100280:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000001:00080000:0.0:1789760700.145398:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972642 is committed 00000100:00100000:2.0:1789760700.148515:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000001:00080000:3.0:1789760700.186390:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972643 is committed 00000100:00100000:0.0:1789760700.197074:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00002000:00010000:0.0:1789760700.267628:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21811200 -> 21815296 00010000:00010000:0.0:1789760700.267666:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2d2c5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21811200->21815295] (req 21811200->21815295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.267683:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2d2c5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21811200->21815295] (req 21811200->21815295) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.267730:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2d2c5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21811200->21815295] (req 21811200->21815295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.267740:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760700.267746:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2d2c5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21811200->21815295] (req 21811200->21815295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.267757:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2d2c5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21811200->21815295] (req 21811200->21815295) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.267782:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2d2c5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21811200->21815295] (req 21811200->21815295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.267791:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2d2c5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21811200->21815295] (req 21811200->21815295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760700.267919:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febfbeae300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2361:x1876699512539136:12345-192.168.203.56@tcp:4:dd.0 Request processed in 69997us (70836us total) trans 4294972644 rc 0/0 00000100:00100000:0.0:1789760700.267932:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5355 00000100:00100000:0.0:1789760700.267950:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 679713319879 time 679761856283 state 0x2 ref 0 00000100:00100000:0.0:1789760700.275234:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760700.275654:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512539264 00000100:00100000:2.0:1789760700.275667:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512539264 00000100:00100000:2.0:1789760700.275774:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 679713319879 time 679761856283 state 0x0 ref 1 00000100:00100000:2.0:1789760700.275795:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 679762116699 time 679761856283 state 0x1 ref 1 00000100:00100000:2.0:1789760700.275831:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 679762116699 time 679840130935 state 0x0 ref 1 00000100:00100000:2.0:1789760700.275856:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febc7b33480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512539264:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760700.275922:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21815296, mtime: 0x6aad94bb, atime: 0x0, ctime: 0x6aad94bb, blocks: 0xa807 00010000:00010000:2.0:1789760700.275932:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2d2cc lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760700.277630:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=679s v=(0 0) 00010000:00010000:2.0:1789760700.277637:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760700.277655:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2d2cc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21815296->21819391] (req 21815296->21819391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760700.277676:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2d2cc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21815296->21819391] (req 21815296->21819391) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760700.329529:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972645 is committed 00002000:00010000:2.0:1789760700.329605:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21815296 -> 21819392 00010000:00010000:2.0:1789760700.329625:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2d2cc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21815296->21819391] (req 21815296->21819391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760700.329640:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2d2cc lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21815296->21819391] (req 21815296->21819391) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760700.329734:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2d2cc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21815296->21819391] (req 21815296->21819391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760700.329748:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760700.329754:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2d2cc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21815296->21819391] (req 21815296->21819391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760700.329764:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2d2cc lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21815296->21819391] (req 21815296->21819391) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760700.329787:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2d2cc lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21815296->21819391] (req 21815296->21819391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760700.329795:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2d2cc lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21815296->21819391] (req 21815296->21819391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760700.330826:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febc7b33480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512539264:12345-192.168.203.56@tcp:4:dd.0 Request processed in 54970us (55602us total) trans 4294972645 rc 0/0 00000100:00100000:3.0:1789760700.330841:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5356 00000100:00100000:3.0:1789760700.330853:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 679762116699 time 679840130935 state 0x2 ref 0 00000100:00100000:3.0:1789760700.341251:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760700.342327:0:3312:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8feac4363100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3312:1876699540215680:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1789760700.342406:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760700.342799:0:6706:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540215680 00000100:00100000:0.0:1789760700.342810:0:6706:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540215680 00000100:00100000:0.0:1789760700.342894:0:6706:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 180 00000100:00100000:0.0:1789760700.342906:0:6706:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac405bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3312:x1876699540215680:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1789760700.342947:0:6706:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 526712832 pending 0 free 3836739584 avail 3805282304 00000100:00100000:0.0:1789760700.344296:0:6706:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac405bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3312:x1876699540215680:12345-0@lo:13:osp-pre-0-0.0 Request processed in 1389us (1886us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760700.344307:0:6706:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 180 00000100:00080000:0.0:1789760700.344518:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1789760700.344546:0:3310:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8feac5307b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3310:1876699540215936:0@lo:400:kworker.0 00000100:00100000:0.0:1789760700.344756:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760700.344799:0:3310:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1789760700.344805:0:3310:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8feac5306680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3310:1876699540215808:0@lo:400:kworker.0 00000100:00100000:0.0:1789760700.344828:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760700.344860:0:8229:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540215936 00000100:00100000:0.0:1789760700.344863:0:8229:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540215936 00000100:00100000:0.0:1789760700.344895:0:8229:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540215808 00000100:00100000:0.0:1789760700.344897:0:8229:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540215808 00000100:00100000:0.0:1789760700.344916:0:8229:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 431 00000100:00100000:0.0:1789760700.344922:0:8229:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febc7b30000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3310:x1876699540215936:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760700.345436:0:8229:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febc7b30000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3310:x1876699540215936:12345-0@lo:400:kworker.0 Request processed in 294us (460us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760700.345685:0:8229:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 431 00000100:00100000:0.0:1789760700.345777:0:8229:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 432 00000100:00100000:0.0:1789760700.345786:0:8229:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febc7b33800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3310:x1876699540215808:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760700.345826:0:8229:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febc7b33800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3310:x1876699540215808:12345-0@lo:400:kworker.0 Request processed in 42us (997us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760700.345832:0:8229:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 432 00000100:00080000:0.0:1789760700.345862:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789760700.345934:0:3312:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8feac4363100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3312:1876699540215680:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1789760700.346031:0:3312:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8feac5304000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3312:1876699540216192:0@lo:400:kworker.0 00000100:00100000:2.0:1789760700.346207:0:3310:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8feac5307b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3310:1876699540215936:0@lo:400:kworker.0 00000100:00100000:2.0:1789760700.346314:0:3310:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8feac5306680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3310:1876699540215808:0@lo:400:kworker.0 00000100:00100000:3.0:1789760700.346518:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760700.346752:0:3311:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8feac5304380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:6df98d54-eb52-4c5c-8fc9-ee9d74f90f98:3311:1876699540215552:0@lo:400:kworker.0 00000100:00100000:2.0:1789760700.346795:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760700.346902:0:6704:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540216192 00000100:00100000:0.0:1789760700.346909:0:6704:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540216192 00000100:00100000:3.0:1789760700.347184:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540215552 00000100:00100000:3.0:1789760700.347189:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540215552 00000100:00080000:2.0:1789760700.347296:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1789760700.347545:0:6705:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 166 00000100:00100000:3.0:1789760700.347685:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 381 00000100:00100000:3.0:1789760700.347716:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febe00d5c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:6df98d54-eb52-4c5c-8fc9-ee9d74f90f98+19:3311:x1876699540215552:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760700.347784:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febe00d5c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:6df98d54-eb52-4c5c-8fc9-ee9d74f90f98+19:3311:x1876699540215552:12345-0@lo:400:kworker.0 Request processed in 92us (988us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760700.347792:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 381 00000100:00100000:3.0:1789760700.347851:0:3311:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8feac5304380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:6df98d54-eb52-4c5c-8fc9-ee9d74f90f98:3311:1876699540215552:0@lo:400:kworker.0 00000100:00100000:3.0:1789760700.347860:0:3311:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8feac5307800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3311:1876699540216064:0@lo:400:kworker.0 00000100:00100000:3.0:1789760700.347890:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760700.348100:0:6705:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac405aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3312:x1876699540216192:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760700.352759:0:3312:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8feac5304000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3312:1876699540216192:0@lo:400:kworker.0 00000100:00100000:2.0:1789760700.352846:0:8229:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540216064 00000100:00100000:2.0:1789760700.352857:0:8229:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540216064 00000100:00100000:2.0:1789760700.352933:0:8229:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 433 00000100:00100000:0.0:1789760700.353118:0:6705:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac405aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3312:x1876699540216192:12345-0@lo:400:kworker.0 Request processed in 5087us (6596us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760700.353398:0:6705:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 166 00000100:00100000:2.0:1789760700.354835:0:8229:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febfb895500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3311:x1876699540216064:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760700.356775:0:8229:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febfb895500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3311:x1876699540216064:12345-0@lo:400:kworker.0 Request processed in 1945us (8871us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760700.356793:0:8229:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 433 00000100:00100000:3.0:1789760700.357807:0:3311:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8feac5307800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3311:1876699540216064:0@lo:400:kworker.0 00000100:00080000:2.0:1789760700.358795:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000001:00080000:0.0:1789760700.401825:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972646 is committed 00000100:00100000:0.0:1789760700.412213:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760700.412329:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512539520 00000100:00100000:3.0:1789760700.412341:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512539520 00000100:00100000:3.0:1789760700.412416:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 679840391351 time 679908682960 state 0x0 ref 1 00000100:00100000:3.0:1789760700.412436:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 679908943376 time 679908682960 state 0x1 ref 1 00000100:00100000:3.0:1789760700.412477:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 679908943376 time 679976776742 state 0x0 ref 1 00000100:00100000:3.0:1789760700.412492:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febc7b32680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512539520:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760700.412561:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21823488, mtime: 0x6aad94bb, atime: 0x0, ctime: 0x6aad94bb, blocks: 0xa807 00010000:00010000:3.0:1789760700.412572:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2d2da lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760700.412593:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=679s v=(0 0) 00010000:00010000:3.0:1789760700.412598:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760700.412606:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2d2da lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21823488->21827583] (req 21823488->21827583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760700.412622:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2d2da lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21823488->21827583] (req 21823488->21827583) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760700.475389:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972647 is committed 00002000:00010000:2.0:1789760700.475442:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21823488 -> 21827584 00010000:00010000:2.0:1789760700.475461:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2d2da lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21823488->21827583] (req 21823488->21827583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760700.475476:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2d2da lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21823488->21827583] (req 21823488->21827583) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760700.475486:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2d2da lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21823488->21827583] (req 21823488->21827583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760700.475494:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760700.475500:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2d2da lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21823488->21827583] (req 21823488->21827583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760700.475510:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2d2da lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21823488->21827583] (req 21823488->21827583) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760700.475537:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2d2da lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21823488->21827583] (req 21823488->21827583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760700.475546:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2d2da lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21823488->21827583] (req 21823488->21827583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760700.476103:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febc7b32680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512539520:12345-192.168.203.56@tcp:4:dd.0 Request processed in 63545us (63834us total) trans 4294972647 rc 0/0 00000100:00100000:2.0:1789760700.476116:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5358 00000100:00100000:2.0:1789760700.476129:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 679908943376 time 679976776742 state 0x2 ref 0 00000100:00100000:0.0:1789760700.482393:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760700.483197:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512539648 00000100:00100000:3.0:1789760700.483211:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512539648 00000100:00100000:3.0:1789760700.483290:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 679908943376 time 679976776742 state 0x0 ref 1 00000100:00100000:3.0:1789760700.483308:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 679977037158 time 679976776742 state 0x1 ref 1 00000100:00100000:3.0:1789760700.483353:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 679977037158 time 680047652613 state 0x0 ref 1 00000100:00100000:3.0:1789760700.483368:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febd1362a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512539648:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760700.483449:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21827584, mtime: 0x6aad94bb, atime: 0x0, ctime: 0x6aad94bb, blocks: 0xa807 00010000:00010000:3.0:1789760700.483463:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2d2e1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760700.483491:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=680s v=(0 0) 00010000:00010000:3.0:1789760700.483497:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760700.483506:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2d2e1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21827584->21831679] (req 21827584->21831679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760700.483523:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2d2e1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21827584->21831679] (req 21827584->21831679) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760700.548041:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972648 is committed 00000100:00100000:0.0:1789760700.558680:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000001:00080000:0.0:1789760700.650801:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972649 is committed 00000100:00100000:0.0:1789760700.656543:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512539904 00000100:00100000:0.0:1789760700.656556:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512539904 00000100:00100000:0.0:1789760700.656627:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 680047913029 time 680124955652 state 0x0 ref 1 00000100:00100000:0.0:1789760700.656655:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 680125216068 time 680124955652 state 0x1 ref 1 00000100:00100000:0.0:1789760700.656694:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 680125216068 time 680220992852 state 0x0 ref 1 00000100:00100000:0.0:1789760700.656731:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5307480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512539904:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760700.656795:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21835776, mtime: 0x6aad94bc, atime: 0x0, ctime: 0x6aad94bc, blocks: 0xa807 00010000:00010000:0.0:1789760700.656806:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2d2ef lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.656828:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=680s v=(0 0) 00010000:00010000:0.0:1789760700.656835:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760700.656843:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2d2ef lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21835776->21839871] (req 21835776->21839871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.656860:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2d2ef lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21835776->21839871] (req 21835776->21839871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760700.779256:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21835776 -> 21839872 00010000:00010000:0.0:1789760700.779275:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2d2ef lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21835776->21839871] (req 21835776->21839871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.779292:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2d2ef lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21835776->21839871] (req 21835776->21839871) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.779302:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2d2ef lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21835776->21839871] (req 21835776->21839871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.779383:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760700.779390:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2d2ef lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21835776->21839871] (req 21835776->21839871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.779401:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2d2ef lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21835776->21839871] (req 21835776->21839871) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.779425:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2d2ef lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21835776->21839871] (req 21835776->21839871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.779433:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2d2ef lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21835776->21839871] (req 21835776->21839871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760700.779601:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5307480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512539904:12345-192.168.203.56@tcp:4:dd.0 Request processed in 122865us (123262us total) trans 4294972650 rc 0/0 00000100:00100000:0.0:1789760700.779615:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5361 00000100:00100000:0.0:1789760700.779627:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 680125216068 time 680220992852 state 0x2 ref 0 00000100:00100000:3.0:1789760700.786845:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00010000:00010000:0.0:1789760700.788289:0:7647:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:2.0:1789760700.848962:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972651 is committed 00000100:00100000:3.0:1789760700.850601:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760700.851916:0:3309:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8febfb377100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3309:1876699540216320:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789760700.851996:0:3309:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760700.852103:0:6706:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540216320 00000100:00100000:2.0:1789760700.852110:0:6706:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540216320 00000100:00100000:2.0:1789760700.852158:0:6706:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 181 00000100:00100000:2.0:1789760700.852167:0:6706:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4363100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3309:x1876699540216320:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1789760700.852202:0:6706:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3859808256 avail 3857711104 00000100:00100000:2.0:1789760700.852268:0:6706:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4363100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3309:x1876699540216320:12345-0@lo:13:osp-pre-1-0.0 Request processed in 101us (270us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760700.852276:0:6706:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 181 00000100:00100000:2.0:1789760700.852332:0:3309:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8febfb377100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3309:1876699540216320:0@lo:13:osp-pre-1-0.0 00000001:00080000:2.0:1789760700.944922:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972652 is committed 00000100:00100000:0.0:1789760700.949939:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760700.951043:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512540288 00000100:00100000:2.0:1789760700.951061:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512540288 00000100:00100000:2.0:1789760700.951145:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 680351681514 time 680415189008 state 0x0 ref 1 00000100:00100000:2.0:1789760700.951166:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 680415449424 time 680415189008 state 0x1 ref 1 00000100:00100000:2.0:1789760700.951263:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 680415449424 time 680515560667 state 0x0 ref 1 00000100:00100000:2.0:1789760700.951282:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febfb374380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512540288:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760700.951348:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21848064, mtime: 0x6aad94bc, atime: 0x0, ctime: 0x6aad94bc, blocks: 0xa807 00010000:00010000:2.0:1789760700.951359:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2d304 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760700.951381:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=680s v=(0 0) 00010000:00010000:2.0:1789760700.951386:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760700.951394:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2d304 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21848064->21852159] (req 21848064->21852159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760700.951413:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2d304 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21848064->21852159] (req 21848064->21852159) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760700.996238:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21848064 -> 21852160 00010000:00010000:3.0:1789760700.996260:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2d304 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21848064->21852159] (req 21848064->21852159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760700.996277:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2d304 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21848064->21852159] (req 21848064->21852159) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760700.996286:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2d304 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21848064->21852159] (req 21848064->21852159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760700.996291:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760700.996295:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2d304 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21848064->21852159] (req 21848064->21852159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760700.996302:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2d304 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21848064->21852159] (req 21848064->21852159) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760700.996324:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2d304 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21848064->21852159] (req 21848064->21852159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760700.996330:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2d304 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21848064->21852159] (req 21848064->21852159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760700.996439:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febfb374380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512540288:12345-192.168.203.56@tcp:4:dd.0 Request processed in 45155us (46519us total) trans 4294972653 rc 0/0 00000100:00100000:3.0:1789760700.996451:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5364 00000100:00100000:3.0:1789760700.996462:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 680415449424 time 680515560667 state 0x2 ref 0 00000100:00100000:0.0:1789760701.004840:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00080000:2.0:1789760701.005923:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:0.0:1789760701.091386:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972654 is committed 00002000:00010000:0.0:1789760701.092027:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21852160 -> 21856256 00010000:00010000:0.0:1789760701.092049:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2d30b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21852160->21856255] (req 21852160->21856255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.092065:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2d30b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21852160->21856255] (req 21852160->21856255) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.092075:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2d30b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21852160->21856255] (req 21852160->21856255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.092084:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760701.092090:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2d30b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21852160->21856255] (req 21852160->21856255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.092100:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2d30b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21852160->21856255] (req 21852160->21856255) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.092131:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2d30b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21852160->21856255] (req 21852160->21856255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.092140:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2d30b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21852160->21856255] (req 21852160->21856255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760701.092392:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febfb375500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512540416:12345-192.168.203.56@tcp:4:dd.0 Request processed in 87199us (87560us total) trans 4294972654 rc 0/0 00000100:00100000:0.0:1789760701.092406:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5365 00000100:00100000:0.0:1789760701.092419:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 680515821083 time 680569474385 state 0x2 ref 0 00000100:00100000:0.0:1789760701.096816:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000001:00080000:0.0:1789760701.164959:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972655 is committed 00000100:00100000:3.0:1789760701.170929:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512540672 00000100:00100000:3.0:1789760701.170945:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512540672 00000100:00100000:3.0:1789760701.171019:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 680569734801 time 680661469096 state 0x0 ref 1 00000100:00100000:3.0:1789760701.171155:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 680661729512 time 680661469096 state 0x1 ref 1 00000100:00100000:3.0:1789760701.171242:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 680661729512 time 680735540337 state 0x0 ref 1 00000100:00100000:3.0:1789760701.171257:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5304700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512540672:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760701.171332:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21860352, mtime: 0x6aad94bc, atime: 0x0, ctime: 0x6aad94bc, blocks: 0xa807 00010000:00010000:3.0:1789760701.171343:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2d319 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760701.171366:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=680s v=(0 0) 00010000:00010000:3.0:1789760701.171371:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760701.171380:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2d319 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21860352->21864447] (req 21860352->21864447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760701.171403:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2d319 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21860352->21864447] (req 21860352->21864447) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760701.258086:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21860352 -> 21864448 00010000:00010000:2.0:1789760701.258109:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2d319 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21860352->21864447] (req 21860352->21864447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760701.258124:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2d319 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21860352->21864447] (req 21860352->21864447) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760701.258195:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2d319 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21860352->21864447] (req 21860352->21864447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760701.258205:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760701.258211:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2d319 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21860352->21864447] (req 21860352->21864447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760701.258223:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2d319 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21860352->21864447] (req 21860352->21864447) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760701.258248:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2d319 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21860352->21864447] (req 21860352->21864447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760701.258255:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2d319 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21860352->21864447] (req 21860352->21864447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760701.258516:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5304700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512540672:12345-192.168.203.56@tcp:4:dd.0 Request processed in 87255us (88029us total) trans 4294972656 rc 0/0 00000100:00100000:2.0:1789760701.258528:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5367 00000100:00100000:2.0:1789760701.258541:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 680661729512 time 680735540337 state 0x2 ref 0 00000100:00100000:2.0:1789760701.268661:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760701.269002:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512540800 00000100:00100000:0.0:1789760701.269018:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512540800 00000001:00080000:0.0:1789760701.356963:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972657 is committed 00000100:00100000:2.0:1789760701.363814:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512540928 00000100:00100000:2.0:1789760701.363828:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512540928 00000100:00100000:2.0:1789760701.363969:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 680735800753 time 680833773104 state 0x0 ref 1 00000100:00100000:2.0:1789760701.363989:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 680834033520 time 680833773104 state 0x1 ref 1 00000100:00100000:2.0:1789760701.364243:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 680834033520 time 680928434352 state 0x0 ref 1 00000100:00100000:2.0:1789760701.364262:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5306a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512540928:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760701.364348:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21868544, mtime: 0x6aad94bc, atime: 0x0, ctime: 0x6aad94bc, blocks: 0xa807 00010000:00010000:2.0:1789760701.364360:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2d327 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760701.364380:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=680s v=(0 0) 00010000:00010000:2.0:1789760701.364386:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760701.364393:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2d327 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21868544->21872639] (req 21868544->21872639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760701.364408:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2d327 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21868544->21872639] (req 21868544->21872639) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760701.425483:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21868544 -> 21872640 00010000:00010000:3.0:1789760701.425512:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2d327 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21868544->21872639] (req 21868544->21872639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760701.425528:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2d327 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21868544->21872639] (req 21868544->21872639) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760701.425538:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2d327 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21868544->21872639] (req 21868544->21872639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760701.425546:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760701.425552:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2d327 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21868544->21872639] (req 21868544->21872639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760701.425563:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2d327 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21868544->21872639] (req 21868544->21872639) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760701.425586:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2d327 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21868544->21872639] (req 21868544->21872639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760701.425595:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2d327 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21868544->21872639] (req 21868544->21872639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760701.425767:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5306a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2359:x1876699512540928:12345-192.168.203.56@tcp:4:dd.0 Request processed in 61503us (62089us total) trans 4294972658 rc 0/0 00000100:00100000:3.0:1789760701.425779:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5369 00000100:00100000:3.0:1789760701.425791:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 680834033520 time 680928434352 state 0x2 ref 0 00000100:00100000:3.0:1789760701.429504:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000001:00080000:0.0:1789760701.429731:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972658 is committed 00000001:00080000:0.0:1789760701.476486:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972659 is committed 00000100:00100000:2.0:1789760701.480414:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512541184 00000100:00100000:2.0:1789760701.480426:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512541184 00000100:00100000:2.0:1789760701.480500:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 680928694768 time 680994460379 state 0x0 ref 1 00000100:00100000:2.0:1789760701.480517:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 680994720795 time 680994460379 state 0x1 ref 1 00000100:00100000:2.0:1789760701.480550:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 680994720795 time 681044849135 state 0x0 ref 1 00000100:00100000:2.0:1789760701.480566:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5305880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512541184:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760701.480648:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21876736, mtime: 0x6aad94bc, atime: 0x0, ctime: 0x6aad94bc, blocks: 0xa807 00010000:00010000:2.0:1789760701.480659:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2d335 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760701.480679:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=681s v=(0 0) 00010000:00010000:2.0:1789760701.480685:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760701.480693:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2d335 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21876736->21880831] (req 21876736->21880831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760701.480826:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2d335 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21876736->21880831] (req 21876736->21880831) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760701.505931:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972660 is committed 00000100:00100000:2.0:1789760701.509085:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760701.509285:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512541312 00000100:00100000:3.0:1789760701.509298:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512541312 00000100:00100000:3.0:1789760701.509434:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 680994720795 time 681044849135 state 0x0 ref 1 00000100:00100000:3.0:1789760701.509452:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 681045109551 time 681044849135 state 0x1 ref 1 00000100:00100000:3.0:1789760701.509490:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 681045109551 time 681073788710 state 0x0 ref 1 00000100:00100000:3.0:1789760701.509507:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febea650380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512541312:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760701.509573:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21880832, mtime: 0x6aad94bc, atime: 0x0, ctime: 0x6aad94bc, blocks: 0xa807 00010000:00010000:3.0:1789760701.509584:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2d33c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760701.509604:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=681s v=(0 0) 00010000:00010000:3.0:1789760701.509609:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760701.509617:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2d33c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21880832->21884927] (req 21880832->21884927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760701.509649:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2d33c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21880832->21884927] (req 21880832->21884927) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760701.538773:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972661 is committed 00010000:00010000:1.0:1789760701.538918:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2d33c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21880832->21884927] (req 21880832->21884927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760701.539161:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febea650380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512541312:12345-192.168.203.56@tcp:4:dd.0 Request processed in 29650us (30082us total) trans 4294972661 rc 0/0 00000100:00100000:1.0:1789760701.539176:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5372 00000100:00100000:1.0:1789760701.539189:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 681045109551 time 681073788710 state 0x2 ref 0 00000100:00100000:3.0:1789760701.544514:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760701.545776:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512541440 00000100:00100000:1.0:1789760701.545800:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512541440 00000100:00100000:0.0:1789760701.545959:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 681045109551 time 681073788710 state 0x0 ref 1 00000100:00100000:0.0:1789760701.545978:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 681074049126 time 681073788710 state 0x1 ref 1 00000100:00100000:0.0:1789760701.546011:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 681074049126 time 681110310670 state 0x0 ref 1 00000100:00100000:0.0:1789760701.546026:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2744380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512541440:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760701.546093:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21884928, mtime: 0x6aad94bd, atime: 0x0, ctime: 0x6aad94bd, blocks: 0xa807 00010000:00010000:0.0:1789760701.546105:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2d343 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.546127:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=681s v=(0 0) 00010000:00010000:0.0:1789760701.546132:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760701.546140:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2d343 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21884928->21889023] (req 21884928->21889023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.546158:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2d343 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21884928->21889023] (req 21884928->21889023) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760701.581830:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21884928 -> 21889024 00010000:00010000:0.0:1789760701.581853:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2d343 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21884928->21889023] (req 21884928->21889023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.581865:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2d343 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21884928->21889023] (req 21884928->21889023) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.581873:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2d343 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21884928->21889023] (req 21884928->21889023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.581878:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760701.581883:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2d343 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21884928->21889023] (req 21884928->21889023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.581890:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2d343 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21884928->21889023] (req 21884928->21889023) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.581909:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2d343 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21884928->21889023] (req 21884928->21889023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.581915:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2d343 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21884928->21889023] (req 21884928->21889023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760701.582015:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2744380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2360:x1876699512541440:12345-192.168.203.56@tcp:4:dd.0 Request processed in 35987us (37512us total) trans 4294972662 rc 0/0 00000100:00100000:0.0:1789760701.582025:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5373 00000100:00100000:0.0:1789760701.582034:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 681074049126 time 681110310670 state 0x2 ref 0 00000001:00080000:3.0:1789760701.582476:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972662 is committed 00000100:00100000:2.0:1789760701.589431:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760701.590068:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512541568 00000100:00100000:1.0:1789760701.590084:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512541568 00000100:00100000:1.0:1789760701.590162:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 681074049126 time 681110310670 state 0x0 ref 1 00000100:00100000:1.0:1789760701.590191:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 681110571086 time 681110310670 state 0x1 ref 1 00000100:00100000:1.0:1789760701.590268:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 681110571086 time 681154566463 state 0x0 ref 1 00000100:00100000:1.0:1789760701.590283:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac3352300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512541568:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760701.590350:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21889024, mtime: 0x6aad94bd, atime: 0x0, ctime: 0x6aad94bd, blocks: 0xa807 00010000:00010000:1.0:1789760701.590361:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2d34a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.590381:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=681s v=(0 0) 00010000:00010000:1.0:1789760701.590386:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760701.590394:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2d34a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21889024->21893119] (req 21889024->21893119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.590412:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2d34a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21889024->21893119] (req 21889024->21893119) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760701.620119:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972663 is committed 00002000:00010000:1.0:1789760701.620264:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21889024 -> 21893120 00010000:00010000:1.0:1789760701.620285:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2d34a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21889024->21893119] (req 21889024->21893119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.620301:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2d34a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21889024->21893119] (req 21889024->21893119) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.620312:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2d34a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21889024->21893119] (req 21889024->21893119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.620325:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760701.620331:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2d34a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21889024->21893119] (req 21889024->21893119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.620342:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2d34a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21889024->21893119] (req 21889024->21893119) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.620365:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2d34a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21889024->21893119] (req 21889024->21893119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.620373:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2d34a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21889024->21893119] (req 21889024->21893119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760701.620532:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac3352300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512541568:12345-192.168.203.56@tcp:4:dd.0 Request processed in 30245us (31109us total) trans 4294972663 rc 0/0 00000100:00100000:1.0:1789760701.620548:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5374 00000100:00100000:1.0:1789760701.620560:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 681110571086 time 681154566463 state 0x2 ref 0 00000100:00100000:2.0:1789760701.629605:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760701.629778:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512541696 00000100:00100000:1.0:1789760701.629791:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512541696 00000100:00100000:1.0:1789760701.629862:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 681110571086 time 681154566463 state 0x0 ref 1 00000100:00100000:1.0:1789760701.629881:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 681154826879 time 681154566463 state 0x1 ref 1 00000100:00100000:1.0:1789760701.629923:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 681154826879 time 681194221915 state 0x0 ref 1 00000100:00100000:1.0:1789760701.629937:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac3353800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512541696:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760701.630008:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21893120, mtime: 0x6aad94bd, atime: 0x0, ctime: 0x6aad94bd, blocks: 0xa807 00010000:00010000:1.0:1789760701.630019:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2d351 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.630039:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=681s v=(0 0) 00010000:00010000:1.0:1789760701.630043:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760701.630050:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2d351 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21893120->21897215] (req 21893120->21897215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.630079:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2d351 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21893120->21897215] (req 21893120->21897215) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760701.680158:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972664 is committed 00002000:00010000:1.0:1789760701.680471:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21893120 -> 21897216 00010000:00010000:1.0:1789760701.680493:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2d351 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21893120->21897215] (req 21893120->21897215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.680509:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2d351 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21893120->21897215] (req 21893120->21897215) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.680520:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2d351 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21893120->21897215] (req 21893120->21897215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.680528:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760701.680535:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2d351 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21893120->21897215] (req 21893120->21897215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.680619:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2d351 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21893120->21897215] (req 21893120->21897215) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.680657:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2d351 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21893120->21897215] (req 21893120->21897215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.680667:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2d351 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21893120->21897215] (req 21893120->21897215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760701.680842:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac3353800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512541696:12345-192.168.203.56@tcp:4:dd.0 Request processed in 50900us (51245us total) trans 4294972664 rc 0/0 00000100:00100000:1.0:1789760701.680855:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5375 00000100:00100000:1.0:1789760701.680866:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 681154826879 time 681194221915 state 0x2 ref 0 00000100:00100000:1.0:1789760701.685004:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760701.685083:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512541824 00000100:00100000:0.0:1789760701.685096:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512541824 00000100:00100000:0.0:1789760701.685165:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 681154826879 time 681194221915 state 0x0 ref 1 00000100:00100000:0.0:1789760701.685184:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 681194482331 time 681194221915 state 0x1 ref 1 00000100:00100000:0.0:1789760701.685221:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 681194482331 time 681249520251 state 0x0 ref 1 00000100:00100000:0.0:1789760701.685235:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5305180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512541824:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760701.685301:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21897216, mtime: 0x6aad94bd, atime: 0x0, ctime: 0x6aad94bd, blocks: 0xa807 00010000:00010000:0.0:1789760701.685312:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2d358 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.685334:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=681s v=(0 0) 00010000:00010000:0.0:1789760701.685340:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760701.685349:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2d358 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21897216->21901311] (req 21897216->21901311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.685366:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2d358 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21897216->21901311] (req 21897216->21901311) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760701.733792:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972665 is committed 00002000:00010000:0.0:1789760701.733914:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21897216 -> 21901312 00010000:00010000:0.0:1789760701.733934:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2d358 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21897216->21901311] (req 21897216->21901311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.733950:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2d358 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21897216->21901311] (req 21897216->21901311) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.733960:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2d358 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21897216->21901311] (req 21897216->21901311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.733967:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760701.733973:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2d358 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21897216->21901311] (req 21897216->21901311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.733982:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2d358 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21897216->21901311] (req 21897216->21901311) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.734006:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2d358 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21897216->21901311] (req 21897216->21901311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.734014:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2d358 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21897216->21901311] (req 21897216->21901311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760701.734236:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5305180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512541824:12345-192.168.203.56@tcp:4:dd.0 Request processed in 48996us (49237us total) trans 4294972665 rc 0/0 00000100:00100000:0.0:1789760701.734250:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5376 00000100:00100000:0.0:1789760701.734262:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 681194482331 time 681249520251 state 0x2 ref 0 00000100:00100000:3.0:1789760701.736900:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760701.736966:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512541952 00000100:00100000:1.0:1789760701.736979:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512541952 00000100:00100000:1.0:1789760701.737105:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 681194482331 time 681249520251 state 0x0 ref 1 00000100:00100000:1.0:1789760701.737124:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 681249780667 time 681249520251 state 0x1 ref 1 00000100:00100000:1.0:1789760701.737161:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 681249780667 time 681301461347 state 0x0 ref 1 00000100:00100000:1.0:1789760701.737175:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2746680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512541952:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760701.737236:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21901312, mtime: 0x6aad94bd, atime: 0x0, ctime: 0x6aad94bd, blocks: 0xa807 00010000:00010000:1.0:1789760701.737246:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2d35f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.737266:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=681s v=(0 0) 00010000:00010000:1.0:1789760701.737272:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760701.737281:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2d35f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21901312->21905407] (req 21901312->21905407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.737297:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2d35f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21901312->21905407] (req 21901312->21905407) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760701.768589:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972666 is committed 00002000:00010000:1.0:1789760701.769409:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21901312 -> 21905408 00010000:00010000:1.0:1789760701.769432:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2d35f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21901312->21905407] (req 21901312->21905407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.769450:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2d35f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21901312->21905407] (req 21901312->21905407) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.769466:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2d35f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21901312->21905407] (req 21901312->21905407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.769475:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760701.769480:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2d35f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21901312->21905407] (req 21901312->21905407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.769491:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2d35f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21901312->21905407] (req 21901312->21905407) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.769520:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2d35f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21901312->21905407] (req 21901312->21905407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.769529:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2d35f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21901312->21905407] (req 21901312->21905407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760701.769667:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2746680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512541952:12345-192.168.203.56@tcp:4:dd.0 Request processed in 32487us (32774us total) trans 4294972666 rc 0/0 00000100:00100000:1.0:1789760701.769686:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5377 00000100:00100000:1.0:1789760701.769752:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 681249780667 time 681301461347 state 0x2 ref 0 00000100:00100000:2.0:1789760701.777925:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760701.778180:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512542080 00000100:00100000:1.0:1789760701.778195:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512542080 00000100:00100000:1.0:1789760701.778272:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 681249780667 time 681301461347 state 0x0 ref 1 00000100:00100000:1.0:1789760701.778293:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 681301721763 time 681301461347 state 0x1 ref 1 00000100:00100000:1.0:1789760701.778455:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 681301721763 time 681342752649 state 0x0 ref 1 00000100:00100000:1.0:1789760701.778470:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac3352a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512542080:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760701.778543:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21905408, mtime: 0x6aad94bd, atime: 0x0, ctime: 0x6aad94bd, blocks: 0xa807 00010000:00010000:1.0:1789760701.778554:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2d366 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.778573:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=681s v=(0 0) 00010000:00010000:1.0:1789760701.778579:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760701.778587:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2d366 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21905408->21909503] (req 21905408->21909503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.778604:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2d366 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21905408->21909503] (req 21905408->21909503) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.810808:0:5836:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:1.0:1789760701.841930:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21905408 -> 21909504 00010000:00010000:1.0:1789760701.841959:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2d366 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21905408->21909503] (req 21905408->21909503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.841975:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2d366 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21905408->21909503] (req 21905408->21909503) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.841986:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2d366 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21905408->21909503] (req 21905408->21909503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.841994:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760701.842001:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2d366 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21905408->21909503] (req 21905408->21909503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.842013:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2d366 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21905408->21909503] (req 21905408->21909503) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.842039:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2d366 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21905408->21909503] (req 21905408->21909503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.842048:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2d366 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21905408->21909503] (req 21905408->21909503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760701.842182:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac3352a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2359:x1876699512542080:12345-192.168.203.56@tcp:4:dd.0 Request processed in 63708us (64267us total) trans 4294972667 rc 0/0 00000100:00100000:1.0:1789760701.842195:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5378 00000100:00100000:1.0:1789760701.842208:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 681301721763 time 681342752649 state 0x2 ref 0 00000001:00080000:2.0:1789760701.842327:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972667 is committed 00000100:00100000:3.0:1789760701.856415:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760701.856983:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512542208 00000100:00100000:0.0:1789760701.856997:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512542208 00000100:00100000:1.0:1789760701.857856:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 681301721763 time 681342752649 state 0x0 ref 1 00000100:00100000:1.0:1789760701.857878:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 681343013065 time 681342752649 state 0x1 ref 1 00000100:00100000:1.0:1789760701.857966:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 681343013065 time 681422264033 state 0x0 ref 1 00000100:00100000:1.0:1789760701.857982:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2747b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512542208:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760701.858051:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21909504, mtime: 0x6aad94bd, atime: 0x0, ctime: 0x6aad94bd, blocks: 0xa807 00010000:00010000:1.0:1789760701.858062:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2d36d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.858085:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=681s v=(0 0) 00010000:00010000:1.0:1789760701.858091:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760701.858099:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2d36d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21909504->21913599] (req 21909504->21913599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.858117:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2d36d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21909504->21913599] (req 21909504->21913599) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760701.907483:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972668 is committed 00002000:00010000:1.0:1789760701.907733:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21909504 -> 21913600 00010000:00010000:1.0:1789760701.907753:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2d36d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21909504->21913599] (req 21909504->21913599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.907769:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2d36d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21909504->21913599] (req 21909504->21913599) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.907780:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2d36d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21909504->21913599] (req 21909504->21913599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.907789:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760701.907796:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2d36d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21909504->21913599] (req 21909504->21913599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.907810:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2d36d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21909504->21913599] (req 21909504->21913599) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.907837:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2d36d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21909504->21913599] (req 21909504->21913599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.907847:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2d36d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21909504->21913599] (req 21909504->21913599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760701.907972:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2747b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512542208:12345-192.168.203.56@tcp:4:dd.0 Request processed in 49987us (51565us total) trans 4294972668 rc 0/0 00000100:00100000:1.0:1789760701.907984:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5379 00000100:00100000:1.0:1789760701.907996:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 681343013065 time 681422264033 state 0x2 ref 0 00000100:00100000:3.0:1789760701.912983:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760701.913059:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512542336 00000100:00100000:1.0:1789760701.913074:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512542336 00000100:00100000:1.0:1789760701.913151:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 681343013065 time 681422264033 state 0x0 ref 1 00000100:00100000:1.0:1789760701.913171:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 681422524449 time 681422264033 state 0x1 ref 1 00000100:00100000:1.0:1789760701.913206:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 681422524449 time 681477505656 state 0x0 ref 1 00000100:00100000:1.0:1789760701.913219:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2747800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512542336:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760701.913283:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21913600, mtime: 0x6aad94bd, atime: 0x0, ctime: 0x6aad94bd, blocks: 0xa807 00010000:00010000:1.0:1789760701.913295:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2d374 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.913317:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=681s v=(0 0) 00010000:00010000:1.0:1789760701.913322:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760701.913329:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2d374 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21913600->21917695] (req 21913600->21917695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.913351:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2d374 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21913600->21917695] (req 21913600->21917695) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760701.948591:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972669 is committed 00002000:00010000:1.0:1789760701.948772:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21913600 -> 21917696 00010000:00010000:1.0:1789760701.948792:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2d374 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21913600->21917695] (req 21913600->21917695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.948819:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2d374 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21913600->21917695] (req 21913600->21917695) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.948829:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2d374 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21913600->21917695] (req 21913600->21917695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.948838:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760701.948844:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2d374 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21913600->21917695] (req 21913600->21917695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.948854:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2d374 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21913600->21917695] (req 21913600->21917695) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.948877:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2d374 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21913600->21917695] (req 21913600->21917695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.948884:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2d374 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21913600->21917695] (req 21913600->21917695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760701.949005:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2747800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512542336:12345-192.168.203.56@tcp:4:dd.0 Request processed in 35780us (36039us total) trans 4294972669 rc 0/0 00000100:00100000:1.0:1789760701.949017:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5380 00000100:00100000:1.0:1789760701.949030:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 681422524449 time 681477505656 state 0x2 ref 0 00000100:00100000:2.0:1789760701.953528:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760701.953588:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512542464 00000100:00100000:1.0:1789760701.953602:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512542464 00000100:00100000:1.0:1789760701.953688:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 681422524449 time 681477505656 state 0x0 ref 1 00000100:00100000:1.0:1789760701.953744:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 681477766072 time 681477505656 state 0x1 ref 1 00000100:00100000:1.0:1789760701.953770:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 681477766072 time 681518070224 state 0x0 ref 1 00000100:00100000:1.0:1789760701.953784:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac3353480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512542464:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760701.953909:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21917696, mtime: 0x6aad94bd, atime: 0x0, ctime: 0x6aad94bd, blocks: 0xa807 00010000:00010000:3.0:1789760701.953922:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2d37b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760701.953942:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=681s v=(0 0) 00010000:00010000:3.0:1789760701.953948:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760701.953956:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2d37b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21917696->21921791] (req 21917696->21921791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760701.953972:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2d37b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21917696->21921791] (req 21917696->21921791) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760702.000076:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972670 is committed 00002000:00010000:1.0:1789760702.000213:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21917696 -> 21921792 00010000:00010000:1.0:1789760702.000234:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2d37b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21917696->21921791] (req 21917696->21921791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.000249:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2d37b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21917696->21921791] (req 21917696->21921791) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.000259:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2d37b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21917696->21921791] (req 21917696->21921791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.000268:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760702.000275:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2d37b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21917696->21921791] (req 21917696->21921791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.000286:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2d37b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21917696->21921791] (req 21917696->21921791) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.000333:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2d37b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21917696->21921791] (req 21917696->21921791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.000343:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2d37b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21917696->21921791] (req 21917696->21921791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760702.000466:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac3353480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512542464:12345-192.168.203.56@tcp:4:dd.0 Request processed in 46675us (46943us total) trans 4294972670 rc 0/0 00000100:00100000:1.0:1789760702.000478:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5381 00000100:00100000:1.0:1789760702.000490:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 681477766072 time 681518070224 state 0x2 ref 0 00000100:00100000:3.0:1789760702.005506:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760702.005882:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512542592 00000100:00100000:1.0:1789760702.005899:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512542592 00000100:00100000:1.0:1789760702.005969:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 681477766072 time 681518070224 state 0x0 ref 1 00000100:00100000:1.0:1789760702.005988:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 681518330640 time 681518070224 state 0x1 ref 1 00000100:00100000:1.0:1789760702.006025:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 681518330640 time 681570324016 state 0x0 ref 1 00000100:00100000:1.0:1789760702.006048:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2744000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512542592:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760702.006114:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21921792, mtime: 0x6aad94bd, atime: 0x0, ctime: 0x6aad94bd, blocks: 0xa807 00010000:00010000:1.0:1789760702.006126:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2d382 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.006156:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=681s v=(0 0) 00010000:00010000:1.0:1789760702.006162:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760702.006170:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2d382 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21921792->21925887] (req 21921792->21925887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.006186:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2d382 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21921792->21925887] (req 21921792->21925887) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760702.041510:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21921792 -> 21925888 00010000:00010000:1.0:1789760702.041538:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2d382 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21921792->21925887] (req 21921792->21925887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.041555:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2d382 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21921792->21925887] (req 21921792->21925887) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.041566:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2d382 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21921792->21925887] (req 21921792->21925887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.041645:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760702.041651:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2d382 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21921792->21925887] (req 21921792->21925887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.041663:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2d382 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21921792->21925887] (req 21921792->21925887) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.041729:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2d382 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21921792->21925887] (req 21921792->21925887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.041739:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2d382 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21921792->21925887] (req 21921792->21925887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760702.041846:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2744000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2358:x1876699512542592:12345-192.168.203.56@tcp:4:dd.0 Request processed in 35794us (36350us total) trans 4294972671 rc 0/0 00000100:00100000:1.0:1789760702.041858:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5382 00000100:00100000:1.0:1789760702.041870:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 681518330640 time 681570324016 state 0x2 ref 0 00000001:00080000:3.0:1789760702.041960:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972671 is committed 00000100:00100000:2.0:1789760702.048861:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760702.049010:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512542720 00000100:00100000:1.0:1789760702.049034:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512542720 00000100:00100000:1.0:1789760702.049110:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 681518330640 time 681570324016 state 0x0 ref 1 00000100:00100000:1.0:1789760702.049130:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 681570584432 time 681570324016 state 0x1 ref 1 00000100:00100000:0.0:1789760702.049255:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 681570584432 time 681613550877 state 0x0 ref 1 00000100:00100000:0.0:1789760702.049273:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac3351180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512542720:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760702.049351:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21925888, mtime: 0x6aad94bd, atime: 0x0, ctime: 0x6aad94bd, blocks: 0xa807 00010000:00010000:0.0:1789760702.049361:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2d389 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760702.049385:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=681s v=(0 0) 00010000:00010000:0.0:1789760702.049391:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760702.049399:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2d389 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21925888->21929983] (req 21925888->21929983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760702.049416:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2d389 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21925888->21929983] (req 21925888->21929983) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760702.082352:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972672 is committed 00002000:00010000:0.0:1789760702.082526:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21925888 -> 21929984 00010000:00010000:0.0:1789760702.082547:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2d389 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21925888->21929983] (req 21925888->21929983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760702.082563:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2d389 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21925888->21929983] (req 21925888->21929983) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760702.082573:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2d389 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21925888->21929983] (req 21925888->21929983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760702.082581:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760702.082586:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2d389 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21925888->21929983] (req 21925888->21929983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760702.082596:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2d389 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21925888->21929983] (req 21925888->21929983) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760702.082620:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2d389 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21925888->21929983] (req 21925888->21929983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760702.082638:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2d389 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21925888->21929983] (req 21925888->21929983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760702.082795:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac3351180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512542720:12345-192.168.203.56@tcp:4:dd.0 Request processed in 33519us (33940us total) trans 4294972672 rc 0/0 00000100:00100000:0.0:1789760702.082811:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5383 00000100:00100000:0.0:1789760702.082823:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 681570584432 time 681613550877 state 0x2 ref 0 00000100:00100000:2.0:1789760702.087749:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760702.087848:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512542848 00000100:00100000:1.0:1789760702.087860:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512542848 00000100:00100000:1.0:1789760702.087935:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 681570584432 time 681613550877 state 0x0 ref 1 00000100:00100000:1.0:1789760702.087954:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 681613811293 time 681613550877 state 0x1 ref 1 00000100:00100000:1.0:1789760702.087986:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 681613811293 time 681652285686 state 0x0 ref 1 00000100:00100000:1.0:1789760702.088001:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febea4b9450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512542848:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760702.088077:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21929984, mtime: 0x6aad94bd, atime: 0x0, ctime: 0x6aad94bd, blocks: 0xa807 00010000:00010000:1.0:1789760702.088087:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2d390 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.088107:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=681s v=(0 0) 00010000:00010000:1.0:1789760702.088113:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760702.088120:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2d390 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21929984->21934079] (req 21929984->21934079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.088135:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2d390 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21929984->21934079] (req 21929984->21934079) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760702.130479:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972673 is committed 00002000:00010000:1.0:1789760702.130667:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21929984 -> 21934080 00010000:00010000:1.0:1789760702.130687:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2d390 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21929984->21934079] (req 21929984->21934079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.130834:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2d390 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21929984->21934079] (req 21929984->21934079) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.130845:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2d390 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21929984->21934079] (req 21929984->21934079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.130853:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760702.130858:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2d390 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21929984->21934079] (req 21929984->21934079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.130870:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2d390 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21929984->21934079] (req 21929984->21934079) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.130896:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2d390 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21929984->21934079] (req 21929984->21934079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.130905:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2d390 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21929984->21934079] (req 21929984->21934079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760702.131122:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febea4b9450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512542848:12345-192.168.203.56@tcp:4:dd.0 Request processed in 43116us (43425us total) trans 4294972673 rc 0/0 00000100:00100000:1.0:1789760702.131137:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5384 00000100:00100000:1.0:1789760702.131160:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 681613811293 time 681652285686 state 0x2 ref 0 00000100:00100000:1.0:1789760702.135798:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760702.135899:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512542976 00000100:00100000:2.0:1789760702.135911:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512542976 00000100:00100000:2.0:1789760702.135984:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 681613811293 time 681652285686 state 0x0 ref 1 00000100:00100000:2.0:1789760702.136414:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 681652546102 time 681652285686 state 0x1 ref 1 00000100:00100000:2.0:1789760702.136483:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 681652546102 time 681700782574 state 0x0 ref 1 00000100:00100000:2.0:1789760702.136498:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5307100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512542976:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760702.136647:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21934080, mtime: 0x6aad94bd, atime: 0x0, ctime: 0x6aad94bd, blocks: 0xa807 00010000:00010000:2.0:1789760702.136660:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2d397 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.136682:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=681s v=(0 0) 00010000:00010000:2.0:1789760702.136687:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760702.136717:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2d397 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21934080->21938175] (req 21934080->21938175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.136735:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2d397 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21934080->21938175] (req 21934080->21938175) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760702.229879:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972674 is committed 00002000:00010000:0.0:1789760702.229979:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21934080 -> 21938176 00010000:00010000:0.0:1789760702.230126:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2d397 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21934080->21938175] (req 21934080->21938175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760702.230249:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2d397 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21934080->21938175] (req 21934080->21938175) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760702.230259:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2d397 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21934080->21938175] (req 21934080->21938175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760702.230267:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760702.230272:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2d397 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21934080->21938175] (req 21934080->21938175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760702.230283:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2d397 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21934080->21938175] (req 21934080->21938175) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760702.230379:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2d397 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21934080->21938175] (req 21934080->21938175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760702.230449:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2d397 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21934080->21938175] (req 21934080->21938175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760702.230587:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5307100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512542976:12345-192.168.203.56@tcp:4:dd.0 Request processed in 94084us (94796us total) trans 4294972674 rc 0/0 00000100:00100000:0.0:1789760702.230599:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5385 00000100:00100000:0.0:1789760702.230611:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 681652546102 time 681700782574 state 0x2 ref 0 00000100:00100000:2.0:1789760702.234546:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760702.234693:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512543104 00000100:00100000:0.0:1789760702.234811:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512543104 00000100:00100000:1.0:1789760702.237785:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 681652546102 time 681700782574 state 0x0 ref 1 00000100:00100000:1.0:1789760702.237812:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 681701042990 time 681700782574 state 0x1 ref 1 00000100:00100000:1.0:1789760702.237908:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 681701042990 time 681802206891 state 0x0 ref 1 00000100:00100000:1.0:1789760702.237935:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac3352300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512543104:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760702.238017:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21938176, mtime: 0x6aad94bd, atime: 0x0, ctime: 0x6aad94bd, blocks: 0xa807 00010000:00010000:1.0:1789760702.238036:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2d39e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.238057:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=681s v=(0 0) 00010000:00010000:1.0:1789760702.238063:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760702.238072:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2d39e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21938176->21942271] (req 21938176->21942271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.238090:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2d39e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21938176->21942271] (req 21938176->21942271) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760702.284531:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972675 is committed 00002000:00010000:1.0:1789760702.284725:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21938176 -> 21942272 00010000:00010000:1.0:1789760702.284746:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2d39e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21938176->21942271] (req 21938176->21942271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.284761:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2d39e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21938176->21942271] (req 21938176->21942271) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.284770:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2d39e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21938176->21942271] (req 21938176->21942271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.284777:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760702.284783:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2d39e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21938176->21942271] (req 21938176->21942271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.284793:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2d39e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21938176->21942271] (req 21938176->21942271) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.284819:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2d39e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21938176->21942271] (req 21938176->21942271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.284834:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2d39e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21938176->21942271] (req 21938176->21942271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760702.284955:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac3352300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512543104:12345-192.168.203.56@tcp:4:dd.0 Request processed in 47027us (50415us total) trans 4294972675 rc 0/0 00000100:00100000:1.0:1789760702.284967:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5386 00000100:00100000:1.0:1789760702.284980:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 681701042990 time 681802206891 state 0x2 ref 0 00000100:00100000:1.0:1789760702.290479:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760702.290550:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512543232 00000100:00100000:3.0:1789760702.290563:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512543232 00000100:00100000:3.0:1789760702.290642:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 681701042990 time 681802206891 state 0x0 ref 1 00000100:00100000:3.0:1789760702.290660:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 681802467307 time 681802206891 state 0x1 ref 1 00000100:00100000:3.0:1789760702.290692:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 681802467307 time 681854991423 state 0x0 ref 1 00000100:00100000:3.0:1789760702.290736:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5305f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512543232:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760702.290803:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21942272, mtime: 0x6aad94bd, atime: 0x0, ctime: 0x6aad94bd, blocks: 0xa807 00010000:00010000:3.0:1789760702.290816:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2d3a5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760702.290837:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=681s v=(0 0) 00010000:00010000:3.0:1789760702.290843:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760702.290851:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2d3a5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21942272->21946367] (req 21942272->21946367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760702.290869:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2d3a5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21942272->21946367] (req 21942272->21946367) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760702.350462:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972676 is committed 00002000:00010000:1.0:1789760702.350753:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21942272 -> 21946368 00010000:00010000:1.0:1789760702.350774:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2d3a5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21942272->21946367] (req 21942272->21946367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.350790:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2d3a5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21942272->21946367] (req 21942272->21946367) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.350801:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2d3a5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21942272->21946367] (req 21942272->21946367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.350809:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760702.350815:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2d3a5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21942272->21946367] (req 21942272->21946367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.350826:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2d3a5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21942272->21946367] (req 21942272->21946367) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.350858:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2d3a5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21942272->21946367] (req 21942272->21946367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.350866:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2d3a5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21942272->21946367] (req 21942272->21946367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760702.350972:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5305f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512543232:12345-192.168.203.56@tcp:4:dd.0 Request processed in 60235us (60499us total) trans 4294972676 rc 0/0 00000100:00100000:1.0:1789760702.350983:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5387 00000100:00100000:1.0:1789760702.351068:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 681802467307 time 681854991423 state 0x2 ref 0 00000100:00100000:2.0:1789760702.358352:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760702.358496:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512543360 00000100:00100000:0.0:1789760702.358510:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512543360 00000100:00100000:0.0:1789760702.358582:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 681802467307 time 681854991423 state 0x0 ref 1 00000100:00100000:0.0:1789760702.358602:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 681855251839 time 681854991423 state 0x1 ref 1 00000100:00100000:0.0:1789760702.358647:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 681855251839 time 681922945573 state 0x0 ref 1 00000100:00100000:0.0:1789760702.358661:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac3353800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512543360:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760702.358913:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21946368, mtime: 0x6aad94bd, atime: 0x0, ctime: 0x6aad94bd, blocks: 0xa807 00010000:00010000:0.0:1789760702.358926:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2d3ac lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760702.358946:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=681s v=(0 0) 00010000:00010000:0.0:1789760702.358951:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760702.358959:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2d3ac lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21946368->21950463] (req 21946368->21950463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760702.358975:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2d3ac lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21946368->21950463] (req 21946368->21950463) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760702.434292:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972677 is committed 00002000:00010000:1.0:1789760702.435555:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21946368 -> 21950464 00010000:00010000:1.0:1789760702.435579:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2d3ac lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21946368->21950463] (req 21946368->21950463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.435593:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2d3ac lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21946368->21950463] (req 21946368->21950463) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.435603:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2d3ac lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21946368->21950463] (req 21946368->21950463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.435611:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760702.435617:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2d3ac lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21946368->21950463] (req 21946368->21950463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.435645:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2d3ac lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21946368->21950463] (req 21946368->21950463) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.435670:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2d3ac lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21946368->21950463] (req 21946368->21950463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.435678:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2d3ac lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21946368->21950463] (req 21946368->21950463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760702.435968:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac3353800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512543360:12345-192.168.203.56@tcp:4:dd.0 Request processed in 77302us (77678us total) trans 4294972677 rc 0/0 00000100:00100000:1.0:1789760702.435982:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5388 00000100:00100000:1.0:1789760702.435995:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 681855251839 time 681922945573 state 0x2 ref 0 00000100:00100000:2.0:1789760702.442290:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760702.443870:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512543488 00000100:00100000:1.0:1789760702.443890:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512543488 00000100:00100000:1.0:1789760702.443977:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 681855251839 time 681922945573 state 0x0 ref 1 00000100:00100000:1.0:1789760702.443998:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 681923205989 time 681922945573 state 0x1 ref 1 00000100:00100000:2.0:1789760702.444268:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 681923205989 time 682008563484 state 0x0 ref 1 00000100:00100000:2.0:1789760702.444287:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac3352a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512543488:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760702.444435:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21950464, mtime: 0x6aad94bd, atime: 0x0, ctime: 0x6aad94bd, blocks: 0xa807 00010000:00010000:2.0:1789760702.444447:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2d3b3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.444467:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=682s v=(0 0) 00010000:00010000:2.0:1789760702.444472:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760702.444480:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2d3b3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21950464->21954559] (req 21950464->21954559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.444499:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2d3b3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21950464->21954559] (req 21950464->21954559) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760702.487510:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972678 is committed 00002000:00010000:1.0:1789760702.487653:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21950464 -> 21954560 00010000:00010000:1.0:1789760702.487673:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2d3b3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21950464->21954559] (req 21950464->21954559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.487690:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2d3b3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21950464->21954559] (req 21950464->21954559) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.487728:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2d3b3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21950464->21954559] (req 21950464->21954559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.487737:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760702.487743:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2d3b3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21950464->21954559] (req 21950464->21954559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.488110:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2d3b3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21950464->21954559] (req 21950464->21954559) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.488137:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2d3b3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21950464->21954559] (req 21950464->21954559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.488146:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2d3b3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21950464->21954559] (req 21950464->21954559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760702.488427:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac3352a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512543488:12345-192.168.203.56@tcp:4:dd.0 Request processed in 44138us (46143us total) trans 4294972678 rc 0/0 00000100:00100000:1.0:1789760702.488441:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5389 00000100:00100000:1.0:1789760702.488453:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 681923205989 time 682008563484 state 0x2 ref 0 00000100:00100000:2.0:1789760702.495169:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760702.495262:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512543616 00000100:00100000:1.0:1789760702.495276:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512543616 00000100:00100000:1.0:1789760702.495368:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 681923205989 time 682008563484 state 0x0 ref 1 00000100:00100000:1.0:1789760702.495388:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 682008823900 time 682008563484 state 0x1 ref 1 00000100:00100000:1.0:1789760702.495422:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 682008823900 time 682059721003 state 0x0 ref 1 00000100:00100000:1.0:1789760702.495438:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac3353480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512543616:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760702.495505:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21954560, mtime: 0x6aad94bd, atime: 0x0, ctime: 0x6aad94bd, blocks: 0xa807 00010000:00010000:1.0:1789760702.495528:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2d3ba lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.495548:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=682s v=(0 0) 00010000:00010000:1.0:1789760702.495553:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760702.495561:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2d3ba lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21954560->21958655] (req 21954560->21958655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.495577:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2d3ba lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21954560->21958655] (req 21954560->21958655) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760702.554791:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21954560 -> 21958656 00002000:00010000:1.0:1789760702.554802:0:13062:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760701 -> 1789760702 00002000:00010000:1.0:1789760702.554805:0:13062:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760701 -> 1789760702 00010000:00010000:1.0:1789760702.554828:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2d3ba lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21954560->21958655] (req 21954560->21958655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.554844:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2d3ba lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21954560->21958655] (req 21954560->21958655) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.554854:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2d3ba lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21954560->21958655] (req 21954560->21958655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.554862:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760702.554868:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2d3ba lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21954560->21958655] (req 21954560->21958655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.554878:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2d3ba lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21954560->21958655] (req 21954560->21958655) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.554902:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2d3ba lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21954560->21958655] (req 21954560->21958655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.554909:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2d3ba lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21954560->21958655] (req 21954560->21958655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760702.555236:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac3353480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2360:x1876699512543616:12345-192.168.203.56@tcp:4:dd.0 Request processed in 59792us (60073us total) trans 4294972679 rc 0/0 00000100:00100000:1.0:1789760702.555252:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5390 00000100:00100000:1.0:1789760702.555264:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 682008823900 time 682059721003 state 0x2 ref 0 00000001:00080000:0.0:1789760702.555268:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972679 is committed 00000100:00100000:2.0:1789760702.561403:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760702.561461:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512543744 00000100:00100000:1.0:1789760702.561476:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512543744 00000100:00100000:1.0:1789760702.561562:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 682008823900 time 682059721003 state 0x0 ref 1 00000100:00100000:1.0:1789760702.561582:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 682059981419 time 682059721003 state 0x1 ref 1 00000100:00100000:1.0:1789760702.561619:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 682059981419 time 682125917786 state 0x0 ref 1 00000100:00100000:1.0:1789760702.561650:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac3351180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512543744:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760702.561756:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21958656, mtime: 0x6aad94be, atime: 0x0, ctime: 0x6aad94be, blocks: 0xa807 00010000:00010000:1.0:1789760702.561766:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2d3c1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.561787:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=682s v=(0 0) 00010000:00010000:1.0:1789760702.561793:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760702.561801:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2d3c1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21958656->21962751] (req 21958656->21962751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.561818:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2d3c1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21958656->21962751] (req 21958656->21962751) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760702.594965:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972680 is committed 00002000:00010000:1.0:1789760702.595099:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21958656 -> 21962752 00010000:00010000:1.0:1789760702.595122:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2d3c1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21958656->21962751] (req 21958656->21962751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.595139:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2d3c1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21958656->21962751] (req 21958656->21962751) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.595150:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2d3c1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21958656->21962751] (req 21958656->21962751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.595158:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760702.595164:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2d3c1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21958656->21962751] (req 21958656->21962751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.595175:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2d3c1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21958656->21962751] (req 21958656->21962751) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.595199:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2d3c1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21958656->21962751] (req 21958656->21962751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.595208:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2d3c1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21958656->21962751] (req 21958656->21962751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760702.595437:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac3351180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512543744:12345-192.168.203.56@tcp:4:dd.0 Request processed in 33782us (34043us total) trans 4294972680 rc 0/0 00000100:00100000:1.0:1789760702.595451:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5391 00000100:00100000:1.0:1789760702.595463:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 682059981419 time 682125917786 state 0x2 ref 0 00000100:00100000:2.0:1789760702.600650:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760702.600775:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512543872 00000100:00100000:1.0:1789760702.600790:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512543872 00000100:00100000:1.0:1789760702.600871:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 682059981419 time 682125917786 state 0x0 ref 1 00000100:00100000:1.0:1789760702.600891:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 682126178202 time 682125917786 state 0x1 ref 1 00000100:00100000:1.0:1789760702.600970:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 682126178202 time 682165269396 state 0x0 ref 1 00000100:00100000:1.0:1789760702.600985:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febff933b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512543872:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760702.601062:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21962752, mtime: 0x6aad94be, atime: 0x0, ctime: 0x6aad94be, blocks: 0xa807 00010000:00010000:1.0:1789760702.601073:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2d3c8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.601096:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=682s v=(0 0) 00010000:00010000:1.0:1789760702.601101:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760702.601109:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2d3c8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21962752->21966847] (req 21962752->21966847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.601126:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2d3c8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21962752->21966847] (req 21962752->21966847) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760702.635291:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21962752 -> 21966848 00000001:00080000:0.0:1789760702.635319:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972681 is committed 00010000:00010000:1.0:1789760702.635383:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2d3c8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21962752->21966847] (req 21962752->21966847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.635400:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2d3c8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21962752->21966847] (req 21962752->21966847) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.635411:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2d3c8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21962752->21966847] (req 21962752->21966847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.635420:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760702.635426:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2d3c8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21962752->21966847] (req 21962752->21966847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.635488:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2d3c8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21962752->21966847] (req 21962752->21966847) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.635516:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2d3c8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21962752->21966847] (req 21962752->21966847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.635525:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2d3c8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21962752->21966847] (req 21962752->21966847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760702.635691:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febff933b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512543872:12345-192.168.203.56@tcp:4:dd.0 Request processed in 34701us (35057us total) trans 4294972681 rc 0/0 00000100:00100000:1.0:1789760702.635757:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5392 00000100:00100000:1.0:1789760702.635770:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 682126178202 time 682165269396 state 0x2 ref 0 00000100:00100000:2.0:1789760702.638643:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760702.638848:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512544000 00000100:00100000:1.0:1789760702.638862:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512544000 00000100:00100000:1.0:1789760702.638936:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 682126178202 time 682165269396 state 0x0 ref 1 00000100:00100000:1.0:1789760702.638954:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 682165529812 time 682165269396 state 0x1 ref 1 00000100:00100000:1.0:1789760702.639000:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 682165529812 time 682203289133 state 0x0 ref 1 00000100:00100000:1.0:1789760702.639014:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febff930380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512544000:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760702.639091:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21966848, mtime: 0x6aad94be, atime: 0x0, ctime: 0x6aad94be, blocks: 0xa807 00010000:00010000:1.0:1789760702.639103:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2d3cf lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.639130:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=682s v=(0 0) 00010000:00010000:1.0:1789760702.639136:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760702.639145:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2d3cf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21966848->21970943] (req 21966848->21970943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.639170:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2d3cf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21966848->21970943] (req 21966848->21970943) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760702.679262:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21966848 -> 21970944 00010000:00010000:1.0:1789760702.679289:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2d3cf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21966848->21970943] (req 21966848->21970943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.679304:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2d3cf lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21966848->21970943] (req 21966848->21970943) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.679315:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2d3cf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21966848->21970943] (req 21966848->21970943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.679323:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760702.679329:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2d3cf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21966848->21970943] (req 21966848->21970943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.679339:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2d3cf lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21966848->21970943] (req 21966848->21970943) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.679428:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2d3cf lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21966848->21970943] (req 21966848->21970943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.679439:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2d3cf lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21966848->21970943] (req 21966848->21970943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760702.679476:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972682 is committed 00000100:00100000:1.0:1789760702.679607:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febff930380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512544000:12345-192.168.203.56@tcp:4:dd.0 Request processed in 40586us (40989us total) trans 4294972682 rc 0/0 00000100:00100000:1.0:1789760702.679621:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5393 00000100:00100000:1.0:1789760702.679642:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 682165529812 time 682203289133 state 0x2 ref 0 00000100:00100000:2.0:1789760702.683406:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760702.683665:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512544128 00000100:00100000:0.0:1789760702.683678:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512544128 00000100:00100000:0.0:1789760702.683803:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 682165529812 time 682203289133 state 0x0 ref 1 00000100:00100000:0.0:1789760702.683832:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 682203549549 time 682203289133 state 0x1 ref 1 00000100:00100000:0.0:1789760702.683865:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 682203549549 time 682248163941 state 0x0 ref 1 00000100:00100000:0.0:1789760702.683881:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febff932a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512544128:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760702.683946:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21970944, mtime: 0x6aad94be, atime: 0x0, ctime: 0x6aad94be, blocks: 0xa807 00010000:00010000:0.0:1789760702.683957:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2d3d6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760702.683977:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=682s v=(0 0) 00010000:00010000:0.0:1789760702.683983:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760702.683990:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2d3d6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21970944->21975039] (req 21970944->21975039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760702.684008:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2d3d6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21970944->21975039] (req 21970944->21975039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760702.726738:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972683 is committed 00002000:00010000:0.0:1789760702.726983:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21970944 -> 21975040 00010000:00010000:0.0:1789760702.727004:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2d3d6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21970944->21975039] (req 21970944->21975039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760702.727076:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2d3d6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21970944->21975039] (req 21970944->21975039) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760702.727084:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2d3d6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21970944->21975039] (req 21970944->21975039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760702.727089:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760702.727094:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2d3d6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21970944->21975039] (req 21970944->21975039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760702.727101:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2d3d6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21970944->21975039] (req 21970944->21975039) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760702.727121:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2d3d6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21970944->21975039] (req 21970944->21975039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760702.727127:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2d3d6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21970944->21975039] (req 21970944->21975039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760702.727255:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febff932a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512544128:12345-192.168.203.56@tcp:4:dd.0 Request processed in 43370us (43863us total) trans 4294972683 rc 0/0 00000100:00100000:0.0:1789760702.727269:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5394 00000100:00100000:0.0:1789760702.727281:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 682203549549 time 682248163941 state 0x2 ref 0 00000100:00100000:3.0:1789760702.730999:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760702.731151:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512544256 00000100:00100000:1.0:1789760702.731165:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512544256 00000100:00100000:1.0:1789760702.731241:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 682203549549 time 682248163941 state 0x0 ref 1 00000100:00100000:1.0:1789760702.731260:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 682248424357 time 682248163941 state 0x1 ref 1 00000100:00100000:1.0:1789760702.731396:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 682248424357 time 682295693868 state 0x0 ref 1 00000100:00100000:1.0:1789760702.731412:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2747480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512544256:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760702.731486:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21975040, mtime: 0x6aad94be, atime: 0x0, ctime: 0x6aad94be, blocks: 0xa807 00010000:00010000:1.0:1789760702.731498:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2d3dd lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.731519:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=682s v=(0 0) 00010000:00010000:1.0:1789760702.731525:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760702.731533:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2d3dd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21975040->21979135] (req 21975040->21979135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.731551:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2d3dd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21975040->21979135] (req 21975040->21979135) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760702.784031:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972684 is committed 00002000:00010000:1.0:1789760702.784315:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21975040 -> 21979136 00010000:00010000:1.0:1789760702.784336:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2d3dd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21975040->21979135] (req 21975040->21979135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.784364:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2d3dd lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21975040->21979135] (req 21975040->21979135) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.784377:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2d3dd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21975040->21979135] (req 21975040->21979135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.784385:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760702.784391:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2d3dd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21975040->21979135] (req 21975040->21979135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.784401:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2d3dd lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21975040->21979135] (req 21975040->21979135) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.784428:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2d3dd lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21975040->21979135] (req 21975040->21979135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.784440:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2d3dd lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21975040->21979135] (req 21975040->21979135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760702.784759:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2747480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512544256:12345-192.168.203.56@tcp:4:dd.0 Request processed in 53342us (53767us total) trans 4294972684 rc 0/0 00000100:00100000:1.0:1789760702.784775:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5395 00000100:00100000:1.0:1789760702.784788:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 682248424357 time 682295693868 state 0x2 ref 0 00000100:00100000:2.0:1789760702.793690:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760702.793785:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512544384 00000100:00100000:1.0:1789760702.793801:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512544384 00000100:00100000:1.0:1789760702.793882:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 682248424357 time 682295693868 state 0x0 ref 1 00000100:00100000:1.0:1789760702.793902:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 682295954284 time 682295693868 state 0x1 ref 1 00000100:00100000:1.0:1789760702.793942:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 682295954284 time 682358240365 state 0x0 ref 1 00000100:00100000:1.0:1789760702.793956:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febff930e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512544384:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760702.794252:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21979136, mtime: 0x6aad94be, atime: 0x0, ctime: 0x6aad94be, blocks: 0xa807 00010000:00010000:1.0:1789760702.794264:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2d3e4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.794285:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=682s v=(0 0) 00010000:00010000:1.0:1789760702.794290:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760702.794298:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2d3e4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21979136->21983231] (req 21979136->21983231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.794553:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2d3e4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21979136->21983231] (req 21979136->21983231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.841469:0:5835:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:0.0:1789760702.868010:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972685 is committed 00002000:00010000:1.0:1789760702.868181:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21979136 -> 21983232 00010000:00010000:1.0:1789760702.868197:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2d3e4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21979136->21983231] (req 21979136->21983231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.868212:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2d3e4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21979136->21983231] (req 21979136->21983231) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.868221:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2d3e4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21979136->21983231] (req 21979136->21983231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.868228:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760702.868233:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2d3e4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21979136->21983231] (req 21979136->21983231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.868242:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2d3e4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21979136->21983231] (req 21979136->21983231) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.868262:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2d3e4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21979136->21983231] (req 21979136->21983231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.868270:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2d3e4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21979136->21983231] (req 21979136->21983231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760702.868429:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febff930e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512544384:12345-192.168.203.56@tcp:4:dd.0 Request processed in 74471us (74748us total) trans 4294972685 rc 0/0 00000100:00100000:1.0:1789760702.868440:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5396 00000100:00100000:1.0:1789760702.868450:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 682295954284 time 682358240365 state 0x2 ref 0 00000100:00100000:2.0:1789760702.872068:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760702.872149:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512544512 00000100:00100000:0.0:1789760702.872159:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512544512 00000100:00100000:0.0:1789760702.872212:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 682295954284 time 682358240365 state 0x0 ref 1 00000100:00100000:0.0:1789760702.872228:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 682358500781 time 682358240365 state 0x1 ref 1 00000100:00100000:0.0:1789760702.872257:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 682358500781 time 682436556828 state 0x0 ref 1 00000100:00100000:0.0:1789760702.872269:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febfc975180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512544512:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760702.872330:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21983232, mtime: 0x6aad94be, atime: 0x0, ctime: 0x6aad94be, blocks: 0xa807 00010000:00010000:0.0:1789760702.872339:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2d3eb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760702.872354:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=682s v=(0 0) 00010000:00010000:0.0:1789760702.872358:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760702.872365:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2d3eb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21983232->21987327] (req 21983232->21987327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760702.872379:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2d3eb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21983232->21987327] (req 21983232->21987327) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760702.924615:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21983232 -> 21987328 00010000:00010000:2.0:1789760702.924648:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2d3eb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21983232->21987327] (req 21983232->21987327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.924660:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2d3eb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21983232->21987327] (req 21983232->21987327) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.924670:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2d3eb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21983232->21987327] (req 21983232->21987327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.924677:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760702.924681:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2d3eb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21983232->21987327] (req 21983232->21987327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.924690:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2d3eb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21983232->21987327] (req 21983232->21987327) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.924737:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2d3eb lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21983232->21987327] (req 21983232->21987327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.924744:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2d3eb lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21983232->21987327] (req 21983232->21987327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760702.924957:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febfc975180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2359:x1876699512544512:12345-192.168.203.56@tcp:4:dd.0 Request processed in 52683us (52893us total) trans 4294972686 rc 0/0 00000100:00100000:2.0:1789760702.924977:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5397 00000100:00100000:2.0:1789760702.924988:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 682358500781 time 682436556828 state 0x2 ref 0 00000001:00080000:0.0:1789760702.925476:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972686 is committed 00000100:00100000:2.0:1789760702.929024:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760702.929129:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512544640 00000100:00100000:1.0:1789760702.929140:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512544640 00000100:00100000:1.0:1789760702.929190:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 682358500781 time 682436556828 state 0x0 ref 1 00000100:00100000:1.0:1789760702.929204:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 682436817244 time 682436556828 state 0x1 ref 1 00000100:00100000:1.0:1789760702.929232:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 682436817244 time 682493532170 state 0x0 ref 1 00000100:00100000:1.0:1789760702.929244:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac3302680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512544640:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760702.929301:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21987328, mtime: 0x6aad94be, atime: 0x0, ctime: 0x6aad94be, blocks: 0xa807 00010000:00010000:1.0:1789760702.929311:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2d3f2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.929327:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=682s v=(0 0) 00010000:00010000:1.0:1789760702.929331:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760702.929338:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2d3f2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21987328->21991423] (req 21987328->21991423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.929351:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2d3f2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21987328->21991423] (req 21987328->21991423) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760703.024735:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21987328 -> 21991424 00010000:00010000:1.0:1789760703.024754:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2d3f2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21987328->21991423] (req 21987328->21991423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760703.024768:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2d3f2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21987328->21991423] (req 21987328->21991423) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760703.024782:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2d3f2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21987328->21991423] (req 21987328->21991423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760703.024790:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760703.024795:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2d3f2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21987328->21991423] (req 21987328->21991423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760703.024804:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2d3f2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21987328->21991423] (req 21987328->21991423) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760703.024825:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2d3f2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21987328->21991423] (req 21987328->21991423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760703.024833:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2d3f2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21987328->21991423] (req 21987328->21991423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760703.024925:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac3302680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2360:x1876699512544640:12345-192.168.203.56@tcp:4:dd.0 Request processed in 95677us (95904us total) trans 4294972687 rc 0/0 00000100:00100000:1.0:1789760703.024936:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5398 00000100:00100000:1.0:1789760703.024945:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 682436817244 time 682493532170 state 0x2 ref 0 00000001:00080000:2.0:1789760703.027137:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972687 is committed 00000100:00100000:1.0:1789760703.028346:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760703.028519:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512544768 00000100:00100000:3.0:1789760703.028529:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512544768 00000100:00100000:3.0:1789760703.028598:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 682436817244 time 682493532170 state 0x0 ref 1 00000100:00100000:3.0:1789760703.028617:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 682493792586 time 682493532170 state 0x1 ref 1 00000100:00100000:3.0:1789760703.028651:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 682493792586 time 682592950878 state 0x0 ref 1 00000100:00100000:3.0:1789760703.028666:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5307480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512544768:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760703.028740:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21991424, mtime: 0x6aad94be, atime: 0x0, ctime: 0x6aad94be, blocks: 0xa807 00010000:00010000:3.0:1789760703.028751:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2d3f9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760703.028767:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=682s v=(0 0) 00010000:00010000:3.0:1789760703.028771:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760703.028778:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2d3f9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21991424->21995519] (req 21991424->21995519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760703.028793:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2d3f9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21991424->21995519] (req 21991424->21995519) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:1.0:1789760703.030596:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:2.0:1789760703.113509:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972688 is committed 00002000:00010000:1.0:1789760703.113612:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21991424 -> 21995520 00010000:00010000:1.0:1789760703.113637:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2d3f9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21991424->21995519] (req 21991424->21995519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760703.113650:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2d3f9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21991424->21995519] (req 21991424->21995519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760703.113660:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2d3f9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21991424->21995519] (req 21991424->21995519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760703.113668:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760703.113673:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2d3f9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21991424->21995519] (req 21991424->21995519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760703.113682:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2d3f9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21991424->21995519] (req 21991424->21995519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760703.113724:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2d3f9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21991424->21995519] (req 21991424->21995519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760703.113732:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2d3f9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21991424->21995519] (req 21991424->21995519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760703.113825:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5307480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512544768:12345-192.168.203.56@tcp:4:dd.0 Request processed in 85157us (85480us total) trans 4294972688 rc 0/0 00000100:00100000:1.0:1789760703.113835:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5399 00000100:00100000:1.0:1789760703.113845:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 682493792586 time 682592950878 state 0x2 ref 0 00000100:00100000:3.0:1789760703.118036:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760703.118152:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512544896 00000100:00100000:2.0:1789760703.118161:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512544896 00000100:00100000:2.0:1789760703.118215:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 682493792586 time 682592950878 state 0x0 ref 1 00000100:00100000:2.0:1789760703.118230:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 682593211294 time 682592950878 state 0x1 ref 1 00000100:00100000:2.0:1789760703.118261:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 682593211294 time 682682561109 state 0x0 ref 1 00000100:00100000:2.0:1789760703.118499:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2744700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512544896:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760703.119415:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21995520, mtime: 0x6aad94be, atime: 0x0, ctime: 0x6aad94be, blocks: 0xa807 00010000:00010000:2.0:1789760703.119428:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2d400 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.119444:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=682s v=(0 0) 00010000:00010000:2.0:1789760703.119449:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760703.119456:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2d400 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21995520->21999615] (req 21995520->21999615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.119469:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2d400 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21995520->21999615] (req 21995520->21999615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760703.201145:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21995520 -> 21999616 00010000:00010000:2.0:1789760703.201165:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2d400 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21995520->21999615] (req 21995520->21999615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.201176:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2d400 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21995520->21999615] (req 21995520->21999615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.201184:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2d400 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21995520->21999615] (req 21995520->21999615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.201191:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760703.201196:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2d400 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21995520->21999615] (req 21995520->21999615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.201205:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2d400 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21995520->21999615] (req 21995520->21999615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.201223:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2d400 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21995520->21999615] (req 21995520->21999615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.201231:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2d400 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21995520->21999615] (req 21995520->21999615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760703.202498:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2744700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2360:x1876699512544896:12345-192.168.203.56@tcp:4:dd.0 Request processed in 84214us (84460us total) trans 4294972689 rc 0/0 00000100:00100000:2.0:1789760703.202512:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5400 00000100:00100000:2.0:1789760703.202523:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 682593211294 time 682682561109 state 0x2 ref 0 00000001:00080000:2.0:1789760703.202582:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972689 is committed 00000100:00100000:2.0:1789760703.207386:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760703.207837:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512545024 00000100:00100000:1.0:1789760703.207847:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512545024 00000100:00100000:1.0:1789760703.207902:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 682593211294 time 682682561109 state 0x0 ref 1 00000100:00100000:1.0:1789760703.207917:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 682682821525 time 682682561109 state 0x1 ref 1 00000100:00100000:1.0:1789760703.207947:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 682682821525 time 682772246469 state 0x0 ref 1 00000100:00100000:1.0:1789760703.207957:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac3300e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512545024:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760703.208007:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21999616, mtime: 0x6aad94be, atime: 0x0, ctime: 0x6aad94be, blocks: 0xa807 00010000:00010000:1.0:1789760703.208024:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2d407 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760703.208038:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=682s v=(0 0) 00010000:00010000:1.0:1789760703.208043:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760703.208049:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2d407 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21999616->22003711] (req 21999616->22003711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760703.208061:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2d407 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21999616->22003711] (req 21999616->22003711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760703.325917:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972690 is committed 00002000:00010000:3.0:1789760703.326340:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21999616 -> 22003712 00010000:00010000:3.0:1789760703.326356:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2d407 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21999616->22003711] (req 21999616->22003711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760703.326368:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2d407 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21999616->22003711] (req 21999616->22003711) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760703.326377:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2d407 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21999616->22003711] (req 21999616->22003711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760703.326385:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760703.326390:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2d407 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21999616->22003711] (req 21999616->22003711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760703.326398:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2d407 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21999616->22003711] (req 21999616->22003711) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760703.326418:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2d407 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21999616->22003711] (req 21999616->22003711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760703.326425:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2d407 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21999616->22003711] (req 21999616->22003711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760703.326524:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac3300e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512545024:12345-192.168.203.56@tcp:4:dd.0 Request processed in 118563us (119141us total) trans 4294972690 rc 0/0 00000100:00100000:3.0:1789760703.326535:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5401 00000100:00100000:3.0:1789760703.326545:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 682682821525 time 682772246469 state 0x2 ref 0 00000100:00100000:3.0:1789760703.331638:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760703.331778:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512545152 00000100:00100000:1.0:1789760703.331795:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512545152 00000100:00100000:1.0:1789760703.331883:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 682682821525 time 682772246469 state 0x0 ref 1 00000100:00100000:1.0:1789760703.331903:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 682772506885 time 682772246469 state 0x1 ref 1 00000100:00100000:1.0:1789760703.331951:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 682772506885 time 682896249902 state 0x0 ref 1 00000100:00100000:1.0:1789760703.331979:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2746a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512545152:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760703.332058:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22003712, mtime: 0x6aad94be, atime: 0x0, ctime: 0x6aad94be, blocks: 0xa807 00010000:00010000:1.0:1789760703.332071:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2d40e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760703.332092:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=682s v=(0 0) 00010000:00010000:1.0:1789760703.332097:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760703.332105:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2d40e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22003712->22007807] (req 22003712->22007807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760703.332122:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2d40e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22003712->22007807] (req 22003712->22007807) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760703.401498:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22003712 -> 22007808 00010000:00010000:2.0:1789760703.401524:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2d40e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22003712->22007807] (req 22003712->22007807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.401548:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2d40e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22003712->22007807] (req 22003712->22007807) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.401559:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2d40e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22003712->22007807] (req 22003712->22007807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.401567:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760703.401574:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2d40e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22003712->22007807] (req 22003712->22007807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.401592:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2d40e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22003712->22007807] (req 22003712->22007807) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.401616:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2d40e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22003712->22007807] (req 22003712->22007807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.401644:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2d40e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22003712->22007807] (req 22003712->22007807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760703.401813:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2746a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2359:x1876699512545152:12345-192.168.203.56@tcp:4:dd.0 Request processed in 69829us (70247us total) trans 4294972691 rc 0/0 00000100:00100000:2.0:1789760703.401827:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5402 00000100:00100000:2.0:1789760703.401839:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 682772506885 time 682896249902 state 0x2 ref 0 00000001:00080000:0.0:1789760703.403359:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972691 is committed 00000100:00100000:2.0:1789760703.406747:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760703.406879:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512545280 00000100:00100000:1.0:1789760703.406893:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512545280 00000100:00100000:1.0:1789760703.406965:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 682772506885 time 682896249902 state 0x0 ref 1 00000100:00100000:1.0:1789760703.406984:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 682896510318 time 682896249902 state 0x1 ref 1 00000100:00100000:1.0:1789760703.407014:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 682896510318 time 682971313952 state 0x0 ref 1 00000100:00100000:1.0:1789760703.407027:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac3302d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512545280:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760703.407100:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22007808, mtime: 0x6aad94be, atime: 0x0, ctime: 0x6aad94be, blocks: 0xa807 00010000:00010000:1.0:1789760703.407111:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2d415 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760703.407139:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=682s v=(0 0) 00010000:00010000:1.0:1789760703.407143:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760703.407150:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2d415 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22007808->22011903] (req 22007808->22011903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760703.407165:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2d415 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22007808->22011903] (req 22007808->22011903) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760703.454633:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972692 is committed 00002000:00010000:2.0:1789760703.454804:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22007808 -> 22011904 00010000:00010000:2.0:1789760703.454826:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2d415 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22007808->22011903] (req 22007808->22011903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.454853:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2d415 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22007808->22011903] (req 22007808->22011903) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.454864:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2d415 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22007808->22011903] (req 22007808->22011903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.454877:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760703.454883:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2d415 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22007808->22011903] (req 22007808->22011903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.454893:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2d415 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22007808->22011903] (req 22007808->22011903) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.454921:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2d415 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22007808->22011903] (req 22007808->22011903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.454929:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2d415 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22007808->22011903] (req 22007808->22011903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760703.455314:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac3302d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512545280:12345-192.168.203.56@tcp:4:dd.0 Request processed in 48281us (48691us total) trans 4294972692 rc 0/0 00000100:00100000:2.0:1789760703.455326:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5403 00000100:00100000:2.0:1789760703.455338:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 682896510318 time 682971313952 state 0x2 ref 0 00000100:00100000:2.0:1789760703.460772:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760703.460988:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512545408 00000100:00100000:3.0:1789760703.461003:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512545408 00000100:00100000:3.0:1789760703.461083:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 682896510318 time 682971313952 state 0x0 ref 1 00000100:00100000:3.0:1789760703.461103:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 682971574368 time 682971313952 state 0x1 ref 1 00000100:00100000:3.0:1789760703.461144:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 682971574368 time 683025443504 state 0x0 ref 1 00000100:00100000:3.0:1789760703.461161:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac3303100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512545408:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760703.461227:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22011904, mtime: 0x6aad94be, atime: 0x0, ctime: 0x6aad94be, blocks: 0xa807 00010000:00010000:3.0:1789760703.461239:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2d41c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760703.461260:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=683s v=(0 0) 00010000:00010000:3.0:1789760703.461266:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760703.461273:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2d41c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22011904->22015999] (req 22011904->22015999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760703.461291:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2d41c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22011904->22015999] (req 22011904->22015999) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760703.522966:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972693 is committed 00002000:00010000:3.0:1789760703.523187:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22011904 -> 22016000 00010000:00010000:3.0:1789760703.523209:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2d41c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22011904->22015999] (req 22011904->22015999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760703.523225:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2d41c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22011904->22015999] (req 22011904->22015999) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760703.523236:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2d41c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22011904->22015999] (req 22011904->22015999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760703.523245:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760703.523251:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2d41c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22011904->22015999] (req 22011904->22015999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760703.523261:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2d41c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22011904->22015999] (req 22011904->22015999) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760703.523285:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2d41c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22011904->22015999] (req 22011904->22015999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760703.523307:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2d41c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22011904->22015999] (req 22011904->22015999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760703.523434:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac3303100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512545408:12345-192.168.203.56@tcp:4:dd.0 Request processed in 62270us (62672us total) trans 4294972693 rc 0/0 00000100:00100000:3.0:1789760703.523455:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5404 00000100:00100000:3.0:1789760703.523467:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 682971574368 time 683025443504 state 0x2 ref 0 00000100:00100000:2.0:1789760703.533062:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760703.533215:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512545536 00000100:00100000:1.0:1789760703.533230:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512545536 00000100:00100000:1.0:1789760703.533319:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 682971574368 time 683025443504 state 0x0 ref 1 00000100:00100000:1.0:1789760703.533340:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 683025703920 time 683025443504 state 0x1 ref 1 00000100:00100000:1.0:1789760703.533378:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 683025703920 time 683097676505 state 0x0 ref 1 00000100:00100000:1.0:1789760703.533392:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac3301c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512545536:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760703.533470:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22016000, mtime: 0x6aad94be, atime: 0x0, ctime: 0x6aad94be, blocks: 0xa807 00010000:00010000:1.0:1789760703.533482:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2d423 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760703.533503:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=683s v=(0 0) 00010000:00010000:1.0:1789760703.533508:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760703.533517:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2d423 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22016000->22020095] (req 22016000->22020095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760703.533535:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2d423 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22016000->22020095] (req 22016000->22020095) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760703.606685:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972694 is committed 00002000:00010000:0.0:1789760703.607031:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22016000 -> 22020096 00002000:00010000:0.0:1789760703.607036:0:13062:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760702 -> 1789760703 00002000:00010000:0.0:1789760703.607039:0:13062:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760702 -> 1789760703 00010000:00010000:0.0:1789760703.607058:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2d423 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22016000->22020095] (req 22016000->22020095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760703.607073:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2d423 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22016000->22020095] (req 22016000->22020095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760703.607084:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2d423 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22016000->22020095] (req 22016000->22020095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760703.607092:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760703.607097:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2d423 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22016000->22020095] (req 22016000->22020095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760703.607107:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2d423 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22016000->22020095] (req 22016000->22020095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760703.607133:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2d423 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22016000->22020095] (req 22016000->22020095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760703.607142:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2d423 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22016000->22020095] (req 22016000->22020095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760703.607385:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac3301c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512545536:12345-192.168.203.56@tcp:4:dd.0 Request processed in 73989us (74334us total) trans 4294972694 rc 0/0 00000100:00100000:0.0:1789760703.607398:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5405 00000100:00100000:0.0:1789760703.607410:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 683025703920 time 683097676505 state 0x2 ref 0 00000100:00100000:0.0:1789760703.610887:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760703.611186:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512545664 00000100:00100000:2.0:1789760703.611199:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512545664 00000100:00100000:2.0:1789760703.611272:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 683025703920 time 683097676505 state 0x0 ref 1 00000100:00100000:2.0:1789760703.611364:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 683097936921 time 683097676505 state 0x1 ref 1 00000100:00100000:2.0:1789760703.611517:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 683097936921 time 683175814928 state 0x0 ref 1 00000100:00100000:2.0:1789760703.611532:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febfb374380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512545664:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760703.611604:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22020096, mtime: 0x6aad94bf, atime: 0x0, ctime: 0x6aad94bf, blocks: 0xa807 00010000:00010000:2.0:1789760703.611616:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2d42a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.611648:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=683s v=(0 0) 00010000:00010000:2.0:1789760703.611654:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760703.611663:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2d42a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22020096->22024191] (req 22020096->22024191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.611680:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2d42a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22020096->22024191] (req 22020096->22024191) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760703.637494:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972695 is committed 00002000:00010000:0.0:1789760703.637589:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22020096 -> 22024192 00002000:00010000:0.0:1789760703.637595:0:6710:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb blocks from disk: 43015 -> 45063 00010000:00010000:0.0:1789760703.637612:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2d42a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22020096->22024191] (req 22020096->22024191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760703.637640:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2d42a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22020096->22024191] (req 22020096->22024191) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760703.637651:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2d42a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22020096->22024191] (req 22020096->22024191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760703.637658:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760703.637664:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2d42a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22020096->22024191] (req 22020096->22024191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760703.637675:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2d42a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22020096->22024191] (req 22020096->22024191) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760703.637739:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2d42a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22020096->22024191] (req 22020096->22024191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760703.637748:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2d42a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22020096->22024191] (req 22020096->22024191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760703.637872:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febfb374380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512545664:12345-192.168.203.56@tcp:4:dd.0 Request processed in 26336us (26991us total) trans 4294972695 rc 0/0 00000100:00100000:0.0:1789760703.637885:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5406 00000100:00100000:0.0:1789760703.637899:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 683097936921 time 683175814928 state 0x2 ref 0 00000100:00100000:2.0:1789760703.656085:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760703.656318:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512545792 00000100:00100000:3.0:1789760703.656330:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512545792 00000100:00100000:3.0:1789760703.656403:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 683097936921 time 683175814928 state 0x0 ref 1 00000100:00100000:3.0:1789760703.656420:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 683176075344 time 683175814928 state 0x1 ref 1 00000100:00100000:3.0:1789760703.656457:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 683176075344 time 683220756404 state 0x0 ref 1 00000100:00100000:3.0:1789760703.656471:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac38b7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512545792:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760703.656540:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22024192, mtime: 0x6aad94bf, atime: 0x0, ctime: 0x6aad94bf, blocks: 0xb007 00010000:00010000:3.0:1789760703.656552:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2d431 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760703.656572:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=683s v=(0 0) 00010000:00010000:3.0:1789760703.656577:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760703.656585:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2d431 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22024192->22028287] (req 22024192->22028287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760703.656601:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2d431 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22024192->22028287] (req 22024192->22028287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760703.725403:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22024192 -> 22028288 00010000:00010000:1.0:1789760703.725430:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2d431 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22024192->22028287] (req 22024192->22028287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760703.725445:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2d431 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22024192->22028287] (req 22024192->22028287) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760703.725456:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2d431 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22024192->22028287] (req 22024192->22028287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760703.725463:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760703.725469:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2d431 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22024192->22028287] (req 22024192->22028287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760703.725479:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2d431 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22024192->22028287] (req 22024192->22028287) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760703.725504:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2d431 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22024192->22028287] (req 22024192->22028287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760703.725512:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2d431 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22024192->22028287] (req 22024192->22028287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760703.726523:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972696 is committed 00000100:00100000:1.0:1789760703.727521:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac38b7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512545792:12345-192.168.203.56@tcp:4:dd.0 Request processed in 71039us (71438us total) trans 4294972696 rc 0/0 00000100:00100000:1.0:1789760703.727540:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5407 00000100:00100000:1.0:1789760703.727554:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 683176075344 time 683220756404 state 0x2 ref 0 00000100:00100000:2.0:1789760703.732969:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760703.733216:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512545920 00000100:00100000:0.0:1789760703.733231:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512545920 00000100:00100000:0.0:1789760703.733787:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 683176075344 time 683220756404 state 0x0 ref 1 00000100:00100000:0.0:1789760703.733807:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 683221016820 time 683220756404 state 0x1 ref 1 00000100:00100000:0.0:1789760703.733904:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 683221016820 time 683298202874 state 0x0 ref 1 00000100:00100000:0.0:1789760703.733924:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac38b4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512545920:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760703.733994:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22028288, mtime: 0x6aad94bf, atime: 0x0, ctime: 0x6aad94bf, blocks: 0xb007 00010000:00010000:0.0:1789760703.734018:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2d438 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760703.734041:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=683s v=(0 0) 00010000:00010000:0.0:1789760703.734047:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760703.734055:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2d438 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22028288->22032383] (req 22028288->22032383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760703.734083:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2d438 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22028288->22032383] (req 22028288->22032383) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760703.753060:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760703.753120:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760703.753746:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512546176 00000100:00100000:3.0:1789760703.753761:0:5847:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512546176 00000100:00100000:3.0:1789760703.753846:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 434 00000100:00100000:3.0:1789760703.753862:0:5847:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febfb375500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+8:2360:x1876699512546176:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:3.0:1789760703.753979:0:5847:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febfb375500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+8:2360:x1876699512546176:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 123us (934us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760703.753987:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 434 00000100:00100000:3.0:1789760703.754026:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512546048 00000100:00100000:3.0:1789760703.754030:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512546048 00000100:00100000:3.0:1789760703.754054:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 382 00000100:00100000:3.0:1789760703.754061:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febe00d6c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:e6cf7267-9b19-462a-a263-14115df15dab+13:2360:x1876699512546048:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:3.0:1789760703.754092:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febe00d6c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:e6cf7267-9b19-462a-a263-14115df15dab+13:2360:x1876699512546048:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 31us (972us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760703.754103:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 382 00000100:00080000:3.0:1789760703.754136:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789760703.754493:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760703.755804:0:5848:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512546304 00000100:00100000:2.0:1789760703.755822:0:5848:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512546304 00000100:00100000:2.0:1789760703.755905:0:5848:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 435 00000100:00100000:2.0:1789760703.755915:0:5848:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febfb377100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:d2c40150-e97b-40da-952b-cb9f18bf7e96+5:2360:x1876699512546304:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:2.0:1789760703.757917:0:5848:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febfb377100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:d2c40150-e97b-40da-952b-cb9f18bf7e96+5:2360:x1876699512546304:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 1760us (3184us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760703.757936:0:5848:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 435 00000100:00080000:3.0:1789760703.760470:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000001:00080000:1.0:1789760703.799179:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972697 is committed 00002000:00010000:0.0:1789760703.799308:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22028288 -> 22032384 00010000:00010000:0.0:1789760703.799328:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2d438 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22028288->22032383] (req 22028288->22032383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760703.799343:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2d438 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22028288->22032383] (req 22028288->22032383) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760703.799353:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2d438 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22028288->22032383] (req 22028288->22032383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760703.799361:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760703.799367:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2d438 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22028288->22032383] (req 22028288->22032383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760703.799378:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2d438 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22028288->22032383] (req 22028288->22032383) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760703.799402:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2d438 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22028288->22032383] (req 22028288->22032383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760703.799410:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2d438 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22028288->22032383] (req 22028288->22032383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760703.799565:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac38b4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512545920:12345-192.168.203.56@tcp:4:dd.0 Request processed in 65641us (66606us total) trans 4294972697 rc 0/0 00000100:00100000:0.0:1789760703.799578:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5408 00000100:00100000:0.0:1789760703.799591:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 683221016820 time 683298202874 state 0x2 ref 0 00000100:00100000:1.0:1789760703.806024:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760703.806109:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512546432 00000100:00100000:0.0:1789760703.806123:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512546432 00000100:00100000:0.0:1789760703.806192:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 683221016820 time 683298202874 state 0x0 ref 1 00000100:00100000:0.0:1789760703.806209:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 683298463290 time 683298202874 state 0x1 ref 1 00000100:00100000:0.0:1789760703.806244:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 683298463290 time 683370543781 state 0x0 ref 1 00000100:00100000:0.0:1789760703.806259:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5304700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512546432:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760703.806335:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22032384, mtime: 0x6aad94bf, atime: 0x0, ctime: 0x6aad94bf, blocks: 0xb007 00010000:00010000:0.0:1789760703.806346:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2d43f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760703.806363:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=683s v=(0 0) 00010000:00010000:0.0:1789760703.806369:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760703.806376:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2d43f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22032384->22036479] (req 22032384->22036479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760703.806392:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2d43f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22032384->22036479] (req 22032384->22036479) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760703.841170:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972698 is committed 00002000:00010000:0.0:1789760703.841307:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22032384 -> 22036480 00010000:00010000:0.0:1789760703.841327:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2d43f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22032384->22036479] (req 22032384->22036479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760703.841343:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2d43f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22032384->22036479] (req 22032384->22036479) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760703.841354:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2d43f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22032384->22036479] (req 22032384->22036479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760703.841363:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760703.841369:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2d43f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22032384->22036479] (req 22032384->22036479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760703.841379:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2d43f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22032384->22036479] (req 22032384->22036479) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760703.841403:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2d43f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22032384->22036479] (req 22032384->22036479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760703.841411:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2d43f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22032384->22036479] (req 22032384->22036479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760703.841529:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5304700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512546432:12345-192.168.203.56@tcp:4:dd.0 Request processed in 35266us (35515us total) trans 4294972698 rc 0/0 00000100:00100000:0.0:1789760703.841541:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5409 00000100:00100000:0.0:1789760703.841553:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 683298463290 time 683370543781 state 0x2 ref 0 00000100:00100000:2.0:1789760703.846223:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760703.846323:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512546560 00000100:00100000:1.0:1789760703.846335:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512546560 00000100:00100000:1.0:1789760703.846411:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 683298463290 time 683370543781 state 0x0 ref 1 00000100:00100000:1.0:1789760703.846430:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 683370804197 time 683370543781 state 0x1 ref 1 00000100:00100000:1.0:1789760703.846462:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 683370804197 time 683410761181 state 0x0 ref 1 00000100:00100000:1.0:1789760703.846476:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac38b7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512546560:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760703.846541:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22036480, mtime: 0x6aad94bf, atime: 0x0, ctime: 0x6aad94bf, blocks: 0xb007 00010000:00010000:1.0:1789760703.846551:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2d446 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760703.846571:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=683s v=(0 0) 00010000:00010000:1.0:1789760703.846577:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760703.846585:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2d446 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22036480->22040575] (req 22036480->22040575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760703.846609:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2d446 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22036480->22040575] (req 22036480->22040575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760703.860212:0:7647:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:1.0:1789760703.882150:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972699 is committed 00002000:00010000:2.0:1789760703.882361:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22036480 -> 22040576 00010000:00010000:2.0:1789760703.882382:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2d446 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22036480->22040575] (req 22036480->22040575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.882398:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2d446 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22036480->22040575] (req 22036480->22040575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.882409:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2d446 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22036480->22040575] (req 22036480->22040575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.882418:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760703.882424:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2d446 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22036480->22040575] (req 22036480->22040575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.882436:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2d446 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22036480->22040575] (req 22036480->22040575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.882461:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2d446 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22036480->22040575] (req 22036480->22040575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.882472:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2d446 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22036480->22040575] (req 22036480->22040575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760703.882597:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac38b7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512546560:12345-192.168.203.56@tcp:4:dd.0 Request processed in 36116us (36382us total) trans 4294972699 rc 0/0 00000100:00100000:2.0:1789760703.882610:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5410 00000100:00100000:2.0:1789760703.882632:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 683370804197 time 683410761181 state 0x2 ref 0 00000100:00100000:1.0:1789760703.887181:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760703.887357:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512546688 00000100:00100000:0.0:1789760703.887372:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512546688 00000100:00100000:0.0:1789760703.887453:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 683370804197 time 683410761181 state 0x0 ref 1 00000100:00100000:0.0:1789760703.887479:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 683411021597 time 683410761181 state 0x1 ref 1 00000100:00100000:0.0:1789760703.887518:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 683411021597 time 683451816660 state 0x0 ref 1 00000100:00100000:0.0:1789760703.887535:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5306a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512546688:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760703.887603:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22040576, mtime: 0x6aad94bf, atime: 0x0, ctime: 0x6aad94bf, blocks: 0xb007 00010000:00010000:0.0:1789760703.887614:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2d44d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760703.887650:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=683s v=(0 0) 00010000:00010000:0.0:1789760703.887656:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760703.887664:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2d44d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22040576->22044671] (req 22040576->22044671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760703.887681:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2d44d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22040576->22044671] (req 22040576->22044671) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760703.963832:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22040576 -> 22044672 00010000:00010000:2.0:1789760703.963858:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2d44d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22040576->22044671] (req 22040576->22044671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.963873:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2d44d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22040576->22044671] (req 22040576->22044671) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.963884:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2d44d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22040576->22044671] (req 22040576->22044671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.963893:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760703.963899:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2d44d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22040576->22044671] (req 22040576->22044671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.963910:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2d44d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22040576->22044671] (req 22040576->22044671) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.963938:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2d44d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22040576->22044671] (req 22040576->22044671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.963948:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2d44d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22040576->22044671] (req 22040576->22044671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760703.963958:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972700 is committed 00000100:00100000:2.0:1789760703.964067:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5306a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512546688:12345-192.168.203.56@tcp:4:dd.0 Request processed in 76529us (76899us total) trans 4294972700 rc 0/0 00000100:00100000:2.0:1789760703.964079:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5411 00000100:00100000:2.0:1789760703.964092:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 683411021597 time 683451816660 state 0x2 ref 0 00000100:00100000:1.0:1789760703.971851:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760703.971959:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512546816 00000100:00100000:0.0:1789760703.971977:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512546816 00000100:00100000:0.0:1789760703.972054:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 683411021597 time 683451816660 state 0x0 ref 1 00000100:00100000:0.0:1789760703.972073:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 683452077076 time 683451816660 state 0x1 ref 1 00000100:00100000:0.0:1789760703.972109:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 683452077076 time 683536408070 state 0x0 ref 1 00000100:00100000:0.0:1789760703.972123:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5305880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512546816:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760703.972191:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22044672, mtime: 0x6aad94bf, atime: 0x0, ctime: 0x6aad94bf, blocks: 0xb007 00010000:00010000:0.0:1789760703.972202:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2d454 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760703.972223:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=683s v=(0 0) 00010000:00010000:0.0:1789760703.972228:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760703.972236:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2d454 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22044672->22048767] (req 22044672->22048767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760703.972253:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2d454 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22044672->22048767] (req 22044672->22048767) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760704.153091:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972701 is committed 00002000:00010000:2.0:1789760704.153275:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22044672 -> 22048768 00010000:00010000:2.0:1789760704.153307:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2d454 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22044672->22048767] (req 22044672->22048767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.153324:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2d454 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22044672->22048767] (req 22044672->22048767) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.153335:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2d454 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22044672->22048767] (req 22044672->22048767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.153343:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760704.153350:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2d454 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22044672->22048767] (req 22044672->22048767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.153360:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2d454 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22044672->22048767] (req 22044672->22048767) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.153387:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2d454 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22044672->22048767] (req 22044672->22048767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.153396:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2d454 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22044672->22048767] (req 22044672->22048767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760704.153520:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5305880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512546816:12345-192.168.203.56@tcp:4:dd.0 Request processed in 181392us (181677us total) trans 4294972701 rc 0/0 00000100:00100000:2.0:1789760704.153537:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5412 00000100:00100000:2.0:1789760704.153550:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 683452077076 time 683536408070 state 0x2 ref 0 00000100:00100000:0.0:1789760704.157852:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760704.158028:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512546944 00000100:00100000:2.0:1789760704.158045:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512546944 00000100:00100000:2.0:1789760704.158135:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 683452077076 time 683536408070 state 0x0 ref 1 00000100:00100000:2.0:1789760704.158159:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 683536668486 time 683536408070 state 0x1 ref 1 00000100:00100000:2.0:1789760704.158199:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 683536668486 time 683722497988 state 0x0 ref 1 00000100:00100000:2.0:1789760704.158215:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4074700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512546944:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760704.158294:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22048768, mtime: 0x6aad94bf, atime: 0x0, ctime: 0x6aad94bf, blocks: 0xb007 00010000:00010000:2.0:1789760704.158307:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2d45b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.158328:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=683s v=(0 0) 00010000:00010000:2.0:1789760704.158333:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760704.158342:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2d45b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22048768->22052863] (req 22048768->22052863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.158359:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2d45b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22048768->22052863] (req 22048768->22052863) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760704.228777:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22048768 -> 22052864 00010000:00010000:2.0:1789760704.228808:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2d45b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22048768->22052863] (req 22048768->22052863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.228826:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2d45b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22048768->22052863] (req 22048768->22052863) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.228837:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2d45b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22048768->22052863] (req 22048768->22052863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.228844:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760704.228851:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2d45b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22048768->22052863] (req 22048768->22052863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.228862:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2d45b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22048768->22052863] (req 22048768->22052863) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.228886:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2d45b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22048768->22052863] (req 22048768->22052863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.228892:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2d45b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22048768->22052863] (req 22048768->22052863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760704.229024:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4074700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2358:x1876699512546944:12345-192.168.203.56@tcp:4:dd.0 Request processed in 70805us (71179us total) trans 4294972702 rc 0/0 00000100:00100000:2.0:1789760704.229037:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5413 00000100:00100000:2.0:1789760704.230666:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 683536668486 time 683722497988 state 0x2 ref 0 00000001:00080000:1.0:1789760704.233201:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972702 is committed 00000100:00100000:1.0:1789760704.236191:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760704.236450:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512547072 00000100:00100000:2.0:1789760704.236465:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512547072 00000100:00100000:2.0:1789760704.236538:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 683536668486 time 683722497988 state 0x0 ref 1 00000100:00100000:2.0:1789760704.236559:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 683722758404 time 683722497988 state 0x1 ref 1 00000100:00100000:2.0:1789760704.236598:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 683722758404 time 683800896825 state 0x0 ref 1 00000100:00100000:2.0:1789760704.236627:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5305180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512547072:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760704.236734:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22052864, mtime: 0x6aad94bf, atime: 0x0, ctime: 0x6aad94bf, blocks: 0xb007 00010000:00010000:2.0:1789760704.236745:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2d462 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.236766:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=683s v=(0 0) 00010000:00010000:2.0:1789760704.236771:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760704.236780:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2d462 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22052864->22056959] (req 22052864->22056959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.236800:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2d462 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22052864->22056959] (req 22052864->22056959) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760704.304459:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972703 is committed 00002000:00010000:3.0:1789760704.304863:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22052864 -> 22056960 00010000:00010000:3.0:1789760704.304887:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2d462 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22052864->22056959] (req 22052864->22056959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760704.304901:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2d462 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22052864->22056959] (req 22052864->22056959) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760704.304912:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2d462 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22052864->22056959] (req 22052864->22056959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760704.304921:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760704.304927:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2d462 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22052864->22056959] (req 22052864->22056959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760704.304938:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2d462 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22052864->22056959] (req 22052864->22056959) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760704.304964:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2d462 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22052864->22056959] (req 22052864->22056959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760704.304974:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2d462 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22052864->22056959] (req 22052864->22056959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760704.305232:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5305180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512547072:12345-192.168.203.56@tcp:4:dd.0 Request processed in 68611us (69049us total) trans 4294972703 rc 0/0 00000100:00100000:0.0:1789760704.305244:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5414 00000100:00100000:0.0:1789760704.305256:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 683722758404 time 683800896825 state 0x2 ref 0 00000100:00100000:1.0:1789760704.322362:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760704.322456:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512547200 00000100:00100000:2.0:1789760704.322470:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512547200 00000100:00100000:2.0:1789760704.322552:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 683722758404 time 683800896825 state 0x0 ref 1 00000100:00100000:2.0:1789760704.322573:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 683801157241 time 683800896825 state 0x1 ref 1 00000100:00100000:2.0:1789760704.322621:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 683801157241 time 683886911093 state 0x0 ref 1 00000100:00100000:2.0:1789760704.322640:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5305500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512547200:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760704.322732:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22056960, mtime: 0x6aad94bf, atime: 0x0, ctime: 0x6aad94bf, blocks: 0xb007 00010000:00010000:2.0:1789760704.322743:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2d469 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.322762:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=683s v=(0 0) 00010000:00010000:2.0:1789760704.322768:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760704.322775:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2d469 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22056960->22061055] (req 22056960->22061055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.322793:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2d469 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22056960->22061055] (req 22056960->22061055) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760704.460486:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972704 is committed 00002000:00010000:3.0:1789760704.460724:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22056960 -> 22061056 00010000:00010000:3.0:1789760704.460746:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2d469 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22056960->22061055] (req 22056960->22061055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760704.460762:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2d469 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22056960->22061055] (req 22056960->22061055) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760704.460772:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2d469 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22056960->22061055] (req 22056960->22061055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760704.460780:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760704.460786:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2d469 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22056960->22061055] (req 22056960->22061055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760704.460796:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2d469 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22056960->22061055] (req 22056960->22061055) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760704.460827:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2d469 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22056960->22061055] (req 22056960->22061055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760704.460836:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2d469 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22056960->22061055] (req 22056960->22061055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760704.461218:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5305500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512547200:12345-192.168.203.56@tcp:4:dd.0 Request processed in 138573us (138865us total) trans 4294972704 rc 0/0 00000100:00100000:3.0:1789760704.461235:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5415 00000100:00100000:3.0:1789760704.461248:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 683801157241 time 683886911093 state 0x2 ref 0 00000100:00100000:3.0:1789760704.474270:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760704.476766:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512547328 00000100:00100000:2.0:1789760704.476786:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512547328 00000100:00100000:3.0:1789760704.477585:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 683801157241 time 683886911093 state 0x0 ref 1 00000100:00100000:3.0:1789760704.477620:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 683887171509 time 683886911093 state 0x1 ref 1 00000100:00100000:3.0:1789760704.477667:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 683887171509 time 684041965401 state 0x0 ref 1 00000100:00100000:3.0:1789760704.477684:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2745880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512547328:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760704.480257:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22061056, mtime: 0x6aad94bf, atime: 0x0, ctime: 0x6aad94bf, blocks: 0xb007 00010000:00010000:0.0:1789760704.480276:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2d470 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760704.480439:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=684s v=(0 0) 00010000:00010000:0.0:1789760704.480445:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760704.480454:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2d470 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22061056->22065151] (req 22061056->22065151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760704.480475:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2d470 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22061056->22065151] (req 22061056->22065151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760704.595834:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972705 is committed 00002000:00010000:3.0:1789760704.598243:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22061056 -> 22065152 00010000:00010000:3.0:1789760704.598270:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2d470 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22061056->22065151] (req 22061056->22065151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760704.598293:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2d470 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22061056->22065151] (req 22061056->22065151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760704.598304:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2d470 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22061056->22065151] (req 22061056->22065151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760704.598313:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760704.598320:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2d470 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22061056->22065151] (req 22061056->22065151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760704.598335:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2d470 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22061056->22065151] (req 22061056->22065151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760704.598361:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2d470 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22061056->22065151] (req 22061056->22065151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760704.598369:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2d470 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22061056->22065151] (req 22061056->22065151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760704.598515:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2745880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512547328:12345-192.168.203.56@tcp:4:dd.0 Request processed in 120827us (124250us total) trans 4294972705 rc 0/0 00000100:00100000:3.0:1789760704.598528:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5416 00000100:00100000:3.0:1789760704.598543:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 683887171509 time 684041965401 state 0x2 ref 0 00000100:00100000:0.0:1789760704.603387:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760704.603542:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512547456 00000100:00100000:1.0:1789760704.603556:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512547456 00000100:00100000:1.0:1789760704.603648:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 683887171509 time 684041965401 state 0x0 ref 1 00000100:00100000:1.0:1789760704.603668:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 684042225817 time 684041965401 state 0x1 ref 1 00000100:00100000:1.0:1789760704.603744:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 684042225817 time 684168043070 state 0x0 ref 1 00000100:00100000:1.0:1789760704.603760:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4074a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512547456:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760704.603829:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22065152, mtime: 0x6aad94bf, atime: 0x0, ctime: 0x6aad94bf, blocks: 0xb007 00010000:00010000:1.0:1789760704.603840:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2d477 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760704.603860:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=684s v=(0 0) 00010000:00010000:1.0:1789760704.603866:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760704.603875:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2d477 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22065152->22069247] (req 22065152->22069247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760704.603895:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2d477 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22065152->22069247] (req 22065152->22069247) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760704.723686:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972706 is committed 00002000:00010000:3.0:1789760704.723903:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22065152 -> 22069248 00002000:00010000:3.0:1789760704.723909:0:13062:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760703 -> 1789760704 00002000:00010000:3.0:1789760704.723913:0:13062:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760703 -> 1789760704 00010000:00010000:3.0:1789760704.723931:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2d477 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22065152->22069247] (req 22065152->22069247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760704.723946:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2d477 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22065152->22069247] (req 22065152->22069247) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760704.723956:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2d477 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22065152->22069247] (req 22065152->22069247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760704.723972:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760704.723979:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2d477 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22065152->22069247] (req 22065152->22069247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760704.723990:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2d477 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22065152->22069247] (req 22065152->22069247) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760704.724024:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2d477 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22065152->22069247] (req 22065152->22069247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760704.724034:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2d477 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22065152->22069247] (req 22065152->22069247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760704.724158:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4074a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512547456:12345-192.168.203.56@tcp:4:dd.0 Request processed in 120393us (120779us total) trans 4294972706 rc 0/0 00000100:00100000:3.0:1789760704.724169:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5417 00000100:00100000:3.0:1789760704.724181:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 684042225817 time 684168043070 state 0x2 ref 0 00000100:00100000:1.0:1789760704.728022:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760704.733264:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512547584 00000100:00100000:3.0:1789760704.733296:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512547584 00000100:00100000:3.0:1789760704.733391:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 684042225817 time 684168043070 state 0x0 ref 1 00000100:00100000:3.0:1789760704.733411:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 684168303486 time 684168043070 state 0x1 ref 1 00000100:00100000:3.0:1789760704.733450:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 684168303486 time 684297749920 state 0x0 ref 1 00000100:00100000:3.0:1789760704.733470:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5304a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512547584:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760704.733540:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22069248, mtime: 0x6aad94c0, atime: 0x0, ctime: 0x6aad94c0, blocks: 0xb007 00010000:00010000:3.0:1789760704.733551:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2d47e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760704.733572:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=684s v=(0 0) 00010000:00010000:3.0:1789760704.733578:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760704.733586:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2d47e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22069248->22073343] (req 22069248->22073343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760704.733604:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2d47e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22069248->22073343] (req 22069248->22073343) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760704.823913:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972707 is committed 00002000:00010000:3.0:1789760704.824030:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22069248 -> 22073344 00010000:00010000:3.0:1789760704.824061:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2d47e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22069248->22073343] (req 22069248->22073343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760704.824077:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2d47e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22069248->22073343] (req 22069248->22073343) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760704.824086:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2d47e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22069248->22073343] (req 22069248->22073343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760704.824094:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760704.824100:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2d47e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22069248->22073343] (req 22069248->22073343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760704.824112:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2d47e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22069248->22073343] (req 22069248->22073343) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760704.824139:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2d47e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22069248->22073343] (req 22069248->22073343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760704.824148:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2d47e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22069248->22073343] (req 22069248->22073343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760704.824310:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5304a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512547584:12345-192.168.203.56@tcp:4:dd.0 Request processed in 90836us (96295us total) trans 4294972707 rc 0/0 00000100:00100000:3.0:1789760704.824321:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5418 00000100:00100000:3.0:1789760704.824334:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 684168303486 time 684297749920 state 0x2 ref 0 00000100:00100000:0.0:1789760704.827482:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760704.828327:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512547712 00000100:00100000:3.0:1789760704.828342:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512547712 00000100:00100000:3.0:1789760704.828422:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 684168303486 time 684297749920 state 0x0 ref 1 00000100:00100000:3.0:1789760704.828442:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 684298010336 time 684297749920 state 0x1 ref 1 00000100:00100000:3.0:1789760704.828520:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 684298010336 time 684392819160 state 0x0 ref 1 00000100:00100000:3.0:1789760704.828537:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4059180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512547712:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760704.828602:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22073344, mtime: 0x6aad94c0, atime: 0x0, ctime: 0x6aad94c0, blocks: 0xb007 00010000:00010000:3.0:1789760704.828632:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2d485 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760704.828654:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=684s v=(0 0) 00010000:00010000:3.0:1789760704.828660:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760704.828668:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2d485 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22073344->22077439] (req 22073344->22077439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760704.828686:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2d485 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22073344->22077439] (req 22073344->22077439) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760704.885791:0:5836:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:2.0:1789760704.954174:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972708 is committed 00002000:00010000:0.0:1789760704.954847:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22073344 -> 22077440 00010000:00010000:0.0:1789760704.954869:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2d485 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22073344->22077439] (req 22073344->22077439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760704.954884:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2d485 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22073344->22077439] (req 22073344->22077439) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760704.954895:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2d485 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22073344->22077439] (req 22073344->22077439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760704.954904:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760704.954910:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2d485 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22073344->22077439] (req 22073344->22077439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760704.954922:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2d485 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22073344->22077439] (req 22073344->22077439) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760704.954945:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2d485 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22073344->22077439] (req 22073344->22077439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760704.954955:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2d485 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22073344->22077439] (req 22073344->22077439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760704.955160:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4059180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512547712:12345-192.168.203.56@tcp:4:dd.0 Request processed in 126618us (127685us total) trans 4294972708 rc 0/0 00000100:00100000:0.0:1789760704.955173:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5419 00000100:00100000:0.0:1789760704.955186:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 684298010336 time 684392819160 state 0x2 ref 0 00000100:00100000:0.0:1789760704.958596:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760704.958763:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512547840 00000100:00100000:3.0:1789760704.958777:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512547840 00000100:00100000:3.0:1789760704.958857:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 684298010336 time 684392819160 state 0x0 ref 1 00000100:00100000:3.0:1789760704.958876:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 684393079576 time 684392819160 state 0x1 ref 1 00000100:00100000:3.0:1789760704.958940:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 684393079576 time 684523238894 state 0x0 ref 1 00000100:00100000:3.0:1789760704.958953:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4059500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512547840:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760704.959028:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22077440, mtime: 0x6aad94c0, atime: 0x0, ctime: 0x6aad94c0, blocks: 0xb007 00010000:00010000:3.0:1789760704.959039:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2d48c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760704.959058:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=684s v=(0 0) 00010000:00010000:3.0:1789760704.959063:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760704.959071:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2d48c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22077440->22081535] (req 22077440->22081535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760704.959095:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2d48c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22077440->22081535] (req 22077440->22081535) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760705.021435:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972709 is committed 00002000:00010000:0.0:1789760705.021569:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22077440 -> 22081536 00010000:00010000:0.0:1789760705.021589:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2d48c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22077440->22081535] (req 22077440->22081535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.021605:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2d48c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22077440->22081535] (req 22077440->22081535) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.021628:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2d48c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22077440->22081535] (req 22077440->22081535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.021637:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760705.021643:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2d48c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22077440->22081535] (req 22077440->22081535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.021654:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2d48c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22077440->22081535] (req 22077440->22081535) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.021687:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2d48c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22077440->22081535] (req 22077440->22081535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.021719:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2d48c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22077440->22081535] (req 22077440->22081535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760705.021895:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4059500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512547840:12345-192.168.203.56@tcp:4:dd.0 Request processed in 62935us (63308us total) trans 4294972709 rc 0/0 00000100:00100000:0.0:1789760705.021907:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5420 00000100:00100000:0.0:1789760705.021920:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 684393079576 time 684523238894 state 0x2 ref 0 00000100:00100000:1.0:1789760705.025921:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760705.026093:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512547968 00000100:00100000:3.0:1789760705.026108:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512547968 00000100:00100000:3.0:1789760705.026239:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 684393079576 time 684523238894 state 0x0 ref 1 00000100:00100000:3.0:1789760705.026259:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 684523499310 time 684523238894 state 0x1 ref 1 00000100:00100000:3.0:1789760705.026391:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 684523499310 time 684590689150 state 0x0 ref 1 00000100:00100000:3.0:1789760705.026407:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5306d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512547968:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760705.026472:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22081536, mtime: 0x6aad94c0, atime: 0x0, ctime: 0x6aad94c0, blocks: 0xb007 00010000:00010000:3.0:1789760705.026482:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2d493 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760705.026501:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=684s v=(0 0) 00010000:00010000:3.0:1789760705.026506:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760705.026514:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2d493 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22081536->22085631] (req 22081536->22085631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760705.026530:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2d493 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22081536->22085631] (req 22081536->22085631) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1789760705.027403:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:1.0:1789760705.061375:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972710 is committed 00002000:00010000:2.0:1789760705.061523:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22081536 -> 22085632 00010000:00010000:2.0:1789760705.061543:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2d493 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22081536->22085631] (req 22081536->22085631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.061558:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2d493 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22081536->22085631] (req 22081536->22085631) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.061574:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2d493 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22081536->22085631] (req 22081536->22085631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.061581:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760705.061586:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2d493 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22081536->22085631] (req 22081536->22085631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.061595:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2d493 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22081536->22085631] (req 22081536->22085631) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.061631:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2d493 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22081536->22085631] (req 22081536->22085631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.061639:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2d493 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22081536->22085631] (req 22081536->22085631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760705.061896:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5306d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512547968:12345-192.168.203.56@tcp:4:dd.0 Request processed in 35485us (35982us total) trans 4294972710 rc 0/0 00000100:00100000:2.0:1789760705.061909:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5421 00000100:00100000:2.0:1789760705.061921:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 684523499310 time 684590689150 state 0x2 ref 0 00000100:00100000:1.0:1789760705.066490:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760705.066605:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512548096 00000100:00100000:0.0:1789760705.066626:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512548096 00000100:00100000:0.0:1789760705.066833:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 684523499310 time 684590689150 state 0x0 ref 1 00000100:00100000:0.0:1789760705.066856:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 684590949566 time 684590689150 state 0x1 ref 1 00000100:00100000:0.0:1789760705.066927:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 684590949566 time 684631225322 state 0x0 ref 1 00000100:00100000:0.0:1789760705.066942:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5306300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512548096:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760705.067019:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22085632, mtime: 0x6aad94c0, atime: 0x0, ctime: 0x6aad94c0, blocks: 0xb007 00010000:00010000:0.0:1789760705.067030:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2d49a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.067054:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=684s v=(0 0) 00010000:00010000:0.0:1789760705.067060:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760705.067068:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2d49a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22085632->22089727] (req 22085632->22089727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.067085:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2d49a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22085632->22089727] (req 22085632->22089727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760705.145112:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22085632 -> 22089728 00010000:00010000:0.0:1789760705.145145:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2d49a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22085632->22089727] (req 22085632->22089727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.145160:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2d49a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22085632->22089727] (req 22085632->22089727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.145171:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2d49a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22085632->22089727] (req 22085632->22089727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.145178:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760705.145184:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2d49a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22085632->22089727] (req 22085632->22089727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.145193:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2d49a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22085632->22089727] (req 22085632->22089727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.145217:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2d49a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22085632->22089727] (req 22085632->22089727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.145227:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2d49a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22085632->22089727] (req 22085632->22089727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760705.145355:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5306300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2359:x1876699512548096:12345-192.168.203.56@tcp:4:dd.0 Request processed in 78409us (78871us total) trans 4294972711 rc 0/0 00000100:00100000:0.0:1789760705.145368:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5422 00000100:00100000:0.0:1789760705.145380:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 684590949566 time 684631225322 state 0x2 ref 0 00000001:00080000:1.0:1789760705.145724:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972711 is committed 00000100:00100000:2.0:1789760705.156116:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760705.158835:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512548224 00000100:00100000:0.0:1789760705.158858:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512548224 00000100:00100000:0.0:1789760705.158954:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 684590949566 time 684631225322 state 0x0 ref 1 00000100:00100000:0.0:1789760705.158975:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 684631485738 time 684631225322 state 0x1 ref 1 00000100:00100000:1.0:1789760705.159122:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 684631485738 time 684723417772 state 0x0 ref 1 00000100:00100000:1.0:1789760705.159142:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac38b4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512548224:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760705.159211:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22089728, mtime: 0x6aad94c0, atime: 0x0, ctime: 0x6aad94c0, blocks: 0xb007 00010000:00010000:1.0:1789760705.159223:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2d4a1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760705.159243:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=684s v=(0 0) 00010000:00010000:1.0:1789760705.159248:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760705.159257:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2d4a1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22089728->22093823] (req 22089728->22093823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760705.159273:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2d4a1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22089728->22093823] (req 22089728->22093823) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760705.303652:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972712 is committed 00002000:00010000:1.0:1789760705.309661:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22089728 -> 22093824 00010000:00010000:1.0:1789760705.309690:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2d4a1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22089728->22093823] (req 22089728->22093823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760705.309718:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2d4a1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22089728->22093823] (req 22089728->22093823) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760705.309728:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2d4a1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22089728->22093823] (req 22089728->22093823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760705.309736:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760705.309741:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2d4a1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22089728->22093823] (req 22089728->22093823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760705.309752:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2d4a1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22089728->22093823] (req 22089728->22093823) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760705.309777:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2d4a1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22089728->22093823] (req 22089728->22093823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760705.309786:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2d4a1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22089728->22093823] (req 22089728->22093823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760705.309910:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac38b4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512548224:12345-192.168.203.56@tcp:4:dd.0 Request processed in 150766us (153803us total) trans 4294972712 rc 0/0 00000100:00100000:1.0:1789760705.309922:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5423 00000100:00100000:1.0:1789760705.309935:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 684631485738 time 684723417772 state 0x2 ref 0 00000100:00100000:1.0:1789760705.313617:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760705.313766:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512548352 00000100:00100000:2.0:1789760705.313781:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512548352 00000100:00100000:2.0:1789760705.313856:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 684631485738 time 684723417772 state 0x0 ref 1 00000100:00100000:2.0:1789760705.313881:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 684723678188 time 684723417772 state 0x1 ref 1 00000100:00100000:2.0:1789760705.313920:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 684723678188 time 684878218549 state 0x0 ref 1 00000100:00100000:2.0:1789760705.313934:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5307800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512548352:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760705.314000:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22093824, mtime: 0x6aad94c0, atime: 0x0, ctime: 0x6aad94c0, blocks: 0xb007 00010000:00010000:2.0:1789760705.314011:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2d4a8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.314030:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=684s v=(0 0) 00010000:00010000:2.0:1789760705.314036:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760705.314045:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2d4a8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22093824->22097919] (req 22093824->22097919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.314073:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2d4a8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22093824->22097919] (req 22093824->22097919) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760705.448851:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972713 is committed 00002000:00010000:0.0:1789760705.449927:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22093824 -> 22097920 00010000:00010000:0.0:1789760705.449965:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2d4a8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22093824->22097919] (req 22093824->22097919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.449980:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2d4a8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22093824->22097919] (req 22093824->22097919) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.449990:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2d4a8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22093824->22097919] (req 22093824->22097919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.449998:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760705.450004:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2d4a8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22093824->22097919] (req 22093824->22097919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.450013:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2d4a8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22093824->22097919] (req 22093824->22097919) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.450036:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2d4a8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22093824->22097919] (req 22093824->22097919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.450043:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2d4a8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22093824->22097919] (req 22093824->22097919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760705.450153:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5307800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512548352:12345-192.168.203.56@tcp:4:dd.0 Request processed in 136216us (136555us total) trans 4294972713 rc 0/0 00000100:00100000:0.0:1789760705.450165:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5424 00000100:00100000:0.0:1789760705.450175:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 684723678188 time 684878218549 state 0x2 ref 0 00000100:00100000:0.0:1789760705.457795:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760705.458171:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512548480 00000100:00100000:1.0:1789760705.458182:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512548480 00000100:00100000:1.0:1789760705.458248:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 684723678188 time 684878218549 state 0x0 ref 1 00000100:00100000:1.0:1789760705.458266:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 684878478965 time 684878218549 state 0x1 ref 1 00000100:00100000:1.0:1789760705.458296:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 684878478965 time 685022595456 state 0x0 ref 1 00000100:00100000:1.0:1789760705.458310:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4058a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512548480:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760705.458366:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22097920, mtime: 0x6aad94c0, atime: 0x0, ctime: 0x6aad94c0, blocks: 0xb007 00010000:00010000:1.0:1789760705.458375:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2d4af lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760705.458393:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=685s v=(0 0) 00010000:00010000:1.0:1789760705.458398:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760705.458404:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2d4af lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22097920->22102015] (req 22097920->22102015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760705.458418:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2d4af lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22097920->22102015] (req 22097920->22102015) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760705.461443:0:3310:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8feac2745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:6df98d54-eb52-4c5c-8fc9-ee9d74f90f98:3310:1876699540216448:0@lo:400:kworker.0 00000100:00100000:0.0:1789760705.461548:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760705.464324:0:3309:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8feac2746d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3309:1876699540216832:0@lo:400:kworker.0 00000100:00100000:2.0:1789760705.464438:0:3309:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760705.464512:0:3312:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8feac2744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3312:1876699540216704:0@lo:400:kworker.0 00000100:00100000:2.0:1789760705.464547:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760705.464572:0:3312:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1789760705.464578:0:3312:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8feac2745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3312:1876699540216576:0@lo:400:kworker.0 00000100:00100000:2.0:1789760705.464615:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760705.464651:0:13064:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540216832 00000100:00100000:2.0:1789760705.464661:0:13064:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540216832 00000100:00100000:2.0:1789760705.464757:0:13064:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 167 00000100:00100000:2.0:1789760705.464768:0:13064:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac38b6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3309:x1876699540216832:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760705.464835:0:13064:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac38b6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3309:x1876699540216832:12345-0@lo:400:kworker.0 Request processed in 69us (397us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760705.464843:0:13064:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 167 00000100:00100000:2.0:1789760705.464875:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540216704 00000100:00100000:2.0:1789760705.464879:0:5847:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540216704 00000100:00100000:2.0:1789760705.464907:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540216576 00000100:00100000:2.0:1789760705.464910:0:5847:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540216576 00000100:00100000:2.0:1789760705.464930:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 436 00000100:00100000:2.0:1789760705.464936:0:5847:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac38b5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3312:x1876699540216704:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760705.464973:0:5847:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac38b5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3312:x1876699540216704:12345-0@lo:400:kworker.0 Request processed in 37us (424us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760705.464980:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 436 00000100:00100000:2.0:1789760705.464992:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 437 00000100:00100000:2.0:1789760705.465195:0:5847:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac38b5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3312:x1876699540216576:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760705.465230:0:5847:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac38b5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3312:x1876699540216576:12345-0@lo:400:kworker.0 Request processed in 233us (625us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760705.465236:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 437 00000100:00080000:2.0:1789760705.465259:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:2.0:1789760705.465265:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1789760705.465302:0:3309:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8feac2746d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3309:1876699540216832:0@lo:400:kworker.0 00000100:00100000:2.0:1789760705.465485:0:3312:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8feac2745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3312:1876699540216576:0@lo:400:kworker.0 00000100:00100000:2.0:1789760705.465497:0:3312:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8feac2744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3312:1876699540216704:0@lo:400:kworker.0 00000100:00100000:0.0:1789760705.469450:0:5842:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540216448 00000100:00100000:0.0:1789760705.469470:0:5842:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540216448 00000100:00100000:0.0:1789760705.469534:0:5842:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 383 00000100:00100000:0.0:1789760705.469546:0:5842:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febe00d7050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:6df98d54-eb52-4c5c-8fc9-ee9d74f90f98+19:3310:x1876699540216448:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760705.469681:0:5842:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febe00d7050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:6df98d54-eb52-4c5c-8fc9-ee9d74f90f98+19:3310:x1876699540216448:12345-0@lo:400:kworker.0 Request processed in 137us (8133us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760705.469689:0:5842:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 383 00000100:00100000:0.0:1789760705.469795:0:3310:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8feac2745180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:6df98d54-eb52-4c5c-8fc9-ee9d74f90f98:3310:1876699540216448:0@lo:400:kworker.0 00000100:00100000:3.0:1789760705.473530:0:3309:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1789760705.473563:0:3309:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8feac2746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3309:1876699540216960:0@lo:400:kworker.0 00000100:00100000:3.0:1789760705.473664:0:3309:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760705.473747:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540216960 00000100:00100000:3.0:1789760705.473757:0:5847:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540216960 00000100:00100000:3.0:1789760705.473830:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 438 00000100:00100000:3.0:1789760705.473840:0:5847:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2745c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3309:x1876699540216960:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760705.473903:0:5847:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2745c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3309:x1876699540216960:12345-0@lo:400:kworker.0 Request processed in 64us (239us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760705.473911:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 438 00000100:00080000:3.0:1789760705.473934:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789760705.473973:0:3309:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8feac2746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3309:1876699540216960:0@lo:400:kworker.0 00000100:00100000:3.0:1789760705.474985:0:3312:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1789760705.475219:0:3312:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8feac2746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3312:1876699540217088:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1789760705.475283:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760705.475331:0:6706:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540217088 00000100:00100000:3.0:1789760705.476430:0:6706:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540217088 00000100:00100000:3.0:1789760705.476503:0:6706:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 182 00000100:00100000:3.0:1789760705.476515:0:6706:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2744e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3312:x1876699540217088:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1789760705.476545:0:6706:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 528285696 pending 1703936 free 3835691008 avail 3824156672 00000100:00100000:3.0:1789760705.476627:0:6706:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2744e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3312:x1876699540217088:12345-0@lo:13:osp-pre-0-0.0 Request processed in 114us (1348us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760705.476635:0:6706:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 182 00000100:00100000:3.0:1789760705.476712:0:3312:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8feac2746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3312:1876699540217088:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1789760705.476772:0:3312:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8feac2746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3312:1876699540217216:0@lo:400:kworker.0 00000100:00100000:3.0:1789760705.476806:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760705.476841:0:10102:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540217216 00000100:00100000:3.0:1789760705.476846:0:10102:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540217216 00000100:00100000:3.0:1789760705.476889:0:10102:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 168 00000100:00100000:3.0:1789760705.476896:0:10102:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2747100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3312:x1876699540217216:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760705.476934:0:10102:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2747100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3312:x1876699540217216:12345-0@lo:400:kworker.0 Request processed in 39us (127us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760705.476940:0:10102:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 168 00000100:00080000:3.0:1789760705.476958:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1789760705.476980:0:3312:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8feac2746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3312:1876699540217216:0@lo:400:kworker.0 00002000:00010000:0.0:1789760705.587498:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22097920 -> 22102016 00010000:00010000:0.0:1789760705.587528:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2d4af lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22097920->22102015] (req 22097920->22102015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.587543:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2d4af lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22097920->22102015] (req 22097920->22102015) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.587552:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2d4af lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22097920->22102015] (req 22097920->22102015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.587559:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760705.587564:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2d4af lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22097920->22102015] (req 22097920->22102015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.587574:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2d4af lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22097920->22102015] (req 22097920->22102015) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.587598:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2d4af lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22097920->22102015] (req 22097920->22102015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760705.588140:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972714 is committed 00010000:00010000:0.0:1789760705.588455:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2d4af lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22097920->22102015] (req 22097920->22102015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760705.594480:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4058a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512548480:12345-192.168.203.56@tcp:4:dd.0 Request processed in 136155us (136676us total) trans 4294972714 rc 0/0 00000100:00100000:1.0:1789760705.594500:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5425 00000100:00100000:1.0:1789760705.594512:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 684878478965 time 685022595456 state 0x2 ref 0 00000100:00100000:1.0:1789760705.607479:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760705.611397:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512548608 00000100:00100000:0.0:1789760705.611419:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512548608 00000100:00100000:0.0:1789760705.611513:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 684878478965 time 685022595456 state 0x0 ref 1 00000100:00100000:0.0:1789760705.611535:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 685022855872 time 685022595456 state 0x1 ref 1 00000100:00100000:0.0:1789760705.611578:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 685022855872 time 685175876961 state 0x0 ref 1 00000100:00100000:0.0:1789760705.611595:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5304000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512548608:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760705.611967:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22102016, mtime: 0x6aad94c0, atime: 0x0, ctime: 0x6aad94c0, blocks: 0xb007 00010000:00010000:0.0:1789760705.611979:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2d4b6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.612125:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=685s v=(0 0) 00010000:00010000:0.0:1789760705.612131:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760705.612139:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2d4b6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22102016->22106111] (req 22102016->22106111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.612156:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2d4b6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22102016->22106111] (req 22102016->22106111) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760705.759498:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972715 is committed 00002000:00010000:0.0:1789760705.759630:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22102016 -> 22106112 00002000:00010000:0.0:1789760705.759636:0:13063:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760704 -> 1789760705 00002000:00010000:0.0:1789760705.759639:0:13063:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760704 -> 1789760705 00010000:00010000:0.0:1789760705.759656:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2d4b6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22102016->22106111] (req 22102016->22106111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.759671:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2d4b6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22102016->22106111] (req 22102016->22106111) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.759687:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2d4b6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22102016->22106111] (req 22102016->22106111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.759804:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760705.759811:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2d4b6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22102016->22106111] (req 22102016->22106111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.759823:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2d4b6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22102016->22106111] (req 22102016->22106111) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.759847:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2d4b6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22102016->22106111] (req 22102016->22106111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.759856:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2d4b6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22102016->22106111] (req 22102016->22106111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760705.759980:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5304000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512548608:12345-192.168.203.56@tcp:4:dd.0 Request processed in 148382us (152506us total) trans 4294972715 rc 0/0 00000100:00100000:0.0:1789760705.760000:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5426 00000100:00100000:0.0:1789760705.760013:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 685022855872 time 685175876961 state 0x2 ref 0 00000100:00100000:3.0:1789760705.781129:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760705.781230:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512548736 00000100:00100000:0.0:1789760705.781242:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512548736 00000100:00100000:0.0:1789760705.781416:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 685022855872 time 685175876961 state 0x0 ref 1 00000100:00100000:0.0:1789760705.781435:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 685176137377 time 685175876961 state 0x1 ref 1 00000100:00100000:0.0:1789760705.781502:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 685176137377 time 685345801231 state 0x0 ref 1 00000100:00100000:0.0:1789760705.781516:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2746300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512548736:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760705.781581:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22106112, mtime: 0x6aad94c1, atime: 0x0, ctime: 0x6aad94c1, blocks: 0xb007 00010000:00010000:0.0:1789760705.781592:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2d4bd lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.781625:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=685s v=(0 0) 00010000:00010000:0.0:1789760705.781630:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760705.781637:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2d4bd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22106112->22110207] (req 22106112->22110207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.781655:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2d4bd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22106112->22110207] (req 22106112->22110207) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760705.847193:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972716 is committed 00002000:00010000:2.0:1789760705.847678:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22106112 -> 22110208 00010000:00010000:2.0:1789760705.847885:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2d4bd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22106112->22110207] (req 22106112->22110207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.847901:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2d4bd lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22106112->22110207] (req 22106112->22110207) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.847912:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2d4bd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22106112->22110207] (req 22106112->22110207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.847920:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760705.847926:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2d4bd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22106112->22110207] (req 22106112->22110207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.847937:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2d4bd lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22106112->22110207] (req 22106112->22110207) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.847960:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2d4bd lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22106112->22110207] (req 22106112->22110207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.847970:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2d4bd lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22106112->22110207] (req 22106112->22110207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760705.848266:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2746300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512548736:12345-192.168.203.56@tcp:4:dd.0 Request processed in 66745us (67144us total) trans 4294972716 rc 0/0 00000100:00100000:2.0:1789760705.849523:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5427 00000100:00100000:2.0:1789760705.849537:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 685176137377 time 685345801231 state 0x2 ref 0 00000100:00100000:1.0:1789760705.853656:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760705.853810:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512548864 00000100:00100000:2.0:1789760705.853824:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512548864 00000100:00100000:2.0:1789760705.853899:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 685176137377 time 685345801231 state 0x0 ref 1 00000100:00100000:2.0:1789760705.853918:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 685346061647 time 685345801231 state 0x1 ref 1 00000100:00100000:2.0:1789760705.853946:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 685346061647 time 685418245254 state 0x0 ref 1 00000100:00100000:2.0:1789760705.853961:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5304380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512548864:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760705.854055:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22110208, mtime: 0x6aad94c1, atime: 0x0, ctime: 0x6aad94c1, blocks: 0xb007 00010000:00010000:2.0:1789760705.854066:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2d4c4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.854086:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=685s v=(0 0) 00010000:00010000:2.0:1789760705.854092:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760705.854100:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2d4c4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22110208->22114303] (req 22110208->22114303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.854116:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2d4c4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22110208->22114303] (req 22110208->22114303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760705.907252:0:5835:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:1.0:1789760705.934456:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972717 is committed 00002000:00010000:3.0:1789760705.934621:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22110208 -> 22114304 00010000:00010000:3.0:1789760705.934643:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2d4c4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22110208->22114303] (req 22110208->22114303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760705.934658:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2d4c4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22110208->22114303] (req 22110208->22114303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760705.934668:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2d4c4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22110208->22114303] (req 22110208->22114303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760705.934676:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760705.934682:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2d4c4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22110208->22114303] (req 22110208->22114303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760705.934693:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2d4c4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22110208->22114303] (req 22110208->22114303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760705.934801:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2d4c4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22110208->22114303] (req 22110208->22114303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760705.934809:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2d4c4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22110208->22114303] (req 22110208->22114303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760705.934935:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5304380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512548864:12345-192.168.203.56@tcp:4:dd.0 Request processed in 80970us (81286us total) trans 4294972717 rc 0/0 00000100:00100000:3.0:1789760705.934947:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5428 00000100:00100000:3.0:1789760705.934959:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 685346061647 time 685418245254 state 0x2 ref 0 00000100:00100000:1.0:1789760705.941574:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760705.941693:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512548992 00000100:00100000:0.0:1789760705.941747:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512548992 00000100:00100000:0.0:1789760705.941824:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 685346061647 time 685418245254 state 0x0 ref 1 00000100:00100000:0.0:1789760705.941844:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 685418505670 time 685418245254 state 0x1 ref 1 00000100:00100000:0.0:1789760705.941883:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 685418505670 time 685506181858 state 0x0 ref 1 00000100:00100000:0.0:1789760705.941899:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5306680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512548992:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760705.941966:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22114304, mtime: 0x6aad94c1, atime: 0x0, ctime: 0x6aad94c1, blocks: 0xb007 00010000:00010000:0.0:1789760705.941978:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2d4cb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.942001:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=685s v=(0 0) 00010000:00010000:0.0:1789760705.942006:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760705.942013:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2d4cb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22114304->22118399] (req 22114304->22118399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.942186:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2d4cb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22114304->22118399] (req 22114304->22118399) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760705.971375:0:3309:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8feac38b4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3309:1876699540217344:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1789760705.971469:0:3309:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760705.971690:0:6706:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540217344 00000100:00100000:2.0:1789760705.971726:0:6706:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540217344 00000100:00100000:2.0:1789760705.971779:0:6706:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 183 00000100:00100000:2.0:1789760705.971790:0:6706:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2745f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3309:x1876699540217344:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1789760705.971833:0:6706:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3859808256 avail 3857711104 00000100:00100000:2.0:1789760705.971889:0:6706:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2745f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3309:x1876699540217344:12345-0@lo:13:osp-pre-1-0.0 Request processed in 99us (419us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760705.971897:0:6706:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 183 00000100:00100000:2.0:1789760705.971955:0:3309:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8feac38b4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3309:1876699540217344:0@lo:13:osp-pre-1-0.0 00000001:00080000:3.0:1789760706.034550:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972718 is committed 00002000:00010000:1.0:1789760706.034765:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22114304 -> 22118400 00010000:00010000:1.0:1789760706.034785:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2d4cb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22114304->22118399] (req 22114304->22118399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760706.034799:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2d4cb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22114304->22118399] (req 22114304->22118399) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760706.034809:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2d4cb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22114304->22118399] (req 22114304->22118399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760706.034816:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760706.034822:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2d4cb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22114304->22118399] (req 22114304->22118399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760706.034833:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2d4cb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22114304->22118399] (req 22114304->22118399) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760706.034857:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2d4cb lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22114304->22118399] (req 22114304->22118399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760706.034866:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2d4cb lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22114304->22118399] (req 22114304->22118399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760706.034975:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5306680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512548992:12345-192.168.203.56@tcp:4:dd.0 Request processed in 93074us (93411us total) trans 4294972718 rc 0/0 00000100:00100000:1.0:1789760706.034988:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5429 00000100:00100000:1.0:1789760706.035010:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 685418505670 time 685506181858 state 0x2 ref 0 00000100:00100000:1.0:1789760706.040290:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760706.041206:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512549120 00000100:00100000:2.0:1789760706.041223:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512549120 00000100:00100000:3.0:1789760706.041521:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 685418505670 time 685506181858 state 0x0 ref 1 00000100:00100000:3.0:1789760706.041541:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 685506442274 time 685506181858 state 0x1 ref 1 00000100:00100000:3.0:1789760706.041634:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 685506442274 time 685605931305 state 0x0 ref 1 00000100:00100000:3.0:1789760706.041652:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5307b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512549120:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760706.041755:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22118400, mtime: 0x6aad94c1, atime: 0x0, ctime: 0x6aad94c1, blocks: 0xb007 00010000:00010000:3.0:1789760706.041766:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2d4d2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760706.041792:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=685s v=(0 0) 00010000:00010000:3.0:1789760706.041797:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760706.041805:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2d4d2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22118400->22122495] (req 22118400->22122495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760706.041820:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2d4d2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22118400->22122495] (req 22118400->22122495) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760706.118364:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972719 is committed 00002000:00010000:3.0:1789760706.118735:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22118400 -> 22122496 00010000:00010000:3.0:1789760706.118755:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2d4d2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22118400->22122495] (req 22118400->22122495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760706.118769:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2d4d2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22118400->22122495] (req 22118400->22122495) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760706.118779:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2d4d2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22118400->22122495] (req 22118400->22122495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760706.118787:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760706.118792:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2d4d2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22118400->22122495] (req 22118400->22122495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760706.118802:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2d4d2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22118400->22122495] (req 22118400->22122495) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760706.118829:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2d4d2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22118400->22122495] (req 22118400->22122495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760706.118839:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2d4d2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22118400->22122495] (req 22118400->22122495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760706.119011:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5307b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512549120:12345-192.168.203.56@tcp:4:dd.0 Request processed in 77356us (78728us total) trans 4294972719 rc 0/0 00000100:00100000:3.0:1789760706.119024:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5430 00000100:00100000:3.0:1789760706.119045:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 685506442274 time 685605931305 state 0x2 ref 0 00000100:00100000:3.0:1789760706.122795:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760706.123019:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512549248 00000100:00100000:2.0:1789760706.123041:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512549248 00000100:00100000:2.0:1789760706.123122:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 685506442274 time 685605931305 state 0x0 ref 1 00000100:00100000:2.0:1789760706.123140:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 685606191721 time 685605931305 state 0x1 ref 1 00000100:00100000:2.0:1789760706.123175:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 685606191721 time 685687474277 state 0x0 ref 1 00000100:00100000:2.0:1789760706.123190:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2745180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512549248:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760706.123257:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22122496, mtime: 0x6aad94c1, atime: 0x0, ctime: 0x6aad94c1, blocks: 0xb007 00010000:00010000:2.0:1789760706.123268:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2d4d9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.123297:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=685s v=(0 0) 00010000:00010000:2.0:1789760706.123302:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760706.123311:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2d4d9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22122496->22126591] (req 22122496->22126591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.123328:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2d4d9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22122496->22126591] (req 22122496->22126591) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760706.187464:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972720 is committed 00002000:00010000:0.0:1789760706.187595:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22122496 -> 22126592 00010000:00010000:0.0:1789760706.187624:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2d4d9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22122496->22126591] (req 22122496->22126591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760706.187639:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2d4d9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22122496->22126591] (req 22122496->22126591) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760706.187649:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2d4d9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22122496->22126591] (req 22122496->22126591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760706.187657:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760706.187663:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2d4d9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22122496->22126591] (req 22122496->22126591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760706.187673:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2d4d9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22122496->22126591] (req 22122496->22126591) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760706.187737:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2d4d9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22122496->22126591] (req 22122496->22126591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760706.187746:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2d4d9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22122496->22126591] (req 22122496->22126591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760706.187976:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2745180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512549248:12345-192.168.203.56@tcp:4:dd.0 Request processed in 64781us (65188us total) trans 4294972720 rc 0/0 00000100:00100000:0.0:1789760706.187988:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5431 00000100:00100000:0.0:1789760706.188010:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 685606191721 time 685687474277 state 0x2 ref 0 00000100:00100000:3.0:1789760706.193202:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760706.193294:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512549376 00000100:00100000:1.0:1789760706.193308:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512549376 00000100:00100000:1.0:1789760706.193389:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 685606191721 time 685687474277 state 0x0 ref 1 00000100:00100000:1.0:1789760706.193410:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 685687734693 time 685687474277 state 0x1 ref 1 00000100:00100000:1.0:1789760706.193446:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 685687734693 time 685757744585 state 0x0 ref 1 00000100:00100000:1.0:1789760706.193462:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2744a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512549376:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760706.193527:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22126592, mtime: 0x6aad94c1, atime: 0x0, ctime: 0x6aad94c1, blocks: 0xb007 00010000:00010000:1.0:1789760706.193539:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2d4e0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760706.193558:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=685s v=(0 0) 00010000:00010000:1.0:1789760706.193564:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760706.193571:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2d4e0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22126592->22130687] (req 22126592->22130687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760706.193588:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2d4e0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22126592->22130687] (req 22126592->22130687) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760706.339623:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22126592 -> 22130688 00010000:00010000:3.0:1789760706.339652:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2d4e0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22126592->22130687] (req 22126592->22130687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760706.339666:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2d4e0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22126592->22130687] (req 22126592->22130687) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760706.339675:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2d4e0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22126592->22130687] (req 22126592->22130687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760706.339682:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760706.339688:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2d4e0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22126592->22130687] (req 22126592->22130687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760706.339746:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2d4e0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22126592->22130687] (req 22126592->22130687) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760706.339772:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2d4e0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22126592->22130687] (req 22126592->22130687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760706.339782:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2d4e0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22126592->22130687] (req 22126592->22130687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760706.340209:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2744a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2359:x1876699512549376:12345-192.168.203.56@tcp:4:dd.0 Request processed in 146740us (147012us total) trans 4294972721 rc 0/0 00000100:00100000:3.0:1789760706.340223:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5432 00000100:00100000:3.0:1789760706.340235:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 685687734693 time 685757744585 state 0x2 ref 0 00000001:00080000:3.0:1789760706.340288:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972721 is committed 00000100:00100000:1.0:1789760706.358187:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760706.358472:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512549504 00000100:00100000:3.0:1789760706.358484:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512549504 00000100:00100000:3.0:1789760706.358561:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 685687734693 time 685757744585 state 0x0 ref 1 00000100:00100000:3.0:1789760706.358581:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 685758005001 time 685757744585 state 0x1 ref 1 00000100:00100000:3.0:1789760706.358629:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 685758005001 time 685922927839 state 0x0 ref 1 00000100:00100000:3.0:1789760706.358646:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febfb89ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512549504:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760706.358746:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22130688, mtime: 0x6aad94c1, atime: 0x0, ctime: 0x6aad94c1, blocks: 0xb007 00010000:00010000:3.0:1789760706.358757:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2d4e7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760706.358777:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=685s v=(0 0) 00010000:00010000:3.0:1789760706.358782:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760706.358790:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2d4e7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22130688->22134783] (req 22130688->22134783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760706.358807:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2d4e7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22130688->22134783] (req 22130688->22134783) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760706.507933:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972722 is committed 00002000:00010000:0.0:1789760706.508468:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22130688 -> 22134784 00010000:00010000:0.0:1789760706.508489:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2d4e7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22130688->22134783] (req 22130688->22134783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760706.508504:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2d4e7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22130688->22134783] (req 22130688->22134783) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760706.508514:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2d4e7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22130688->22134783] (req 22130688->22134783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760706.508523:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760706.508530:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2d4e7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22130688->22134783] (req 22130688->22134783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760706.508547:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2d4e7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22130688->22134783] (req 22130688->22134783) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760706.508577:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2d4e7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22130688->22134783] (req 22130688->22134783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760706.508586:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2d4e7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22130688->22134783] (req 22130688->22134783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760706.508877:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febfb89ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512549504:12345-192.168.203.56@tcp:4:dd.0 Request processed in 150227us (150698us total) trans 4294972722 rc 0/0 00000100:00100000:0.0:1789760706.508890:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5433 00000100:00100000:0.0:1789760706.508902:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 685758005001 time 685922927839 state 0x2 ref 0 00000100:00100000:2.0:1789760706.514508:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760706.514731:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512549632 00000100:00100000:0.0:1789760706.514746:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512549632 00000100:00100000:0.0:1789760706.514823:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 685758005001 time 685922927839 state 0x0 ref 1 00000100:00100000:0.0:1789760706.514842:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 685923188255 time 685922927839 state 0x1 ref 1 00000100:00100000:0.0:1789760706.514872:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 685923188255 time 686079172110 state 0x0 ref 1 00000100:00100000:0.0:1789760706.514887:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac38b4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512549632:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760706.514948:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22134784, mtime: 0x6aad94c1, atime: 0x0, ctime: 0x6aad94c1, blocks: 0xb007 00010000:00010000:0.0:1789760706.514959:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2d4ee lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760706.514978:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=686s v=(0 0) 00010000:00010000:0.0:1789760706.514984:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760706.514992:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2d4ee lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22134784->22138879] (req 22134784->22138879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760706.515009:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2d4ee lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22134784->22138879] (req 22134784->22138879) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760706.601963:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972723 is committed 00002000:00010000:0.0:1789760706.602212:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22134784 -> 22138880 00002000:00010000:0.0:1789760706.602217:0:6709:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760705 -> 1789760706 00002000:00010000:0.0:1789760706.602220:0:6709:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760705 -> 1789760706 00010000:00010000:0.0:1789760706.602236:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2d4ee lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22134784->22138879] (req 22134784->22138879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760706.602252:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2d4ee lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22134784->22138879] (req 22134784->22138879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760706.602262:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2d4ee lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22134784->22138879] (req 22134784->22138879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760706.602283:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760706.602290:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2d4ee lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22134784->22138879] (req 22134784->22138879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760706.602300:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2d4ee lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22134784->22138879] (req 22134784->22138879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760706.602342:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2d4ee lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22134784->22138879] (req 22134784->22138879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760706.602350:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2d4ee lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22134784->22138879] (req 22134784->22138879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760706.602474:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac38b4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512549632:12345-192.168.203.56@tcp:4:dd.0 Request processed in 87583us (87973us total) trans 4294972723 rc 0/0 00000100:00100000:0.0:1789760706.602486:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5434 00000100:00100000:0.0:1789760706.602497:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 685923188255 time 686079172110 state 0x2 ref 0 00000100:00100000:3.0:1789760706.609618:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760706.610152:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512549760 00000100:00100000:1.0:1789760706.610166:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512549760 00000100:00100000:1.0:1789760706.610247:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 685923188255 time 686079172110 state 0x0 ref 1 00000100:00100000:1.0:1789760706.610268:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 686079432526 time 686079172110 state 0x1 ref 1 00000100:00100000:1.0:1789760706.610323:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 686079432526 time 686174622194 state 0x0 ref 1 00000100:00100000:1.0:1789760706.610345:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2745500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512549760:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760706.610420:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22138880, mtime: 0x6aad94c2, atime: 0x0, ctime: 0x6aad94c2, blocks: 0xb007 00010000:00010000:1.0:1789760706.610432:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2d4f5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760706.610452:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=686s v=(0 0) 00010000:00010000:1.0:1789760706.610458:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760706.610466:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2d4f5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22138880->22142975] (req 22138880->22142975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760706.610484:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2d4f5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22138880->22142975] (req 22138880->22142975) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760706.713245:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972724 is committed 00002000:00010000:2.0:1789760706.713770:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22138880 -> 22142976 00010000:00010000:2.0:1789760706.713792:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2d4f5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22138880->22142975] (req 22138880->22142975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.713808:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2d4f5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22138880->22142975] (req 22138880->22142975) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.713818:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2d4f5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22138880->22142975] (req 22138880->22142975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.713825:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760706.713831:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2d4f5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22138880->22142975] (req 22138880->22142975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.713840:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2d4f5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22138880->22142975] (req 22138880->22142975) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.713865:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2d4f5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22138880->22142975] (req 22138880->22142975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.713885:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2d4f5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22138880->22142975] (req 22138880->22142975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760706.714228:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2745500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512549760:12345-192.168.203.56@tcp:4:dd.0 Request processed in 103877us (104626us total) trans 4294972724 rc 0/0 00000100:00100000:2.0:1789760706.714243:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5435 00000100:00100000:2.0:1789760706.714255:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 686079432526 time 686174622194 state 0x2 ref 0 00000100:00100000:1.0:1789760706.732015:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760706.732218:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512549888 00000100:00100000:3.0:1789760706.732230:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512549888 00000100:00100000:3.0:1789760706.732323:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 686079432526 time 686174622194 state 0x0 ref 1 00000100:00100000:3.0:1789760706.732476:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 686174882610 time 686174622194 state 0x1 ref 1 00000100:00100000:3.0:1789760706.732516:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 686174882610 time 686296814878 state 0x0 ref 1 00000100:00100000:3.0:1789760706.732533:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febfb89fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512549888:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760706.732612:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22142976, mtime: 0x6aad94c2, atime: 0x0, ctime: 0x6aad94c2, blocks: 0xb007 00010000:00010000:3.0:1789760706.732626:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2d4fc lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760706.732653:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=686s v=(0 0) 00010000:00010000:3.0:1789760706.732659:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760706.732668:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2d4fc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22142976->22147071] (req 22142976->22147071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760706.732686:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2d4fc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22142976->22147071] (req 22142976->22147071) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760706.858099:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22142976 -> 22147072 00010000:00010000:3.0:1789760706.858127:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2d4fc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22142976->22147071] (req 22142976->22147071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760706.858143:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2d4fc lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22142976->22147071] (req 22142976->22147071) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760706.858153:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2d4fc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22142976->22147071] (req 22142976->22147071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760706.858161:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760706.858167:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2d4fc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22142976->22147071] (req 22142976->22147071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760706.858177:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2d4fc lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22142976->22147071] (req 22142976->22147071) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760706.858201:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2d4fc lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22142976->22147071] (req 22142976->22147071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760706.858211:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2d4fc lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22142976->22147071] (req 22142976->22147071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760706.858373:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febfb89fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2358:x1876699512549888:12345-192.168.203.56@tcp:4:dd.0 Request processed in 125836us (126365us total) trans 4294972725 rc 0/0 00000100:00100000:3.0:1789760706.858386:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5436 00000100:00100000:3.0:1789760706.858400:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 686174882610 time 686296814878 state 0x2 ref 0 00000100:00100000:2.0:1789760706.868267:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760706.868492:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512550016 00000100:00100000:0.0:1789760706.868505:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512550016 00000100:00100000:0.0:1789760706.868579:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 686174882610 time 686296814878 state 0x0 ref 1 00000100:00100000:0.0:1789760706.868611:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 686297075294 time 686296814878 state 0x1 ref 1 00000100:00100000:0.0:1789760706.868648:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 686297075294 time 686432947469 state 0x0 ref 1 00000100:00100000:0.0:1789760706.868664:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac38b5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2361:x1876699512550016:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760706.868770:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22147072, mtime: 0x6aad94c2, atime: 0x0, ctime: 0x6aad94c2, blocks: 0xb007 00010000:00010000:0.0:1789760706.868781:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2d503 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760706.868801:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=686s v=(0 0) 00010000:00010000:0.0:1789760706.868808:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760706.868816:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2d503 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22147072->22151167] (req 22147072->22151167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760706.868832:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2d503 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22147072->22151167] (req 22147072->22151167) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760706.877911:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972725 is committed 00000001:00080000:1.0:1789760706.931837:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972726 is committed 00010000:00010000:0.0:1789760706.932157:0:7647:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:0.0:1789760706.932373:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22147072 -> 22151168 00010000:00010000:0.0:1789760706.932393:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2d503 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22147072->22151167] (req 22147072->22151167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760706.932409:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2d503 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22147072->22151167] (req 22147072->22151167) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760706.932418:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2d503 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22147072->22151167] (req 22147072->22151167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760706.932426:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760706.932432:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2d503 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22147072->22151167] (req 22147072->22151167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760706.932443:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2d503 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22147072->22151167] (req 22147072->22151167) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760706.932465:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2d503 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22147072->22151167] (req 22147072->22151167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760706.932479:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2d503 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22147072->22151167] (req 22147072->22151167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760706.932773:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac38b5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512550016:12345-192.168.203.56@tcp:4:dd.0 Request processed in 64105us (64516us total) trans 4294972726 rc 0/0 00000100:00100000:0.0:1789760706.932786:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5437 00000100:00100000:0.0:1789760706.932799:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 686297075294 time 686432947469 state 0x2 ref 0 00000100:00100000:0.0:1789760706.938529:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760706.939901:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512550144 00000100:00100000:2.0:1789760706.939919:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512550144 00000100:00100000:2.0:1789760706.940004:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 686297075294 time 686432947469 state 0x0 ref 1 00000100:00100000:2.0:1789760706.940024:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 686433207885 time 686432947469 state 0x1 ref 1 00000100:00100000:3.0:1789760706.940142:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 686433207885 time 686504438539 state 0x0 ref 1 00000100:00100000:3.0:1789760706.940160:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac405ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512550144:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760706.940227:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22151168, mtime: 0x6aad94c2, atime: 0x0, ctime: 0x6aad94c2, blocks: 0xb007 00010000:00010000:3.0:1789760706.940239:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2d50a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760706.940260:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=686s v=(0 0) 00010000:00010000:3.0:1789760706.940274:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760706.940283:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2d50a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22151168->22155263] (req 22151168->22155263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760706.940302:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2d50a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22151168->22155263] (req 22151168->22155263) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760706.998191:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972727 is committed 00002000:00010000:3.0:1789760706.998834:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22151168 -> 22155264 00010000:00010000:3.0:1789760706.998856:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2d50a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22151168->22155263] (req 22151168->22155263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760706.998870:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2d50a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22151168->22155263] (req 22151168->22155263) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760706.998880:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2d50a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22151168->22155263] (req 22151168->22155263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760706.998888:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760706.998894:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2d50a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22151168->22155263] (req 22151168->22155263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760706.998903:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2d50a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22151168->22155263] (req 22151168->22155263) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760706.998925:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2d50a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22151168->22155263] (req 22151168->22155263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760706.998933:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2d50a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22151168->22155263] (req 22151168->22155263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760706.999123:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac405ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512550144:12345-192.168.203.56@tcp:4:dd.0 Request processed in 58961us (60601us total) trans 4294972727 rc 0/0 00000100:00100000:3.0:1789760706.999136:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5438 00000100:00100000:3.0:1789760706.999148:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 686433207885 time 686504438539 state 0x2 ref 0 00000100:00100000:0.0:1789760707.010226:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760707.010380:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512550272 00000100:00100000:3.0:1789760707.010394:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512550272 00000100:00100000:3.0:1789760707.010489:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 686433207885 time 686504438539 state 0x0 ref 1 00000100:00100000:3.0:1789760707.010509:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 686504698955 time 686504438539 state 0x1 ref 1 00000100:00100000:3.0:1789760707.010542:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 686504698955 time 686574841704 state 0x0 ref 1 00000100:00100000:3.0:1789760707.010559:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4058e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512550272:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760707.010632:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22155264, mtime: 0x6aad94c2, atime: 0x0, ctime: 0x6aad94c2, blocks: 0xb007 00010000:00010000:3.0:1789760707.010642:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2d511 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760707.010660:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=686s v=(0 0) 00010000:00010000:3.0:1789760707.010665:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760707.010672:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2d511 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22155264->22159359] (req 22155264->22159359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760707.010688:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2d511 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22155264->22159359] (req 22155264->22159359) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1789760707.011364:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:3.0:1789760707.123394:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972728 is committed 00002000:00010000:0.0:1789760707.123634:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22155264 -> 22159360 00010000:00010000:0.0:1789760707.123655:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2d511 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22155264->22159359] (req 22155264->22159359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760707.123670:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2d511 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22155264->22159359] (req 22155264->22159359) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760707.123680:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2d511 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22155264->22159359] (req 22155264->22159359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760707.123689:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760707.123722:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2d511 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22155264->22159359] (req 22155264->22159359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760707.123739:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2d511 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22155264->22159359] (req 22155264->22159359) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760707.123764:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2d511 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22155264->22159359] (req 22155264->22159359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760707.123773:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2d511 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22155264->22159359] (req 22155264->22159359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760707.123897:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4058e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512550272:12345-192.168.203.56@tcp:4:dd.0 Request processed in 113336us (113681us total) trans 4294972728 rc 0/0 00000100:00100000:0.0:1789760707.123910:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5439 00000100:00100000:0.0:1789760707.123921:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 686504698955 time 686574841704 state 0x2 ref 0 00000100:00100000:2.0:1789760707.144353:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760707.144497:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512550400 00000100:00100000:0.0:1789760707.144511:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512550400 00000100:00100000:0.0:1789760707.144587:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 686504698955 time 686574841704 state 0x0 ref 1 00000100:00100000:0.0:1789760707.144615:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 686575102120 time 686574841704 state 0x1 ref 1 00000100:00100000:0.0:1789760707.144649:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 686575102120 time 686708948269 state 0x0 ref 1 00000100:00100000:0.0:1789760707.144662:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac38b6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512550400:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760707.144758:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22159360, mtime: 0x6aad94c2, atime: 0x0, ctime: 0x6aad94c2, blocks: 0xb007 00010000:00010000:0.0:1789760707.144770:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2d518 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760707.144791:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=686s v=(0 0) 00010000:00010000:0.0:1789760707.144796:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760707.144804:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2d518 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22159360->22163455] (req 22159360->22163455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760707.144820:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2d518 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22159360->22163455] (req 22159360->22163455) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760707.280346:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972729 is committed 00002000:00010000:0.0:1789760707.280556:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22159360 -> 22163456 00010000:00010000:0.0:1789760707.280577:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2d518 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22159360->22163455] (req 22159360->22163455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760707.280604:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2d518 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22159360->22163455] (req 22159360->22163455) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760707.280614:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2d518 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22159360->22163455] (req 22159360->22163455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760707.280622:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760707.280628:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2d518 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22159360->22163455] (req 22159360->22163455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760707.280638:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2d518 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22159360->22163455] (req 22159360->22163455) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760707.280663:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2d518 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22159360->22163455] (req 22159360->22163455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760707.280672:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2d518 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22159360->22163455] (req 22159360->22163455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760707.280826:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac38b6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512550400:12345-192.168.203.56@tcp:4:dd.0 Request processed in 136159us (136482us total) trans 4294972729 rc 0/0 00000100:00100000:0.0:1789760707.280839:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5440 00000100:00100000:0.0:1789760707.280850:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 686575102120 time 686708948269 state 0x2 ref 0 00000100:00100000:3.0:1789760707.287787:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760707.287903:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512550528 00000100:00100000:0.0:1789760707.287916:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512550528 00000100:00100000:0.0:1789760707.287990:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 686575102120 time 686708948269 state 0x0 ref 1 00000100:00100000:0.0:1789760707.288010:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 686709208685 time 686708948269 state 0x1 ref 1 00000100:00100000:0.0:1789760707.288048:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 686709208685 time 686852347238 state 0x0 ref 1 00000100:00100000:0.0:1789760707.288063:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febea4b9050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512550528:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760707.288128:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22163456, mtime: 0x6aad94c2, atime: 0x0, ctime: 0x6aad94c2, blocks: 0xb007 00010000:00010000:0.0:1789760707.288138:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2d51f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760707.288159:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=686s v=(0 0) 00010000:00010000:0.0:1789760707.288165:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760707.288173:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2d51f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22163456->22167551] (req 22163456->22167551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760707.288196:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2d51f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22163456->22167551] (req 22163456->22167551) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760707.343833:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972730 is committed 00002000:00010000:2.0:1789760707.343971:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22163456 -> 22167552 00010000:00010000:2.0:1789760707.343992:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2d51f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22163456->22167551] (req 22163456->22167551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760707.344008:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2d51f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22163456->22167551] (req 22163456->22167551) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760707.344019:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2d51f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22163456->22167551] (req 22163456->22167551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760707.344027:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760707.344033:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2d51f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22163456->22167551] (req 22163456->22167551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760707.344043:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2d51f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22163456->22167551] (req 22163456->22167551) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760707.344073:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2d51f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22163456->22167551] (req 22163456->22167551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760707.344082:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2d51f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22163456->22167551] (req 22163456->22167551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760707.344197:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febea4b9050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512550528:12345-192.168.203.56@tcp:4:dd.0 Request processed in 56130us (56416us total) trans 4294972730 rc 0/0 00000100:00100000:2.0:1789760707.344210:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5441 00000100:00100000:2.0:1789760707.344221:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 686709208685 time 686852347238 state 0x2 ref 0 00000100:00100000:0.0:1789760707.352000:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760707.352092:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512550656 00000100:00100000:2.0:1789760707.352173:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512550656 00000100:00100000:2.0:1789760707.352249:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 686709208685 time 686852347238 state 0x0 ref 1 00000100:00100000:2.0:1789760707.352278:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 686852607654 time 686852347238 state 0x1 ref 1 00000100:00100000:2.0:1789760707.352316:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 686852607654 time 686916615401 state 0x0 ref 1 00000100:00100000:2.0:1789760707.352331:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4058380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512550656:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760707.352400:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22167552, mtime: 0x6aad94c2, atime: 0x0, ctime: 0x6aad94c2, blocks: 0xb007 00010000:00010000:2.0:1789760707.352411:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2d526 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760707.352432:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=686s v=(0 0) 00010000:00010000:2.0:1789760707.352438:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760707.352446:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2d526 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22167552->22171647] (req 22167552->22171647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760707.352464:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2d526 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22167552->22171647] (req 22167552->22171647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760707.438458:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22167552 -> 22171648 00010000:00010000:0.0:1789760707.438485:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2d526 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22167552->22171647] (req 22167552->22171647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760707.438501:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2d526 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22167552->22171647] (req 22167552->22171647) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760707.438512:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2d526 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22167552->22171647] (req 22167552->22171647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760707.438520:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760707.438525:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2d526 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22167552->22171647] (req 22167552->22171647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760707.438536:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2d526 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22167552->22171647] (req 22167552->22171647) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760707.438564:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2d526 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22167552->22171647] (req 22167552->22171647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760707.438573:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2d526 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22167552->22171647] (req 22167552->22171647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760707.438852:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4058380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2360:x1876699512550656:12345-192.168.203.56@tcp:4:dd.0 Request processed in 86517us (86855us total) trans 4294972731 rc 0/0 00000100:00100000:0.0:1789760707.438866:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5442 00000100:00100000:0.0:1789760707.438878:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 686852607654 time 686916615401 state 0x2 ref 0 00000001:00080000:1.0:1789760707.439514:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972731 is committed 00000100:00100000:0.0:1789760707.485389:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760707.485754:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512550784 00000100:00100000:1.0:1789760707.485769:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512550784 00000100:00100000:1.0:1789760707.485909:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 686852607654 time 686916615401 state 0x0 ref 1 00000100:00100000:1.0:1789760707.485928:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 686916875817 time 686916615401 state 0x1 ref 1 00000100:00100000:1.0:1789760707.486012:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 686916875817 time 687050310701 state 0x0 ref 1 00000100:00100000:1.0:1789760707.486028:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4059180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512550784:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760707.486096:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22171648, mtime: 0x6aad94c2, atime: 0x0, ctime: 0x6aad94c2, blocks: 0xb007 00010000:00010000:1.0:1789760707.488394:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2d52d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760707.488425:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=687s v=(0 0) 00010000:00010000:1.0:1789760707.488431:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760707.488439:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2d52d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22171648->22175743] (req 22171648->22175743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760707.488459:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2d52d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22171648->22175743] (req 22171648->22175743) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760707.573511:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972732 is committed 00002000:00010000:2.0:1789760707.573678:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22171648 -> 22175744 00010000:00010000:2.0:1789760707.573732:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2d52d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22171648->22175743] (req 22171648->22175743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760707.573748:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2d52d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22171648->22175743] (req 22171648->22175743) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760707.573760:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2d52d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22171648->22175743] (req 22171648->22175743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760707.573768:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760707.573775:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2d52d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22171648->22175743] (req 22171648->22175743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760707.573884:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2d52d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22171648->22175743] (req 22171648->22175743) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760707.573911:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2d52d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22171648->22175743] (req 22171648->22175743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760707.573920:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2d52d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22171648->22175743] (req 22171648->22175743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760707.574045:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4059180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512550784:12345-192.168.203.56@tcp:4:dd.0 Request processed in 88013us (88664us total) trans 4294972732 rc 0/0 00000100:00100000:2.0:1789760707.574058:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5443 00000100:00100000:2.0:1789760707.574070:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 686916875817 time 687050310701 state 0x2 ref 0 00000100:00100000:3.0:1789760707.582991:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760707.588288:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512550912 00000100:00100000:2.0:1789760707.588313:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512550912 00000100:00100000:0.0:1789760707.588514:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 686916875817 time 687050310701 state 0x0 ref 1 00000100:00100000:0.0:1789760707.588535:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 687050571117 time 687050310701 state 0x1 ref 1 00000100:00100000:0.0:1789760707.588576:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 687050571117 time 687152874439 state 0x0 ref 1 00000100:00100000:0.0:1789760707.588604:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2746d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512550912:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760707.588671:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22175744, mtime: 0x6aad94c2, atime: 0x0, ctime: 0x6aad94c2, blocks: 0xb007 00010000:00010000:0.0:1789760707.588681:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2d534 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760707.588868:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=687s v=(0 0) 00010000:00010000:0.0:1789760707.588875:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760707.588883:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2d534 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22175744->22179839] (req 22175744->22179839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760707.588901:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2d534 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22175744->22179839] (req 22175744->22179839) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760707.695727:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972733 is committed 00002000:00010000:3.0:1789760707.695819:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22175744 -> 22179840 00002000:00010000:3.0:1789760707.695824:0:6708:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760706 -> 1789760707 00002000:00010000:3.0:1789760707.695827:0:6708:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760706 -> 1789760707 00010000:00010000:3.0:1789760707.695844:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2d534 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22175744->22179839] (req 22175744->22179839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760707.695858:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2d534 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22175744->22179839] (req 22175744->22179839) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760707.695868:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2d534 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22175744->22179839] (req 22175744->22179839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760707.695877:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760707.695883:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2d534 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22175744->22179839] (req 22175744->22179839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760707.695893:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2d534 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22175744->22179839] (req 22175744->22179839) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760707.695917:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2d534 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22175744->22179839] (req 22175744->22179839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760707.695925:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2d534 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22175744->22179839] (req 22175744->22179839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760707.696049:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2746d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512550912:12345-192.168.203.56@tcp:4:dd.0 Request processed in 107454us (113066us total) trans 4294972733 rc 0/0 00000100:00100000:3.0:1789760707.696065:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5444 00000100:00100000:3.0:1789760707.696077:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 687050571117 time 687152874439 state 0x2 ref 0 00000100:00100000:0.0:1789760707.702936:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760707.703058:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512551040 00000100:00100000:2.0:1789760707.703072:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512551040 00000100:00100000:2.0:1789760707.703147:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 687050571117 time 687152874439 state 0x0 ref 1 00000100:00100000:2.0:1789760707.703166:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 687153134855 time 687152874439 state 0x1 ref 1 00000100:00100000:2.0:1789760707.703196:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 687153134855 time 687267495675 state 0x0 ref 1 00000100:00100000:2.0:1789760707.703211:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4059500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512551040:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760707.703291:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22179840, mtime: 0x6aad94c3, atime: 0x0, ctime: 0x6aad94c3, blocks: 0xb007 00010000:00010000:2.0:1789760707.703302:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2d53b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760707.703320:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=687s v=(0 0) 00010000:00010000:2.0:1789760707.703326:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760707.703334:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2d53b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22179840->22183935] (req 22179840->22183935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760707.703351:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2d53b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22179840->22183935] (req 22179840->22183935) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760707.771823:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972734 is committed 00002000:00010000:0.0:1789760707.772058:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22179840 -> 22183936 00010000:00010000:0.0:1789760707.772080:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2d53b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22179840->22183935] (req 22179840->22183935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760707.772097:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2d53b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22179840->22183935] (req 22179840->22183935) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760707.772107:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2d53b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22179840->22183935] (req 22179840->22183935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760707.772116:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760707.772122:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2d53b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22179840->22183935] (req 22179840->22183935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760707.772132:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2d53b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22179840->22183935] (req 22179840->22183935) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760707.772155:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2d53b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22179840->22183935] (req 22179840->22183935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760707.772164:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2d53b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22179840->22183935] (req 22179840->22183935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760707.774265:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4059500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512551040:12345-192.168.203.56@tcp:4:dd.0 Request processed in 71036us (71323us total) trans 4294972734 rc 0/0 00000100:00100000:0.0:1789760707.774288:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5445 00000100:00100000:0.0:1789760707.774304:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 687153134855 time 687267495675 state 0x2 ref 0 00000100:00100000:0.0:1789760707.787835:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760707.787949:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512551168 00000100:00100000:2.0:1789760707.787963:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512551168 00000100:00100000:2.0:1789760707.788038:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 687153134855 time 687267495675 state 0x0 ref 1 00000100:00100000:2.0:1789760707.788057:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 687267756091 time 687267495675 state 0x1 ref 1 00000100:00100000:2.0:1789760707.788106:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 687267756091 time 687352398440 state 0x0 ref 1 00000100:00100000:2.0:1789760707.788122:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4058a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512551168:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760707.788197:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22183936, mtime: 0x6aad94c3, atime: 0x0, ctime: 0x6aad94c3, blocks: 0xb007 00010000:00010000:2.0:1789760707.788209:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2d542 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760707.788228:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=687s v=(0 0) 00010000:00010000:2.0:1789760707.788233:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760707.788241:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2d542 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22183936->22188031] (req 22183936->22188031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760707.788256:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2d542 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22183936->22188031] (req 22183936->22188031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760707.840962:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972735 is committed 00002000:00010000:2.0:1789760707.841160:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22183936 -> 22188032 00010000:00010000:2.0:1789760707.841180:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2d542 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22183936->22188031] (req 22183936->22188031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760707.841201:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2d542 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22183936->22188031] (req 22183936->22188031) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760707.841211:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2d542 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22183936->22188031] (req 22183936->22188031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760707.841219:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760707.841226:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2d542 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22183936->22188031] (req 22183936->22188031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760707.841236:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2d542 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22183936->22188031] (req 22183936->22188031) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760707.841271:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2d542 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22183936->22188031] (req 22183936->22188031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760707.841281:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2d542 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22183936->22188031] (req 22183936->22188031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760707.841400:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4058a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512551168:12345-192.168.203.56@tcp:4:dd.0 Request processed in 53274us (53572us total) trans 4294972735 rc 0/0 00000100:00100000:2.0:1789760707.841414:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5446 00000100:00100000:2.0:1789760707.841426:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 687267756091 time 687352398440 state 0x2 ref 0 00000100:00100000:0.0:1789760707.845015:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760707.845833:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512551296 00000100:00100000:3.0:1789760707.845847:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512551296 00000100:00100000:3.0:1789760707.845920:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 687267756091 time 687352398440 state 0x0 ref 1 00000100:00100000:3.0:1789760707.845938:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 687352658856 time 687352398440 state 0x1 ref 1 00000100:00100000:3.0:1789760707.845970:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 687352658856 time 687410269997 state 0x0 ref 1 00000100:00100000:3.0:1789760707.845992:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac405ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512551296:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760707.846060:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22188032, mtime: 0x6aad94c3, atime: 0x0, ctime: 0x6aad94c3, blocks: 0xb007 00010000:00010000:3.0:1789760707.846071:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2d549 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760707.846092:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=687s v=(0 0) 00010000:00010000:3.0:1789760707.846097:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760707.846105:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2d549 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22188032->22192127] (req 22188032->22192127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760707.846122:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2d549 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22188032->22192127] (req 22188032->22192127) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760707.902765:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22188032 -> 22192128 00010000:00010000:2.0:1789760707.902793:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2d549 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22188032->22192127] (req 22188032->22192127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760707.902809:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2d549 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22188032->22192127] (req 22188032->22192127) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760707.902820:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2d549 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22188032->22192127] (req 22188032->22192127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760707.902829:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760707.902835:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2d549 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22188032->22192127] (req 22188032->22192127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760707.902846:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2d549 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22188032->22192127] (req 22188032->22192127) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760707.902869:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2d549 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22188032->22192127] (req 22188032->22192127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760707.902878:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2d549 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22188032->22192127] (req 22188032->22192127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760707.903017:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac405ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2361:x1876699512551296:12345-192.168.203.56@tcp:4:dd.0 Request processed in 57021us (58009us total) trans 4294972736 rc 0/0 00000100:00100000:2.0:1789760707.903030:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5447 00000100:00100000:2.0:1789760707.903042:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 687352658856 time 687410269997 state 0x2 ref 0 00000001:00080000:0.0:1789760707.907981:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972736 is committed 00000100:00100000:0.0:1789760707.914429:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760707.917055:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512551424 00000100:00100000:2.0:1789760707.917076:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512551424 00000100:00100000:2.0:1789760707.917162:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 687352658856 time 687410269997 state 0x0 ref 1 00000100:00100000:2.0:1789760707.917183:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 687410530413 time 687410269997 state 0x1 ref 1 00000100:00100000:2.0:1789760707.917220:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 687410530413 time 687481519683 state 0x0 ref 1 00000100:00100000:2.0:1789760707.917235:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4058e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512551424:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760707.917311:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22192128, mtime: 0x6aad94c3, atime: 0x0, ctime: 0x6aad94c3, blocks: 0xb007 00010000:00010000:2.0:1789760707.917321:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2d550 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760707.917341:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=687s v=(0 0) 00010000:00010000:2.0:1789760707.917346:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760707.917353:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2d550 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22192128->22196223] (req 22192128->22196223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760707.917368:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2d550 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22192128->22196223] (req 22192128->22196223) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760707.964293:0:5836:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:3.0:1789760707.995818:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22192128 -> 22196224 00010000:00010000:3.0:1789760707.995845:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2d550 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22192128->22196223] (req 22192128->22196223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760707.995862:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2d550 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22192128->22196223] (req 22192128->22196223) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760707.995872:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2d550 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22192128->22196223] (req 22192128->22196223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760707.995880:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760707.995886:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2d550 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22192128->22196223] (req 22192128->22196223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760707.995895:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2d550 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22192128->22196223] (req 22192128->22196223) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760707.995919:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2d550 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22192128->22196223] (req 22192128->22196223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760707.995926:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2d550 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22192128->22196223] (req 22192128->22196223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760707.996051:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4058e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2358:x1876699512551424:12345-192.168.203.56@tcp:4:dd.0 Request processed in 78813us (81631us total) trans 4294972737 rc 0/0 00000100:00100000:3.0:1789760707.996063:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5448 00000100:00100000:3.0:1789760707.996075:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 687410530413 time 687481519683 state 0x2 ref 0 00000001:00080000:0.0:1789760707.996249:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972737 is committed 00000100:00100000:1.0:1789760708.000277:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760708.000431:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512551552 00000100:00100000:2.0:1789760708.000447:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512551552 00000100:00100000:2.0:1789760708.000532:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 687410530413 time 687481519683 state 0x0 ref 1 00000100:00100000:2.0:1789760708.000551:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 687481780099 time 687481519683 state 0x1 ref 1 00000100:00100000:2.0:1789760708.000598:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 687481780099 time 687564897031 state 0x0 ref 1 00000100:00100000:2.0:1789760708.000611:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febfb89ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512551552:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760708.000670:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22196224, mtime: 0x6aad94c3, atime: 0x0, ctime: 0x6aad94c3, blocks: 0xb007 00010000:00010000:2.0:1789760708.000681:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2d557 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.000724:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=687s v=(0 0) 00010000:00010000:2.0:1789760708.000730:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760708.000738:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2d557 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22196224->22200319] (req 22196224->22200319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.000754:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2d557 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22196224->22200319] (req 22196224->22200319) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760708.062964:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972738 is committed 00002000:00010000:3.0:1789760708.063430:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22196224 -> 22200320 00010000:00010000:3.0:1789760708.063451:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2d557 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22196224->22200319] (req 22196224->22200319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760708.063465:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2d557 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22196224->22200319] (req 22196224->22200319) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760708.063486:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2d557 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22196224->22200319] (req 22196224->22200319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760708.063495:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760708.063500:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2d557 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22196224->22200319] (req 22196224->22200319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760708.063510:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2d557 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22196224->22200319] (req 22196224->22200319) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760708.063534:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2d557 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22196224->22200319] (req 22196224->22200319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760708.063541:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2d557 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22196224->22200319] (req 22196224->22200319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760708.063683:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febfb89ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512551552:12345-192.168.203.56@tcp:4:dd.0 Request processed in 63064us (63423us total) trans 4294972738 rc 0/0 00000100:00100000:3.0:1789760708.063744:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5449 00000100:00100000:3.0:1789760708.063757:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 687481780099 time 687564897031 state 0x2 ref 0 00000100:00100000:2.0:1789760708.079220:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760708.079521:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512551680 00000100:00100000:3.0:1789760708.079536:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512551680 00000100:00100000:3.0:1789760708.079621:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 687481780099 time 687564897031 state 0x0 ref 1 00000100:00100000:3.0:1789760708.079640:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 687565157447 time 687564897031 state 0x1 ref 1 00000100:00100000:3.0:1789760708.079675:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 687565157447 time 687643974214 state 0x0 ref 1 00000100:00100000:3.0:1789760708.079689:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac38b7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512551680:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760708.079805:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22200320, mtime: 0x6aad94c3, atime: 0x0, ctime: 0x6aad94c3, blocks: 0xb007 00010000:00010000:3.0:1789760708.079816:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2d55e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760708.079836:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=687s v=(0 0) 00010000:00010000:3.0:1789760708.079842:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760708.079850:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2d55e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22200320->22204415] (req 22200320->22204415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760708.079866:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2d55e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22200320->22204415] (req 22200320->22204415) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760708.171635:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972739 is committed 00002000:00010000:2.0:1789760708.171783:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22200320 -> 22204416 00010000:00010000:2.0:1789760708.171803:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2d55e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22200320->22204415] (req 22200320->22204415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.171818:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2d55e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22200320->22204415] (req 22200320->22204415) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.171828:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2d55e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22200320->22204415] (req 22200320->22204415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.171843:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760708.171849:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2d55e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22200320->22204415] (req 22200320->22204415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.171859:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2d55e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22200320->22204415] (req 22200320->22204415) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.171881:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2d55e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22200320->22204415] (req 22200320->22204415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.171888:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2d55e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22200320->22204415] (req 22200320->22204415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760708.172108:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac38b7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512551680:12345-192.168.203.56@tcp:4:dd.0 Request processed in 92408us (92891us total) trans 4294972739 rc 0/0 00000100:00100000:2.0:1789760708.172120:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5450 00000100:00100000:2.0:1789760708.172132:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 687565157447 time 687643974214 state 0x2 ref 0 00000100:00100000:1.0:1789760708.176418:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760708.176608:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512551808 00000100:00100000:3.0:1789760708.176620:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512551808 00000100:00100000:3.0:1789760708.176691:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 687565157447 time 687643974214 state 0x0 ref 1 00000100:00100000:3.0:1789760708.176876:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 687644234630 time 687643974214 state 0x1 ref 1 00000100:00100000:3.0:1789760708.177012:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 687644234630 time 687741309259 state 0x0 ref 1 00000100:00100000:3.0:1789760708.177028:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febfb89fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512551808:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760708.177095:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22204416, mtime: 0x6aad94c3, atime: 0x0, ctime: 0x6aad94c3, blocks: 0xb007 00010000:00010000:3.0:1789760708.177220:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2d565 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760708.177239:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=687s v=(0 0) 00010000:00010000:3.0:1789760708.177245:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760708.177252:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2d565 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22204416->22208511] (req 22204416->22208511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760708.177278:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2d565 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22204416->22208511] (req 22204416->22208511) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760708.261971:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972740 is committed 00002000:00010000:0.0:1789760708.262104:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22204416 -> 22208512 00010000:00010000:0.0:1789760708.262126:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2d565 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22204416->22208511] (req 22204416->22208511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.262141:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2d565 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22204416->22208511] (req 22204416->22208511) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.262151:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2d565 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22204416->22208511] (req 22204416->22208511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.262160:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760708.262166:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2d565 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22204416->22208511] (req 22204416->22208511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.262177:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2d565 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22204416->22208511] (req 22204416->22208511) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.262201:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2d565 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22204416->22208511] (req 22204416->22208511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.262210:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2d565 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22204416->22208511] (req 22204416->22208511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760708.262340:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febfb89fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512551808:12345-192.168.203.56@tcp:4:dd.0 Request processed in 85309us (85930us total) trans 4294972740 rc 0/0 00000100:00100000:0.0:1789760708.262352:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5451 00000100:00100000:0.0:1789760708.262364:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 687644234630 time 687741309259 state 0x2 ref 0 00000100:00100000:1.0:1789760708.274902:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760708.275005:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512551936 00000100:00100000:3.0:1789760708.275018:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512551936 00000100:00100000:3.0:1789760708.275090:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 687644234630 time 687741309259 state 0x0 ref 1 00000100:00100000:3.0:1789760708.275110:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 687741569675 time 687741309259 state 0x1 ref 1 00000100:00100000:3.0:1789760708.275141:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 687741569675 time 687839440665 state 0x0 ref 1 00000100:00100000:3.0:1789760708.275155:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac28a1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512551936:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760708.275217:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22208512, mtime: 0x6aad94c3, atime: 0x0, ctime: 0x6aad94c3, blocks: 0xb007 00010000:00010000:3.0:1789760708.275227:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2d56c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760708.275246:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=687s v=(0 0) 00010000:00010000:3.0:1789760708.275251:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760708.275270:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2d56c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22208512->22212607] (req 22208512->22212607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760708.275288:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2d56c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22208512->22212607] (req 22208512->22212607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760708.349904:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972741 is committed 00002000:00010000:0.0:1789760708.350160:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22208512 -> 22212608 00010000:00010000:0.0:1789760708.350180:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2d56c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22208512->22212607] (req 22208512->22212607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.350196:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2d56c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22208512->22212607] (req 22208512->22212607) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.350207:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2d56c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22208512->22212607] (req 22208512->22212607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.350214:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760708.350221:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2d56c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22208512->22212607] (req 22208512->22212607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.350232:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2d56c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22208512->22212607] (req 22208512->22212607) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.350378:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2d56c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22208512->22212607] (req 22208512->22212607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.350388:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2d56c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22208512->22212607] (req 22208512->22212607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760708.350759:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac28a1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512551936:12345-192.168.203.56@tcp:4:dd.0 Request processed in 75597us (75863us total) trans 4294972741 rc 0/0 00000100:00100000:0.0:1789760708.350771:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5452 00000100:00100000:0.0:1789760708.350784:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 687741569675 time 687839440665 state 0x2 ref 0 00000100:00100000:3.0:1789760708.368394:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760708.368744:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512552064 00000100:00100000:2.0:1789760708.368757:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512552064 00000100:00100000:2.0:1789760708.368830:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 687741569675 time 687839440665 state 0x0 ref 1 00000100:00100000:2.0:1789760708.368849:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 687839701081 time 687839440665 state 0x1 ref 1 00000100:00100000:2.0:1789760708.368887:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 687839701081 time 687933186389 state 0x0 ref 1 00000100:00100000:2.0:1789760708.368901:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2744380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512552064:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760708.368967:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22212608, mtime: 0x6aad94c3, atime: 0x0, ctime: 0x6aad94c3, blocks: 0xb007 00010000:00010000:2.0:1789760708.368977:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2d573 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.368997:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=687s v=(0 0) 00010000:00010000:2.0:1789760708.369002:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760708.369010:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2d573 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22212608->22216703] (req 22212608->22216703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.371007:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2d573 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22212608->22216703] (req 22212608->22216703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760708.490986:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972742 is committed 00002000:00010000:0.0:1789760708.491197:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22212608 -> 22216704 00010000:00010000:0.0:1789760708.491217:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2d573 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22212608->22216703] (req 22212608->22216703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.491232:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2d573 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22212608->22216703] (req 22212608->22216703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.491242:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2d573 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22212608->22216703] (req 22212608->22216703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.491251:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760708.491256:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2d573 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22212608->22216703] (req 22212608->22216703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.491273:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2d573 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22212608->22216703] (req 22212608->22216703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.491301:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2d573 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22212608->22216703] (req 22212608->22216703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.491310:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2d573 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22212608->22216703] (req 22212608->22216703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760708.491437:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2744380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512552064:12345-192.168.203.56@tcp:4:dd.0 Request processed in 122532us (123053us total) trans 4294972742 rc 0/0 00000100:00100000:0.0:1789760708.491449:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5453 00000100:00100000:0.0:1789760708.491461:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 687839701081 time 687933186389 state 0x2 ref 0 00000100:00100000:1.0:1789760708.497343:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760708.498659:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512552192 00000100:00100000:0.0:1789760708.498679:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512552192 00000100:00100000:0.0:1789760708.498784:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 687839701081 time 687933186389 state 0x0 ref 1 00000100:00100000:0.0:1789760708.498803:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 687933446805 time 687933186389 state 0x1 ref 1 00000100:00100000:0.0:1789760708.498837:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 687933446805 time 688063136391 state 0x0 ref 1 00000100:00100000:0.0:1789760708.498854:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac28a2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512552192:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760708.498920:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22216704, mtime: 0x6aad94c3, atime: 0x0, ctime: 0x6aad94c3, blocks: 0xb007 00010000:00010000:0.0:1789760708.498931:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2d57a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.498952:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=688s v=(0 0) 00010000:00010000:0.0:1789760708.498957:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760708.498965:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2d57a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22216704->22220799] (req 22216704->22220799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.498982:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2d57a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22216704->22220799] (req 22216704->22220799) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760708.536653:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972743 is committed 00002000:00010000:0.0:1789760708.538178:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22216704 -> 22220800 00002000:00010000:0.0:1789760708.538189:0:6710:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760707 -> 1789760708 00002000:00010000:0.0:1789760708.538192:0:6710:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760707 -> 1789760708 00010000:00010000:0.0:1789760708.538211:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2d57a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22216704->22220799] (req 22216704->22220799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.538226:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2d57a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22216704->22220799] (req 22216704->22220799) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.538236:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2d57a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22216704->22220799] (req 22216704->22220799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.538245:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760708.538250:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2d57a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22216704->22220799] (req 22216704->22220799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.538260:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2d57a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22216704->22220799] (req 22216704->22220799) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.538286:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2d57a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22216704->22220799] (req 22216704->22220799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.538294:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2d57a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22216704->22220799] (req 22216704->22220799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760708.538469:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac28a2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512552192:12345-192.168.203.56@tcp:4:dd.0 Request processed in 39613us (41135us total) trans 4294972743 rc 0/0 00000100:00100000:0.0:1789760708.538490:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5454 00000100:00100000:0.0:1789760708.538502:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 687933446805 time 688063136391 state 0x2 ref 0 00000100:00100000:1.0:1789760708.543517:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760708.543668:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512552320 00000100:00100000:3.0:1789760708.543682:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512552320 00000100:00100000:3.0:1789760708.543796:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 687933446805 time 688063136391 state 0x0 ref 1 00000100:00100000:3.0:1789760708.543815:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 688063396807 time 688063136391 state 0x1 ref 1 00000100:00100000:3.0:1789760708.543851:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 688063396807 time 688108150631 state 0x0 ref 1 00000100:00100000:3.0:1789760708.543864:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac28a0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512552320:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760708.543924:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22220800, mtime: 0x6aad94c4, atime: 0x0, ctime: 0x6aad94c4, blocks: 0xb007 00010000:00010000:3.0:1789760708.543936:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2d581 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760708.543956:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=688s v=(0 0) 00010000:00010000:3.0:1789760708.543962:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760708.543971:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2d581 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22220800->22224895] (req 22220800->22224895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760708.543987:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2d581 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22220800->22224895] (req 22220800->22224895) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760708.607887:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972744 is committed 00002000:00010000:0.0:1789760708.608025:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22220800 -> 22224896 00010000:00010000:0.0:1789760708.608045:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2d581 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22220800->22224895] (req 22220800->22224895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.608064:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2d581 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22220800->22224895] (req 22220800->22224895) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.608074:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2d581 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22220800->22224895] (req 22220800->22224895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.608083:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760708.608089:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2d581 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22220800->22224895] (req 22220800->22224895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.608113:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2d581 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22220800->22224895] (req 22220800->22224895) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.608138:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2d581 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22220800->22224895] (req 22220800->22224895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.608147:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2d581 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22220800->22224895] (req 22220800->22224895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760708.608260:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac28a0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512552320:12345-192.168.203.56@tcp:4:dd.0 Request processed in 64381us (64741us total) trans 4294972744 rc 0/0 00000100:00100000:0.0:1789760708.608272:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5455 00000100:00100000:0.0:1789760708.608284:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 688063396807 time 688108150631 state 0x2 ref 0 00000100:00100000:1.0:1789760708.614902:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760708.616778:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512552448 00000100:00100000:0.0:1789760708.616798:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512552448 00000100:00100000:1.0:1789760708.617386:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 688063396807 time 688108150631 state 0x0 ref 1 00000100:00100000:1.0:1789760708.617409:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 688108411047 time 688108150631 state 0x1 ref 1 00000100:00100000:1.0:1789760708.617456:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 688108411047 time 688181754320 state 0x0 ref 1 00000100:00100000:1.0:1789760708.617482:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac28a3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512552448:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760708.617550:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22224896, mtime: 0x6aad94c4, atime: 0x0, ctime: 0x6aad94c4, blocks: 0xb007 00010000:00010000:1.0:1789760708.617562:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2d588 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760708.617590:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=688s v=(0 0) 00010000:00010000:1.0:1789760708.617595:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760708.617604:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2d588 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22224896->22228991] (req 22224896->22228991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760708.617621:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2d588 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22224896->22228991] (req 22224896->22228991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760708.723818:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972745 is committed 00002000:00010000:3.0:1789760708.723962:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22224896 -> 22228992 00010000:00010000:3.0:1789760708.723994:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2d588 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22224896->22228991] (req 22224896->22228991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760708.724018:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2d588 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22224896->22228991] (req 22224896->22228991) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760708.724032:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2d588 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22224896->22228991] (req 22224896->22228991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760708.724041:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760708.724048:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2d588 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22224896->22228991] (req 22224896->22228991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760708.724058:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2d588 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22224896->22228991] (req 22224896->22228991) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760708.724082:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2d588 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22224896->22228991] (req 22224896->22228991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760708.724090:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2d588 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22224896->22228991] (req 22224896->22228991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760708.724222:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac28a3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512552448:12345-192.168.203.56@tcp:4:dd.0 Request processed in 106746us (109324us total) trans 4294972745 rc 0/0 00000100:00100000:3.0:1789760708.724235:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5456 00000100:00100000:3.0:1789760708.724248:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 688108411047 time 688181754320 state 0x2 ref 0 00000100:00100000:0.0:1789760708.730824:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760708.731064:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512552576 00000100:00100000:3.0:1789760708.731079:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512552576 00000100:00100000:3.0:1789760708.731160:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 688108411047 time 688181754320 state 0x0 ref 1 00000100:00100000:3.0:1789760708.731179:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 688182014736 time 688181754320 state 0x1 ref 1 00000100:00100000:3.0:1789760708.731218:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 688182014736 time 688295516853 state 0x0 ref 1 00000100:00100000:3.0:1789760708.731231:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febfbeae300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512552576:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760708.731527:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22228992, mtime: 0x6aad94c4, atime: 0x0, ctime: 0x6aad94c4, blocks: 0xb007 00010000:00010000:3.0:1789760708.731540:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2d58f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760708.731562:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=688s v=(0 0) 00010000:00010000:3.0:1789760708.731567:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760708.731575:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2d58f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22228992->22233087] (req 22228992->22233087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760708.731605:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2d58f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22228992->22233087] (req 22228992->22233087) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760708.772602:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972746 is committed 00002000:00010000:0.0:1789760708.772768:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22228992 -> 22233088 00010000:00010000:0.0:1789760708.772788:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2d58f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22228992->22233087] (req 22228992->22233087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.772804:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2d58f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22228992->22233087] (req 22228992->22233087) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.772815:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2d58f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22228992->22233087] (req 22228992->22233087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.772824:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760708.772829:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2d58f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22228992->22233087] (req 22228992->22233087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.772839:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2d58f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22228992->22233087] (req 22228992->22233087) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.772865:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2d58f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22228992->22233087] (req 22228992->22233087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.772874:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2d58f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22228992->22233087] (req 22228992->22233087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760708.773007:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febfbeae300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512552576:12345-192.168.203.56@tcp:4:dd.0 Request processed in 41770us (42189us total) trans 4294972746 rc 0/0 00000100:00100000:0.0:1789760708.773401:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5457 00000100:00100000:0.0:1789760708.773415:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 688182014736 time 688295516853 state 0x2 ref 0 00000100:00100000:0.0:1789760708.776449:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760708.776660:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512552704 00000100:00100000:2.0:1789760708.776674:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512552704 00000100:00100000:2.0:1789760708.776775:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 688182014736 time 688295516853 state 0x0 ref 1 00000100:00100000:2.0:1789760708.776796:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 688295777269 time 688295516853 state 0x1 ref 1 00000100:00100000:2.0:1789760708.776835:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 688295777269 time 688341134107 state 0x0 ref 1 00000100:00100000:2.0:1789760708.776849:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4362d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512552704:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760708.776916:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22233088, mtime: 0x6aad94c4, atime: 0x0, ctime: 0x6aad94c4, blocks: 0xb007 00010000:00010000:2.0:1789760708.776927:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2d596 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.776948:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=688s v=(0 0) 00010000:00010000:2.0:1789760708.776954:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760708.776963:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2d596 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22233088->22237183] (req 22233088->22237183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.777147:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2d596 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22233088->22237183] (req 22233088->22237183) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760708.810554:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972747 is committed 00002000:00010000:0.0:1789760708.810724:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22233088 -> 22237184 00010000:00010000:0.0:1789760708.810746:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2d596 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22233088->22237183] (req 22233088->22237183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.810762:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2d596 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22233088->22237183] (req 22233088->22237183) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.810780:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2d596 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22233088->22237183] (req 22233088->22237183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.810788:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760708.810794:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2d596 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22233088->22237183] (req 22233088->22237183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.810805:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2d596 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22233088->22237183] (req 22233088->22237183) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.810832:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2d596 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22233088->22237183] (req 22233088->22237183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.810841:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2d596 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22233088->22237183] (req 22233088->22237183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760708.810965:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4362d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512552704:12345-192.168.203.56@tcp:4:dd.0 Request processed in 34112us (34525us total) trans 4294972747 rc 0/0 00000100:00100000:0.0:1789760708.810987:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5458 00000100:00100000:0.0:1789760708.810999:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 688295777269 time 688341134107 state 0x2 ref 0 00000100:00100000:2.0:1789760708.815308:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760708.815885:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512552832 00000100:00100000:3.0:1789760708.815900:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512552832 00000100:00100000:3.0:1789760708.815998:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 688295777269 time 688341134107 state 0x0 ref 1 00000100:00100000:3.0:1789760708.816029:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 688341394523 time 688341134107 state 0x1 ref 1 00000100:00100000:3.0:1789760708.816065:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 688341394523 time 688380363545 state 0x0 ref 1 00000100:00100000:3.0:1789760708.816080:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac38b4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512552832:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760708.816156:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22237184, mtime: 0x6aad94c4, atime: 0x0, ctime: 0x6aad94c4, blocks: 0xb007 00010000:00010000:3.0:1789760708.816167:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2d59d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760708.816188:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=688s v=(0 0) 00010000:00010000:3.0:1789760708.816193:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760708.816202:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2d59d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22237184->22241279] (req 22237184->22241279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760708.816219:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2d59d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22237184->22241279] (req 22237184->22241279) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760708.853657:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972748 is committed 00002000:00010000:3.0:1789760708.853849:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22237184 -> 22241280 00010000:00010000:3.0:1789760708.853872:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2d59d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22237184->22241279] (req 22237184->22241279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760708.853894:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2d59d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22237184->22241279] (req 22237184->22241279) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760708.853904:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2d59d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22237184->22241279] (req 22237184->22241279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760708.853912:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760708.853918:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2d59d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22237184->22241279] (req 22237184->22241279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760708.853928:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2d59d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22237184->22241279] (req 22237184->22241279) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760708.853955:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2d59d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22237184->22241279] (req 22237184->22241279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760708.853963:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2d59d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22237184->22241279] (req 22237184->22241279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760708.854564:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac38b4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512552832:12345-192.168.203.56@tcp:4:dd.0 Request processed in 38478us (39270us total) trans 4294972748 rc 0/0 00000100:00100000:3.0:1789760708.854575:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5459 00000100:00100000:3.0:1789760708.854601:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 688341394523 time 688380363545 state 0x2 ref 0 00000100:00100000:2.0:1789760708.858925:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760708.859191:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512552960 00000100:00100000:0.0:1789760708.859206:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512552960 00000100:00100000:0.0:1789760708.859416:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 688341394523 time 688380363545 state 0x0 ref 1 00000100:00100000:0.0:1789760708.859443:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 688380623961 time 688380363545 state 0x1 ref 1 00000100:00100000:0.0:1789760708.859525:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 688380623961 time 688423822730 state 0x0 ref 1 00000100:00100000:0.0:1789760708.859542:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac38b7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512552960:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760708.859626:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22241280, mtime: 0x6aad94c4, atime: 0x0, ctime: 0x6aad94c4, blocks: 0xb007 00010000:00010000:0.0:1789760708.859637:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2d5a4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.859660:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=688s v=(0 0) 00010000:00010000:0.0:1789760708.859665:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760708.859672:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2d5a4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22241280->22245375] (req 22241280->22245375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.859688:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2d5a4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22241280->22245375] (req 22241280->22245375) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760708.873110:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760708.873155:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760708.873259:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760708.873301:0:8229:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512553216 00000100:00100000:0.0:1789760708.873314:0:8229:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512553216 00000100:00100000:0.0:1789760708.873390:0:8229:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512553344 00000100:00100000:0.0:1789760708.873394:0:8229:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512553344 00000100:00100000:0.0:1789760708.873425:0:8229:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 439 00000100:00100000:0.0:1789760708.873440:0:8229:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4361500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+8:2359:x1876699512553216:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:1.0:1789760708.874761:0:5848:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 440 00000100:00100000:1.0:1789760708.874784:0:5848:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4361180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:d2c40150-e97b-40da-952b-cb9f18bf7e96+5:2359:x1876699512553344:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:1.0:1789760708.875421:0:5848:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4361180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:d2c40150-e97b-40da-952b-cb9f18bf7e96+5:2359:x1876699512553344:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 642us (2261us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760708.875432:0:5848:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 440 00000100:00100000:2.0:1789760708.876944:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512553088 00000100:00100000:2.0:1789760708.876958:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512553088 00000100:00100000:2.0:1789760708.877039:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 384 00000100:00100000:2.0:1789760708.877051:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febc3ebf050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:e6cf7267-9b19-462a-a263-14115df15dab+13:2358:x1876699512553088:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:0.0:1789760708.878488:0:8229:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4361500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+8:2359:x1876699512553216:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 5026us (5367us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760708.878508:0:8229:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 439 00000100:00080000:0.0:1789760708.878558:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789760708.883214:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febc3ebf050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:e6cf7267-9b19-462a-a263-14115df15dab+13:2358:x1876699512553088:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 6027us (9816us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760708.883400:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 384 00000001:00080000:1.0:1789760708.948181:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972749 is committed 00002000:00010000:0.0:1789760708.948429:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22241280 -> 22245376 00010000:00010000:0.0:1789760708.948449:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2d5a4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22241280->22245375] (req 22241280->22245375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.948465:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2d5a4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22241280->22245375] (req 22241280->22245375) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.948543:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2d5a4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22241280->22245375] (req 22241280->22245375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.948552:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760708.948558:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2d5a4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22241280->22245375] (req 22241280->22245375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.948569:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2d5a4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22241280->22245375] (req 22241280->22245375) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.948603:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2d5a4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22241280->22245375] (req 22241280->22245375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.948612:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2d5a4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22241280->22245375] (req 22241280->22245375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760708.948825:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac38b7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512552960:12345-192.168.203.56@tcp:4:dd.0 Request processed in 89278us (89906us total) trans 4294972749 rc 0/0 00000100:00100000:0.0:1789760708.948838:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5460 00000100:00100000:0.0:1789760708.948850:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 688380623961 time 688423822730 state 0x2 ref 0 00000100:00100000:3.0:1789760708.958794:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760708.958955:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512553472 00000100:00100000:0.0:1789760708.958969:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512553472 00000100:00100000:0.0:1789760708.959043:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 688380623961 time 688423822730 state 0x0 ref 1 00000100:00100000:0.0:1789760708.959063:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 688424083146 time 688423822730 state 0x1 ref 1 00000100:00100000:0.0:1789760708.959113:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 688424083146 time 688523411693 state 0x0 ref 1 00000100:00100000:0.0:1789760708.959128:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2746680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512553472:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760708.959201:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22245376, mtime: 0x6aad94c4, atime: 0x0, ctime: 0x6aad94c4, blocks: 0xb007 00010000:00010000:0.0:1789760708.959212:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2d5ab lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.959232:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=688s v=(0 0) 00010000:00010000:0.0:1789760708.959237:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760708.959245:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2d5ab lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22245376->22249471] (req 22245376->22249471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.959274:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2d5ab lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22245376->22249471] (req 22245376->22249471) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760708.981825:0:5835:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:0.0:1789760709.071941:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22245376 -> 22249472 00010000:00010000:0.0:1789760709.071968:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2d5ab lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22245376->22249471] (req 22245376->22249471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760709.071984:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2d5ab lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22245376->22249471] (req 22245376->22249471) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760709.071995:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2d5ab lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22245376->22249471] (req 22245376->22249471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760709.072004:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760709.072021:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2d5ab lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22245376->22249471] (req 22245376->22249471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760709.072031:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2d5ab lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22245376->22249471] (req 22245376->22249471) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760709.072057:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2d5ab lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22245376->22249471] (req 22245376->22249471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760709.072066:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2d5ab lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22245376->22249471] (req 22245376->22249471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760709.072186:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2746680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2361:x1876699512553472:12345-192.168.203.56@tcp:4:dd.0 Request processed in 113054us (113399us total) trans 4294972750 rc 0/0 00000100:00100000:0.0:1789760709.072203:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5461 00000100:00100000:0.0:1789760709.072215:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 688424083146 time 688523411693 state 0x2 ref 0 00000001:00080000:1.0:1789760709.073649:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972750 is committed 00000100:00100000:1.0:1789760709.080354:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760709.080656:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512553600 00000100:00100000:3.0:1789760709.080669:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512553600 00000100:00100000:3.0:1789760709.080799:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 688424083146 time 688523411693 state 0x0 ref 1 00000100:00100000:3.0:1789760709.080819:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 688523672109 time 688523411693 state 0x1 ref 1 00000100:00080000:0.0:1789760709.080915:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1789760709.080980:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 688523672109 time 688645275545 state 0x0 ref 1 00000100:00100000:2.0:1789760709.080997:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2fb8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512553600:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760709.081074:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22249472, mtime: 0x6aad94c4, atime: 0x0, ctime: 0x6aad94c4, blocks: 0xb007 00010000:00010000:2.0:1789760709.081084:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2d5b2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.081108:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=688s v=(0 0) 00010000:00010000:2.0:1789760709.081113:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760709.081122:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2d5b2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22249472->22253567] (req 22249472->22253567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.081137:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2d5b2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22249472->22253567] (req 22249472->22253567) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760709.205560:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22249472 -> 22253568 00010000:00010000:2.0:1789760709.205595:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2d5b2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22249472->22253567] (req 22249472->22253567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.205612:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2d5b2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22249472->22253567] (req 22249472->22253567) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.205623:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2d5b2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22249472->22253567] (req 22249472->22253567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.205631:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760709.205637:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2d5b2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22249472->22253567] (req 22249472->22253567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.205647:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2d5b2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22249472->22253567] (req 22249472->22253567) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.205673:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2d5b2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22249472->22253567] (req 22249472->22253567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.205683:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2d5b2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22249472->22253567] (req 22249472->22253567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760709.205827:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2fb8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2358:x1876699512553600:12345-192.168.203.56@tcp:4:dd.0 Request processed in 124827us (125480us total) trans 4294972751 rc 0/0 00000100:00100000:2.0:1789760709.205840:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5462 00000100:00100000:2.0:1789760709.205851:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 688523672109 time 688645275545 state 0x2 ref 0 00000001:00080000:1.0:1789760709.207640:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972751 is committed 00000100:00100000:1.0:1789760709.210823:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760709.210954:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512553728 00000100:00100000:0.0:1789760709.210978:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512553728 00000100:00100000:0.0:1789760709.211062:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 688523672109 time 688645275545 state 0x0 ref 1 00000100:00100000:0.0:1789760709.211081:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 688645535961 time 688645275545 state 0x1 ref 1 00000100:00100000:0.0:1789760709.211124:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 688645535961 time 688775423283 state 0x0 ref 1 00000100:00100000:0.0:1789760709.211139:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2fb9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512553728:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760709.211208:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22253568, mtime: 0x6aad94c4, atime: 0x0, ctime: 0x6aad94c4, blocks: 0xb007 00010000:00010000:0.0:1789760709.211220:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2d5b9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760709.211241:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=688s v=(0 0) 00010000:00010000:0.0:1789760709.211454:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760709.211464:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2d5b9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22253568->22257663] (req 22253568->22257663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760709.211496:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2d5b9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22253568->22257663] (req 22253568->22257663) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760709.268622:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972752 is committed 00002000:00010000:2.0:1789760709.268828:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22253568 -> 22257664 00010000:00010000:2.0:1789760709.268850:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2d5b9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22253568->22257663] (req 22253568->22257663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.268866:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2d5b9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22253568->22257663] (req 22253568->22257663) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.268876:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2d5b9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22253568->22257663] (req 22253568->22257663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.268883:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760709.268889:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2d5b9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22253568->22257663] (req 22253568->22257663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.268898:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2d5b9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22253568->22257663] (req 22253568->22257663) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.268928:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2d5b9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22253568->22257663] (req 22253568->22257663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.268937:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2d5b9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22253568->22257663] (req 22253568->22257663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760709.269689:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2fb9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512553728:12345-192.168.203.56@tcp:4:dd.0 Request processed in 58544us (58871us total) trans 4294972752 rc 0/0 00000100:00100000:2.0:1789760709.269746:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5463 00000100:00100000:2.0:1789760709.269758:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 688645535961 time 688775423283 state 0x2 ref 0 00000100:00100000:0.0:1789760709.282629:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760709.282877:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512553856 00000100:00100000:1.0:1789760709.282891:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512553856 00000100:00100000:1.0:1789760709.282978:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 688645535961 time 688775423283 state 0x0 ref 1 00000100:00100000:1.0:1789760709.282997:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 688775683699 time 688775423283 state 0x1 ref 1 00000100:00100000:1.0:1789760709.283034:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 688775683699 time 688847333517 state 0x0 ref 1 00000100:00100000:1.0:1789760709.283049:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febc7b33480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512553856:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760709.283506:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22257664, mtime: 0x6aad94c4, atime: 0x0, ctime: 0x6aad94c4, blocks: 0xb007 00010000:00010000:1.0:1789760709.283520:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2d5c0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.283540:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=688s v=(0 0) 00010000:00010000:1.0:1789760709.283545:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760709.283552:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2d5c0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22257664->22261759] (req 22257664->22261759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.283574:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2d5c0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22257664->22261759] (req 22257664->22261759) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760709.355719:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972753 is committed 00002000:00010000:0.0:1789760709.355877:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22257664 -> 22261760 00010000:00010000:0.0:1789760709.355900:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2d5c0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22257664->22261759] (req 22257664->22261759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760709.355916:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2d5c0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22257664->22261759] (req 22257664->22261759) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760709.355926:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2d5c0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22257664->22261759] (req 22257664->22261759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760709.355934:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760709.355940:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2d5c0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22257664->22261759] (req 22257664->22261759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760709.355950:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2d5c0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22257664->22261759] (req 22257664->22261759) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760709.355987:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2d5c0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22257664->22261759] (req 22257664->22261759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760709.355996:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2d5c0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22257664->22261759] (req 22257664->22261759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760709.356139:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febc7b33480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512553856:12345-192.168.203.56@tcp:4:dd.0 Request processed in 73078us (73511us total) trans 4294972753 rc 0/0 00000100:00100000:0.0:1789760709.356153:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5464 00000100:00100000:0.0:1789760709.356167:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 688775683699 time 688847333517 state 0x2 ref 0 00000100:00100000:1.0:1789760709.361826:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760709.361981:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512553984 00000100:00100000:3.0:1789760709.361994:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512553984 00000100:00100000:3.0:1789760709.362079:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 688775683699 time 688847333517 state 0x0 ref 1 00000100:00100000:3.0:1789760709.362109:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 688847593933 time 688847333517 state 0x1 ref 1 00000100:00100000:3.0:1789760709.362234:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 688847593933 time 688926531298 state 0x0 ref 1 00000100:00100000:3.0:1789760709.362259:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2fb8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512553984:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760709.362624:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22261760, mtime: 0x6aad94c4, atime: 0x0, ctime: 0x6aad94c4, blocks: 0xb007 00010000:00010000:3.0:1789760709.362635:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2d5c7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760709.362657:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=688s v=(0 0) 00010000:00010000:3.0:1789760709.362662:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760709.362670:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2d5c7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22261760->22265855] (req 22261760->22265855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760709.362685:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2d5c7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22261760->22265855] (req 22261760->22265855) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760709.414874:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22261760 -> 22265856 00010000:00010000:0.0:1789760709.414902:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2d5c7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22261760->22265855] (req 22261760->22265855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760709.414919:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2d5c7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22261760->22265855] (req 22261760->22265855) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760709.414930:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2d5c7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22261760->22265855] (req 22261760->22265855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760709.414939:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760709.414944:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2d5c7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22261760->22265855] (req 22261760->22265855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760709.414954:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2d5c7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22261760->22265855] (req 22261760->22265855) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760709.414987:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2d5c7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22261760->22265855] (req 22261760->22265855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760709.414997:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2d5c7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22261760->22265855] (req 22261760->22265855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760709.415426:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2fb8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2358:x1876699512553984:12345-192.168.203.56@tcp:4:dd.0 Request processed in 53170us (53604us total) trans 4294972754 rc 0/0 00000100:00100000:0.0:1789760709.415446:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5465 00000100:00100000:0.0:1789760709.415459:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 688847593933 time 688926531298 state 0x2 ref 0 00000001:00080000:1.0:1789760709.415903:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972754 is committed 00000100:00100000:1.0:1789760709.421397:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760709.421632:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512554112 00000100:00100000:2.0:1789760709.421647:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512554112 00000100:00100000:2.0:1789760709.421853:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 688847593933 time 688926531298 state 0x0 ref 1 00000100:00100000:2.0:1789760709.421873:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 688926791714 time 688926531298 state 0x1 ref 1 00000100:00100000:2.0:1789760709.421941:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 688926791714 time 688986239650 state 0x0 ref 1 00000100:00100000:2.0:1789760709.421957:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2fba680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512554112:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760709.422045:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22265856, mtime: 0x6aad94c4, atime: 0x0, ctime: 0x6aad94c4, blocks: 0xb007 00010000:00010000:2.0:1789760709.422056:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2d5ce lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.422077:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=688s v=(0 0) 00010000:00010000:2.0:1789760709.422083:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760709.422100:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2d5ce lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22265856->22269951] (req 22265856->22269951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.422118:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2d5ce lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22265856->22269951] (req 22265856->22269951) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760709.473991:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972755 is committed 00002000:00010000:1.0:1789760709.474690:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22265856 -> 22269952 00010000:00010000:1.0:1789760709.474737:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2d5ce lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22265856->22269951] (req 22265856->22269951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.474753:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2d5ce lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22265856->22269951] (req 22265856->22269951) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.474764:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2d5ce lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22265856->22269951] (req 22265856->22269951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.474772:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760709.474778:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2d5ce lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22265856->22269951] (req 22265856->22269951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.474790:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2d5ce lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22265856->22269951] (req 22265856->22269951) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.474817:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2d5ce lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22265856->22269951] (req 22265856->22269951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.474825:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2d5ce lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22265856->22269951] (req 22265856->22269951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760709.474962:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2fba680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512554112:12345-192.168.203.56@tcp:4:dd.0 Request processed in 53001us (53574us total) trans 4294972755 rc 0/0 00000100:00100000:1.0:1789760709.475095:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5466 00000100:00100000:1.0:1789760709.475108:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 688926791714 time 688986239650 state 0x2 ref 0 00000100:00100000:0.0:1789760709.480352:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760709.480466:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512554240 00000100:00100000:1.0:1789760709.480479:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512554240 00000100:00100000:1.0:1789760709.480616:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 688926791714 time 688986239650 state 0x0 ref 1 00000100:00100000:1.0:1789760709.480637:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 688986500066 time 688986239650 state 0x1 ref 1 00000100:00100000:1.0:1789760709.480747:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 688986500066 time 689045045409 state 0x0 ref 1 00000100:00100000:1.0:1789760709.480762:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febc7b32680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512554240:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760709.480829:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22269952, mtime: 0x6aad94c4, atime: 0x0, ctime: 0x6aad94c4, blocks: 0xb007 00010000:00010000:1.0:1789760709.480839:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2d5d5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.480861:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=689s v=(0 0) 00010000:00010000:1.0:1789760709.480866:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760709.480874:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2d5d5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22269952->22274047] (req 22269952->22274047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.480897:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2d5d5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22269952->22274047] (req 22269952->22274047) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760709.565919:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972756 is committed 00002000:00010000:1.0:1789760709.566228:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22269952 -> 22274048 00010000:00010000:1.0:1789760709.566249:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2d5d5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22269952->22274047] (req 22269952->22274047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.566264:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2d5d5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22269952->22274047] (req 22269952->22274047) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.566274:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2d5d5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22269952->22274047] (req 22269952->22274047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.566282:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760709.566288:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2d5d5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22269952->22274047] (req 22269952->22274047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.566298:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2d5d5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22269952->22274047] (req 22269952->22274047) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.566323:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2d5d5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22269952->22274047] (req 22269952->22274047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.566332:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2d5d5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22269952->22274047] (req 22269952->22274047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760709.566472:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febc7b32680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512554240:12345-192.168.203.56@tcp:4:dd.0 Request processed in 85699us (86120us total) trans 4294972756 rc 0/0 00000100:00100000:1.0:1789760709.566548:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5467 00000100:00100000:1.0:1789760709.566561:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 688986500066 time 689045045409 state 0x2 ref 0 00000100:00100000:3.0:1789760709.570834:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760709.571272:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512554368 00000100:00100000:0.0:1789760709.571287:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512554368 00000100:00100000:0.0:1789760709.571360:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 688986500066 time 689045045409 state 0x0 ref 1 00000100:00100000:0.0:1789760709.571379:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 689045305825 time 689045045409 state 0x1 ref 1 00000100:00100000:0.0:1789760709.571417:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 689045305825 time 689135715850 state 0x0 ref 1 00000100:00100000:0.0:1789760709.571431:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2747b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512554368:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760709.571597:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22274048, mtime: 0x6aad94c4, atime: 0x0, ctime: 0x6aad94c4, blocks: 0xb007 00010000:00010000:0.0:1789760709.571609:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2d5dc lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760709.571632:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=689s v=(0 0) 00010000:00010000:0.0:1789760709.571637:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760709.571646:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2d5dc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22274048->22278143] (req 22274048->22278143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760709.571662:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2d5dc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22274048->22278143] (req 22274048->22278143) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760709.658503:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972757 is committed 00002000:00010000:0.0:1789760709.658835:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22274048 -> 22278144 00002000:00010000:0.0:1789760709.658841:0:13063:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760708 -> 1789760709 00002000:00010000:0.0:1789760709.658844:0:13063:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760708 -> 1789760709 00010000:00010000:0.0:1789760709.658862:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2d5dc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22274048->22278143] (req 22274048->22278143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760709.658877:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2d5dc lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22274048->22278143] (req 22274048->22278143) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760709.658887:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2d5dc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22274048->22278143] (req 22274048->22278143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760709.658896:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760709.658901:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2d5dc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22274048->22278143] (req 22274048->22278143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760709.658911:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2d5dc lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22274048->22278143] (req 22274048->22278143) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760709.658934:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2d5dc lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22274048->22278143] (req 22274048->22278143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760709.658957:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2d5dc lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22274048->22278143] (req 22274048->22278143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760709.659180:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2747b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512554368:12345-192.168.203.56@tcp:4:dd.0 Request processed in 87744us (88353us total) trans 4294972757 rc 0/0 00000100:00100000:0.0:1789760709.659194:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5468 00000100:00100000:0.0:1789760709.659207:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 689045305825 time 689135715850 state 0x2 ref 0 00000100:00100000:2.0:1789760709.666219:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760709.666314:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512554496 00000100:00100000:1.0:1789760709.666328:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512554496 00000100:00100000:1.0:1789760709.666400:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 689045305825 time 689135715850 state 0x0 ref 1 00000100:00100000:1.0:1789760709.666417:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 689135976266 time 689135715850 state 0x1 ref 1 00000100:00100000:1.0:1789760709.666452:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 689135976266 time 689230751316 state 0x0 ref 1 00000100:00100000:1.0:1789760709.666467:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac38b4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512554496:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760709.666532:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22278144, mtime: 0x6aad94c5, atime: 0x0, ctime: 0x6aad94c5, blocks: 0xb007 00010000:00010000:1.0:1789760709.666543:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2d5e3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.666564:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=689s v=(0 0) 00010000:00010000:1.0:1789760709.666569:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760709.666585:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2d5e3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22278144->22282239] (req 22278144->22282239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.666603:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2d5e3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22278144->22282239] (req 22278144->22282239) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760709.743534:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972758 is committed 00002000:00010000:3.0:1789760709.743769:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22278144 -> 22282240 00010000:00010000:3.0:1789760709.743790:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2d5e3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22278144->22282239] (req 22278144->22282239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760709.743805:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2d5e3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22278144->22282239] (req 22278144->22282239) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760709.743816:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2d5e3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22278144->22282239] (req 22278144->22282239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760709.743825:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760709.743831:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2d5e3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22278144->22282239] (req 22278144->22282239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760709.743844:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2d5e3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22278144->22282239] (req 22278144->22282239) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760709.743869:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2d5e3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22278144->22282239] (req 22278144->22282239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760709.743879:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2d5e3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22278144->22282239] (req 22278144->22282239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760709.744180:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac38b4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512554496:12345-192.168.203.56@tcp:4:dd.0 Request processed in 77536us (77798us total) trans 4294972758 rc 0/0 00000100:00100000:3.0:1789760709.744195:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5469 00000100:00100000:3.0:1789760709.744207:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 689135976266 time 689230751316 state 0x2 ref 0 00000100:00100000:2.0:1789760709.756054:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760709.756224:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512554624 00000100:00100000:1.0:1789760709.756239:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512554624 00000100:00100000:1.0:1789760709.756339:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 689135976266 time 689230751316 state 0x0 ref 1 00000100:00100000:1.0:1789760709.756362:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 689231011732 time 689230751316 state 0x1 ref 1 00000100:00100000:1.0:1789760709.756401:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 689231011732 time 689320700934 state 0x0 ref 1 00000100:00100000:1.0:1789760709.756416:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac38b4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512554624:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760709.756495:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22282240, mtime: 0x6aad94c5, atime: 0x0, ctime: 0x6aad94c5, blocks: 0xb007 00010000:00010000:1.0:1789760709.756508:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2d5ea lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.756533:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=689s v=(0 0) 00010000:00010000:1.0:1789760709.756539:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760709.756549:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2d5ea lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22282240->22286335] (req 22282240->22286335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.756566:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2d5ea lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22282240->22286335] (req 22282240->22286335) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760709.857972:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972759 is committed 00002000:00010000:1.0:1789760709.858120:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22282240 -> 22286336 00010000:00010000:1.0:1789760709.858142:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2d5ea lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22282240->22286335] (req 22282240->22286335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.858164:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2d5ea lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22282240->22286335] (req 22282240->22286335) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.858173:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2d5ea lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22282240->22286335] (req 22282240->22286335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.858181:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760709.858186:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2d5ea lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22282240->22286335] (req 22282240->22286335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.858196:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2d5ea lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22282240->22286335] (req 22282240->22286335) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.858221:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2d5ea lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22282240->22286335] (req 22282240->22286335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.858229:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2d5ea lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22282240->22286335] (req 22282240->22286335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760709.858378:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac38b4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512554624:12345-192.168.203.56@tcp:4:dd.0 Request processed in 101958us (102331us total) trans 4294972759 rc 0/0 00000100:00100000:1.0:1789760709.858389:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5470 00000100:00100000:1.0:1789760709.858401:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 689231011732 time 689320700934 state 0x2 ref 0 00000100:00100000:1.0:1789760709.871796:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760709.871871:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512554752 00000100:00100000:0.0:1789760709.871883:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512554752 00000100:00100000:0.0:1789760709.871954:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 689231011732 time 689320700934 state 0x0 ref 1 00000100:00100000:0.0:1789760709.871972:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 689320961350 time 689320700934 state 0x1 ref 1 00000100:00100000:0.0:1789760709.872007:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 689320961350 time 689436307110 state 0x0 ref 1 00000100:00100000:0.0:1789760709.872020:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2fbb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512554752:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760709.872079:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22286336, mtime: 0x6aad94c5, atime: 0x0, ctime: 0x6aad94c5, blocks: 0xb007 00010000:00010000:0.0:1789760709.872089:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2d5f1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760709.872108:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=689s v=(0 0) 00010000:00010000:0.0:1789760709.872112:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760709.872120:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2d5f1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22286336->22290431] (req 22286336->22290431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760709.872137:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2d5f1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22286336->22290431] (req 22286336->22290431) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760709.984517:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22286336 -> 22290432 00010000:00010000:1.0:1789760709.984544:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2d5f1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22286336->22290431] (req 22286336->22290431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.984561:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2d5f1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22286336->22290431] (req 22286336->22290431) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.984580:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2d5f1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22286336->22290431] (req 22286336->22290431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.984589:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760709.984594:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2d5f1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22286336->22290431] (req 22286336->22290431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.984603:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2d5f1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22286336->22290431] (req 22286336->22290431) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.984628:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2d5f1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22286336->22290431] (req 22286336->22290431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.984636:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2d5f1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22286336->22290431] (req 22286336->22290431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760709.984799:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2fbb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2360:x1876699512554752:12345-192.168.203.56@tcp:4:dd.0 Request processed in 112773us (113009us total) trans 4294972760 rc 0/0 00000100:00100000:1.0:1789760709.984812:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5471 00000100:00100000:1.0:1789760709.984824:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 689320961350 time 689436307110 state 0x2 ref 0 00000001:00080000:2.0:1789760709.985090:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972760 is committed 00000100:00100000:2.0:1789760709.991598:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760709.992000:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512554880 00000100:00100000:3.0:1789760709.992015:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512554880 00000100:00100000:3.0:1789760709.992091:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 689320961350 time 689436307110 state 0x0 ref 1 00000100:00100000:3.0:1789760709.992110:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 689436567526 time 689436307110 state 0x1 ref 1 00000100:00100000:3.0:1789760709.992146:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 689436567526 time 689556445645 state 0x0 ref 1 00000100:00100000:3.0:1789760709.992160:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac38b5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512554880:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760709.992229:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22290432, mtime: 0x6aad94c5, atime: 0x0, ctime: 0x6aad94c5, blocks: 0xb007 00010000:00010000:3.0:1789760709.992358:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2d5f8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760709.992380:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=689s v=(0 0) 00010000:00010000:3.0:1789760709.992385:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760709.992394:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2d5f8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22290432->22294527] (req 22290432->22294527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760709.992411:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2d5f8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22290432->22294527] (req 22290432->22294527) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.002842:0:7647:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:2.0:1789760710.064445:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972761 is committed 00002000:00010000:1.0:1789760710.064636:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22290432 -> 22294528 00010000:00010000:1.0:1789760710.064657:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2d5f8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22290432->22294527] (req 22290432->22294527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760710.064673:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2d5f8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22290432->22294527] (req 22290432->22294527) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760710.064682:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2d5f8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22290432->22294527] (req 22290432->22294527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760710.064689:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760710.064770:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2d5f8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22290432->22294527] (req 22290432->22294527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760710.064782:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2d5f8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22290432->22294527] (req 22290432->22294527) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760710.064807:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2d5f8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22290432->22294527] (req 22290432->22294527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760710.064816:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2d5f8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22290432->22294527] (req 22290432->22294527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760710.064949:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac38b5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512554880:12345-192.168.203.56@tcp:4:dd.0 Request processed in 72784us (73358us total) trans 4294972761 rc 0/0 00000100:00100000:1.0:1789760710.064961:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5472 00000100:00100000:1.0:1789760710.065197:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 689436567526 time 689556445645 state 0x2 ref 0 00000100:00100000:0.0:1789760710.080934:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760710.081057:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512555008 00000100:00100000:1.0:1789760710.081072:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512555008 00000100:00100000:1.0:1789760710.081159:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 689436567526 time 689556445645 state 0x0 ref 1 00000100:00100000:1.0:1789760710.081180:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 689556706061 time 689556445645 state 0x1 ref 1 00000100:00100000:1.0:1789760710.081217:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 689556706061 time 689645516440 state 0x0 ref 1 00000100:00100000:1.0:1789760710.081231:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febfb894700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512555008:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760710.081311:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22294528, mtime: 0x6aad94c5, atime: 0x0, ctime: 0x6aad94c5, blocks: 0xb007 00010000:00010000:1.0:1789760710.081323:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2d5ff lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760710.081344:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=689s v=(0 0) 00010000:00010000:1.0:1789760710.081349:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760710.081356:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2d5ff lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22294528->22298623] (req 22294528->22298623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760710.081374:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2d5ff lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22294528->22298623] (req 22294528->22298623) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760710.135193:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972762 is committed 00002000:00010000:0.0:1789760710.135354:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22294528 -> 22298624 00010000:00010000:0.0:1789760710.135375:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2d5ff lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22294528->22298623] (req 22294528->22298623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.135392:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2d5ff lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22294528->22298623] (req 22294528->22298623) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.135402:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2d5ff lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22294528->22298623] (req 22294528->22298623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.135412:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760710.135418:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2d5ff lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22294528->22298623] (req 22294528->22298623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.135427:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2d5ff lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22294528->22298623] (req 22294528->22298623) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.135452:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2d5ff lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22294528->22298623] (req 22294528->22298623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.135525:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2d5ff lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22294528->22298623] (req 22294528->22298623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760710.135798:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febfb894700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512555008:12345-192.168.203.56@tcp:4:dd.0 Request processed in 54563us (54871us total) trans 4294972762 rc 0/0 00000100:00100000:0.0:1789760710.135811:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5473 00000100:00100000:0.0:1789760710.135824:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 689556706061 time 689645516440 state 0x2 ref 0 00000100:00100000:2.0:1789760710.149764:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760710.149884:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512555136 00000100:00100000:1.0:1789760710.149899:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512555136 00000100:00100000:1.0:1789760710.149986:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 689556706061 time 689645516440 state 0x0 ref 1 00000100:00100000:1.0:1789760710.150007:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 689645776856 time 689645516440 state 0x1 ref 1 00000100:00100000:1.0:1789760710.150036:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 689645776856 time 689714335426 state 0x0 ref 1 00000100:00100000:1.0:1789760710.150051:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac38b6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512555136:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760710.150125:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22298624, mtime: 0x6aad94c5, atime: 0x0, ctime: 0x6aad94c5, blocks: 0xb007 00010000:00010000:1.0:1789760710.150137:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2d606 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760710.150161:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=689s v=(0 0) 00010000:00010000:1.0:1789760710.150166:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760710.150174:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2d606 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22298624->22302719] (req 22298624->22302719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760710.150192:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2d606 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22298624->22302719] (req 22298624->22302719) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760710.223760:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972763 is committed 00002000:00010000:1.0:1789760710.224131:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22298624 -> 22302720 00010000:00010000:1.0:1789760710.224152:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2d606 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22298624->22302719] (req 22298624->22302719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760710.224168:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2d606 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22298624->22302719] (req 22298624->22302719) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760710.224178:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2d606 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22298624->22302719] (req 22298624->22302719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760710.224186:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760710.224193:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2d606 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22298624->22302719] (req 22298624->22302719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760710.224201:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2d606 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22298624->22302719] (req 22298624->22302719) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760710.224225:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2d606 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22298624->22302719] (req 22298624->22302719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760710.224233:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2d606 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22298624->22302719] (req 22298624->22302719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760710.224358:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac38b6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512555136:12345-192.168.203.56@tcp:4:dd.0 Request processed in 74303us (74654us total) trans 4294972763 rc 0/0 00000100:00100000:1.0:1789760710.224369:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5474 00000100:00100000:1.0:1789760710.224381:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 689645776856 time 689714335426 state 0x2 ref 0 00000100:00100000:1.0:1789760710.228926:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760710.229042:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512555264 00000100:00100000:3.0:1789760710.229055:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512555264 00000100:00100000:3.0:1789760710.229134:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 689645776856 time 689714335426 state 0x0 ref 1 00000100:00100000:3.0:1789760710.229154:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 689714595842 time 689714335426 state 0x1 ref 1 00000100:00100000:3.0:1789760710.229182:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 689714595842 time 689793481863 state 0x0 ref 1 00000100:00100000:3.0:1789760710.229197:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2fb8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512555264:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760710.229274:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22302720, mtime: 0x6aad94c5, atime: 0x0, ctime: 0x6aad94c5, blocks: 0xb007 00010000:00010000:3.0:1789760710.229286:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2d60d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.229314:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=689s v=(0 0) 00010000:00010000:3.0:1789760710.229320:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760710.229328:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2d60d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22302720->22306815] (req 22302720->22306815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.229344:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2d60d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22302720->22306815] (req 22302720->22306815) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760710.280925:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972764 is committed 00002000:00010000:0.0:1789760710.281230:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22302720 -> 22306816 00010000:00010000:0.0:1789760710.281263:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2d60d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22302720->22306815] (req 22302720->22306815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.281281:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2d60d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22302720->22306815] (req 22302720->22306815) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.281292:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2d60d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22302720->22306815] (req 22302720->22306815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.281301:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760710.281308:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2d60d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22302720->22306815] (req 22302720->22306815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.281319:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2d60d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22302720->22306815] (req 22302720->22306815) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.281345:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2d60d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22302720->22306815] (req 22302720->22306815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.281353:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2d60d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22302720->22306815] (req 22302720->22306815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760710.281477:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2fb8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512555264:12345-192.168.203.56@tcp:4:dd.0 Request processed in 52277us (52560us total) trans 4294972764 rc 0/0 00000100:00100000:0.0:1789760710.281488:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5475 00000100:00100000:0.0:1789760710.281501:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 689714595842 time 689793481863 state 0x2 ref 0 00000100:00100000:0.0:1789760710.285784:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760710.286037:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512555392 00000100:00100000:1.0:1789760710.286052:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512555392 00000100:00100000:1.0:1789760710.286120:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 689714595842 time 689793481863 state 0x0 ref 1 00000100:00100000:1.0:1789760710.286138:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 689793742279 time 689793481863 state 0x1 ref 1 00000100:00100000:1.0:1789760710.286169:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 689793742279 time 689850468415 state 0x0 ref 1 00000100:00100000:1.0:1789760710.286183:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febfb897100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512555392:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760710.286251:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22306816, mtime: 0x6aad94c5, atime: 0x0, ctime: 0x6aad94c5, blocks: 0xb007 00010000:00010000:1.0:1789760710.286262:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2d614 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760710.286282:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=689s v=(0 0) 00010000:00010000:1.0:1789760710.286287:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760710.286293:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2d614 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22306816->22310911] (req 22306816->22310911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760710.286311:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2d614 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22306816->22310911] (req 22306816->22310911) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760710.367719:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22306816 -> 22310912 00000001:00080000:0.0:1789760710.367736:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972765 is committed 00010000:00010000:1.0:1789760710.367749:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2d614 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22306816->22310911] (req 22306816->22310911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760710.367766:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2d614 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22306816->22310911] (req 22306816->22310911) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760710.367778:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2d614 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22306816->22310911] (req 22306816->22310911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760710.367786:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760710.367792:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2d614 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22306816->22310911] (req 22306816->22310911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760710.367802:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2d614 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22306816->22310911] (req 22306816->22310911) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760710.367828:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2d614 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22306816->22310911] (req 22306816->22310911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760710.367837:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2d614 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22306816->22310911] (req 22306816->22310911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760710.368106:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febfb897100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512555392:12345-192.168.203.56@tcp:4:dd.0 Request processed in 81918us (82329us total) trans 4294972765 rc 0/0 00000100:00100000:1.0:1789760710.368118:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5476 00000100:00100000:1.0:1789760710.368130:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 689793742279 time 689850468415 state 0x2 ref 0 00000100:00100000:1.0:1789760710.375452:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760710.375565:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512555520 00000100:00100000:0.0:1789760710.375588:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512555520 00000100:00100000:0.0:1789760710.375662:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 689793742279 time 689850468415 state 0x0 ref 1 00000100:00100000:0.0:1789760710.375682:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 689850728831 time 689850468415 state 0x1 ref 1 00000100:00100000:0.0:1789760710.375757:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 689850728831 time 689940056277 state 0x0 ref 1 00000100:00100000:0.0:1789760710.375771:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2fb9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512555520:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760710.375842:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22310912, mtime: 0x6aad94c5, atime: 0x0, ctime: 0x6aad94c5, blocks: 0xb007 00010000:00010000:0.0:1789760710.375852:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2d61b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.375873:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=689s v=(0 0) 00010000:00010000:0.0:1789760710.375878:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760710.375897:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2d61b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22310912->22315007] (req 22310912->22315007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.375921:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2d61b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22310912->22315007] (req 22310912->22315007) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760710.512744:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972766 is committed 00002000:00010000:3.0:1789760710.512963:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22310912 -> 22315008 00010000:00010000:3.0:1789760710.512993:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2d61b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22310912->22315007] (req 22310912->22315007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.513017:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2d61b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22310912->22315007] (req 22310912->22315007) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.513028:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2d61b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22310912->22315007] (req 22310912->22315007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.513037:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760710.513044:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2d61b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22310912->22315007] (req 22310912->22315007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.513055:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2d61b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22310912->22315007] (req 22310912->22315007) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.513090:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2d61b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22310912->22315007] (req 22310912->22315007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.513099:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2d61b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22310912->22315007] (req 22310912->22315007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760710.513227:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2fb9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512555520:12345-192.168.203.56@tcp:4:dd.0 Request processed in 137450us (137784us total) trans 4294972766 rc 0/0 00000100:00100000:3.0:1789760710.513239:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5477 00000100:00100000:3.0:1789760710.513251:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 689850728831 time 689940056277 state 0x2 ref 0 00000100:00100000:2.0:1789760710.517724:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760710.518347:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512555648 00000100:00100000:0.0:1789760710.518360:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512555648 00000100:00100000:0.0:1789760710.518436:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 689850728831 time 689940056277 state 0x0 ref 1 00000100:00100000:0.0:1789760710.518455:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 689940316693 time 689940056277 state 0x1 ref 1 00000100:00100000:0.0:1789760710.518495:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 689940316693 time 690082794685 state 0x0 ref 1 00000100:00100000:0.0:1789760710.518511:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac38b4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512555648:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760710.518581:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22315008, mtime: 0x6aad94c5, atime: 0x0, ctime: 0x6aad94c5, blocks: 0xb007 00010000:00010000:0.0:1789760710.518593:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2d622 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.518613:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=690s v=(0 0) 00010000:00010000:0.0:1789760710.518619:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760710.518626:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2d622 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22315008->22319103] (req 22315008->22319103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.518644:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2d622 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22315008->22319103] (req 22315008->22319103) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760710.583217:0:3310:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8feac2747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3310:1876699540217472:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1789760710.583327:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760710.583395:0:3310:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8febea650380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3310:1876699540217984:0@lo:400:kworker.0 00000100:00100000:1.0:1789760710.583422:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760710.583452:0:3310:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1789760710.585104:0:3311:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8feac2fbaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:6df98d54-eb52-4c5c-8fc9-ee9d74f90f98:3311:1876699540217600:0@lo:400:kworker.0 00000100:00100000:3.0:1789760710.585187:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760710.586068:0:3310:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8febea651500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3310:1876699540217856:0@lo:400:kworker.0 00000100:00100000:1.0:1789760710.586134:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760710.586182:0:3312:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8febc6bd6300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3312:1876699540217728:0@lo:400:kworker.0 00000100:00100000:1.0:1789760710.586217:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760710.586264:0:6706:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540217472 00000100:00100000:1.0:1789760710.586276:0:6706:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540217472 00000100:00100000:1.0:1789760710.586798:0:6706:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 184 00000100:00100000:1.0:1789760710.586822:0:6706:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febea772680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3310:x1876699540217472:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1789760710.586855:0:6706:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 528154624 pending 1703936 free 3835691008 avail 3833593856 00000100:00100000:1.0:1789760710.586914:0:6706:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febea772680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3310:x1876699540217472:12345-0@lo:13:osp-pre-0-0.0 Request processed in 107us (3587us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760710.586922:0:6706:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 184 00000100:00100000:1.0:1789760710.586949:0:6705:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540217984 00000100:00100000:1.0:1789760710.586954:0:6705:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540217984 00000100:00100000:0.0:1789760710.587314:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540217600 00000100:00100000:0.0:1789760710.587319:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540217600 00000100:00100000:0.0:1789760710.587354:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 385 00000100:00100000:0.0:1789760710.587362:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febc3ebe850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:6df98d54-eb52-4c5c-8fc9-ee9d74f90f98+19:3311:x1876699540217600:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760710.587411:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febc3ebe850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:6df98d54-eb52-4c5c-8fc9-ee9d74f90f98+19:3311:x1876699540217600:12345-0@lo:400:kworker.0 Request processed in 50us (2225us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760710.587419:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 385 00000100:00100000:1.0:1789760710.587577:0:6705:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 169 00000100:00100000:1.0:1789760710.587585:0:6705:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febea770380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3310:x1876699540217984:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760710.587732:0:6705:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febea770380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3310:x1876699540217984:12345-0@lo:400:kworker.0 Request processed in 148us (4309us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760710.587740:0:6705:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 169 00000100:00100000:1.0:1789760710.587764:0:5848:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540217856 00000100:00100000:1.0:1789760710.587768:0:5848:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540217856 00000100:00100000:1.0:1789760710.587799:0:5848:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540217728 00000100:00100000:1.0:1789760710.587802:0:5848:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540217728 00000100:00100000:1.0:1789760710.587823:0:5848:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 441 00000100:00100000:1.0:1789760710.587829:0:5848:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febea771c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3310:x1876699540217856:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760710.587862:0:5848:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febea771c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3310:x1876699540217856:12345-0@lo:400:kworker.0 Request processed in 33us (1728us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760710.587869:0:5848:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 441 00000100:00100000:1.0:1789760710.587880:0:5848:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 442 00000100:00100000:1.0:1789760710.587886:0:5848:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac38b3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3312:x1876699540217728:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760710.587920:0:5848:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac38b3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3312:x1876699540217728:12345-0@lo:400:kworker.0 Request processed in 34us (1701us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760710.587926:0:5848:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 442 00000100:00080000:1.0:1789760710.587947:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:1.0:1789760710.587953:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1789760710.595952:0:3311:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8feac2fbaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:6df98d54-eb52-4c5c-8fc9-ee9d74f90f98:3311:1876699540217600:0@lo:400:kworker.0 00000100:00100000:3.0:1789760710.595991:0:3311:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8feac4217100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3311:1876699540218112:0@lo:400:kworker.0 00000100:00100000:3.0:1789760710.596089:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760710.597954:0:3310:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8febea651500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3310:1876699540217856:0@lo:400:kworker.0 00000100:00100000:1.0:1789760710.598952:0:3310:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8feac2747800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3310:1876699540217472:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1789760710.598977:0:3310:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8febea650380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3310:1876699540217984:0@lo:400:kworker.0 00000100:00100000:1.0:1789760710.599060:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540218112 00000100:00100000:1.0:1789760710.599068:0:5847:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540218112 00000100:00100000:1.0:1789760710.599155:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 443 00000100:00100000:1.0:1789760710.599166:0:5847:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2744000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3311:x1876699540218112:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760710.599286:0:5847:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2744000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3311:x1876699540218112:12345-0@lo:400:kworker.0 Request processed in 121us (3197us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760710.599750:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 443 00000100:00100000:1.0:1789760710.599821:0:3312:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8febc6bd6300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3312:1876699540217728:0@lo:400:kworker.0 00000100:00100000:1.0:1789760710.599835:0:3312:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8febea771880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3312:1876699540218240:0@lo:400:kworker.0 00000100:00100000:1.0:1789760710.599886:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760710.600018:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540218240 00000100:00100000:1.0:1789760710.600023:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540218240 00000100:00100000:1.0:1789760710.600070:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 170 00000100:00100000:1.0:1789760710.600080:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac38b3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3312:x1876699540218240:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760710.600127:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac38b3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3312:x1876699540218240:12345-0@lo:400:kworker.0 Request processed in 49us (241us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760710.600134:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 170 00000100:00080000:1.0:1789760710.600160:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:1.0:1789760710.600166:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1789760710.600184:0:3312:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8febea771880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3312:1876699540218240:0@lo:400:kworker.0 00000100:00100000:3.0:1789760710.600849:0:3311:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8feac4217100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3311:1876699540218112:0@lo:400:kworker.0 00000001:00080000:2.0:1789760710.620807:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972767 is committed 00002000:00010000:3.0:1789760710.621018:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22315008 -> 22319104 00002000:00010000:3.0:1789760710.621025:0:6710:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760709 -> 1789760710 00002000:00010000:3.0:1789760710.621029:0:6710:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760709 -> 1789760710 00010000:00010000:3.0:1789760710.621048:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2d622 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22315008->22319103] (req 22315008->22319103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.621062:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2d622 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22315008->22319103] (req 22315008->22319103) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.621073:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2d622 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22315008->22319103] (req 22315008->22319103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.621082:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760710.621089:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2d622 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22315008->22319103] (req 22315008->22319103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.621100:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2d622 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22315008->22319103] (req 22315008->22319103) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.621127:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2d622 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22315008->22319103] (req 22315008->22319103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.621136:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2d622 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22315008->22319103] (req 22315008->22319103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760710.621444:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac38b4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512555648:12345-192.168.203.56@tcp:4:dd.0 Request processed in 102930us (103758us total) trans 4294972767 rc 0/0 00000100:00100000:3.0:1789760710.625822:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5478 00000100:00100000:3.0:1789760710.625846:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 689940316693 time 690082794685 state 0x2 ref 0 00000100:00100000:2.0:1789760710.626441:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760710.626581:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512555776 00000100:00100000:0.0:1789760710.626592:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512555776 00000100:00100000:0.0:1789760710.626661:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 689940316693 time 690082794685 state 0x0 ref 1 00000100:00100000:0.0:1789760710.626678:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 690083055101 time 690082794685 state 0x1 ref 1 00000100:00100000:0.0:1789760710.626740:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 690083055101 time 690191039287 state 0x0 ref 1 00000100:00100000:0.0:1789760710.626754:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac38b6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512555776:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760710.626828:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22319104, mtime: 0x6aad94c6, atime: 0x0, ctime: 0x6aad94c6, blocks: 0xb007 00010000:00010000:0.0:1789760710.626839:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2d629 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.626859:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=690s v=(0 0) 00010000:00010000:0.0:1789760710.626865:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760710.626873:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2d629 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22319104->22323199] (req 22319104->22323199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.626895:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2d629 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22319104->22323199] (req 22319104->22323199) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760710.676355:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972768 is committed 00002000:00010000:1.0:1789760710.676680:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22319104 -> 22323200 00010000:00010000:1.0:1789760710.676724:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2d629 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22319104->22323199] (req 22319104->22323199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760710.676742:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2d629 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22319104->22323199] (req 22319104->22323199) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760710.676754:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2d629 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22319104->22323199] (req 22319104->22323199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760710.676763:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760710.676769:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2d629 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22319104->22323199] (req 22319104->22323199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760710.676779:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2d629 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22319104->22323199] (req 22319104->22323199) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760710.676806:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2d629 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22319104->22323199] (req 22319104->22323199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760710.676814:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2d629 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22319104->22323199] (req 22319104->22323199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760710.676943:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac38b6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512555776:12345-192.168.203.56@tcp:4:dd.0 Request processed in 50183us (50499us total) trans 4294972768 rc 0/0 00000100:00100000:1.0:1789760710.676956:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5479 00000100:00100000:1.0:1789760710.676968:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 690083055101 time 690191039287 state 0x2 ref 0 00000100:00100000:2.0:1789760710.686505:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760710.686678:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512555904 00000100:00100000:3.0:1789760710.686693:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512555904 00000100:00100000:3.0:1789760710.686808:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 690083055101 time 690191039287 state 0x0 ref 1 00000100:00100000:3.0:1789760710.686828:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 690191299703 time 690191039287 state 0x1 ref 1 00000100:00100000:3.0:1789760710.686912:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 690191299703 time 690251209481 state 0x0 ref 1 00000100:00100000:3.0:1789760710.686927:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac38b5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512555904:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760710.687012:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22323200, mtime: 0x6aad94c6, atime: 0x0, ctime: 0x6aad94c6, blocks: 0xb007 00010000:00010000:3.0:1789760710.687023:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2d630 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.687043:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=690s v=(0 0) 00010000:00010000:3.0:1789760710.687049:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760710.687057:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2d630 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22323200->22327295] (req 22323200->22327295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.687073:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2d630 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22323200->22327295] (req 22323200->22327295) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760710.740773:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22323200 -> 22327296 00010000:00010000:3.0:1789760710.740801:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2d630 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22323200->22327295] (req 22323200->22327295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.740816:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2d630 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22323200->22327295] (req 22323200->22327295) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.740826:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2d630 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22323200->22327295] (req 22323200->22327295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.740834:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760710.740840:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2d630 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22323200->22327295] (req 22323200->22327295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.740849:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2d630 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22323200->22327295] (req 22323200->22327295) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.740871:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2d630 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22323200->22327295] (req 22323200->22327295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.740879:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2d630 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22323200->22327295] (req 22323200->22327295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760710.742640:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac38b5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2361:x1876699512555904:12345-192.168.203.56@tcp:4:dd.0 Request processed in 55698us (56133us total) trans 4294972769 rc 0/0 00000100:00100000:3.0:1789760710.742657:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5480 00000100:00100000:3.0:1789760710.742668:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 690191299703 time 690251209481 state 0x2 ref 0 00000001:00080000:1.0:1789760710.742885:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972769 is committed 00000100:00100000:2.0:1789760710.757876:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760710.758118:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512556032 00000100:00100000:3.0:1789760710.758132:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512556032 00000100:00100000:3.0:1789760710.758209:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 690191299703 time 690251209481 state 0x0 ref 1 00000100:00100000:3.0:1789760710.758230:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 690251469897 time 690251209481 state 0x1 ref 1 00000100:00100000:3.0:1789760710.758277:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 690251469897 time 690322575851 state 0x0 ref 1 00000100:00100000:3.0:1789760710.758292:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac3352300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512556032:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760710.758362:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22327296, mtime: 0x6aad94c6, atime: 0x0, ctime: 0x6aad94c6, blocks: 0xb007 00010000:00010000:3.0:1789760710.758373:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2d637 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.758393:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=690s v=(0 0) 00010000:00010000:3.0:1789760710.758398:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760710.758406:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2d637 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22327296->22331391] (req 22327296->22331391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.758422:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2d637 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22327296->22331391] (req 22327296->22331391) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760710.796848:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972770 is committed 00002000:00010000:2.0:1789760710.797057:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22327296 -> 22331392 00010000:00010000:2.0:1789760710.797078:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2d637 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22327296->22331391] (req 22327296->22331391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.797095:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2d637 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22327296->22331391] (req 22327296->22331391) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.797111:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2d637 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22327296->22331391] (req 22327296->22331391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.797120:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760710.797126:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2d637 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22327296->22331391] (req 22327296->22331391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.797137:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2d637 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22327296->22331391] (req 22327296->22331391) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.797163:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2d637 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22327296->22331391] (req 22327296->22331391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.797172:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2d637 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22327296->22331391] (req 22327296->22331391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760710.797415:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac3352300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512556032:12345-192.168.203.56@tcp:4:dd.0 Request processed in 39120us (39548us total) trans 4294972770 rc 0/0 00000100:00100000:2.0:1789760710.797428:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5481 00000100:00100000:2.0:1789760710.797439:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 690251469897 time 690322575851 state 0x2 ref 0 00000100:00100000:2.0:1789760710.808414:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760710.811833:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512556160 00000100:00100000:1.0:1789760710.811854:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512556160 00000100:00100000:2.0:1789760710.812056:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 690251469897 time 690322575851 state 0x0 ref 1 00000100:00100000:2.0:1789760710.812076:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 690322836267 time 690322575851 state 0x1 ref 1 00000100:00100000:2.0:1789760710.812333:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 690322836267 time 690376630904 state 0x0 ref 1 00000100:00100000:2.0:1789760710.812350:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac3353800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512556160:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760710.812419:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22331392, mtime: 0x6aad94c6, atime: 0x0, ctime: 0x6aad94c6, blocks: 0xb007 00010000:00010000:2.0:1789760710.812429:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2d63e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.812448:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=690s v=(0 0) 00010000:00010000:2.0:1789760710.812453:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760710.812526:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2d63e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22331392->22335487] (req 22331392->22335487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.812544:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2d63e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22331392->22335487] (req 22331392->22335487) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760710.878494:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972771 is committed 00002000:00010000:0.0:1789760710.878795:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22331392 -> 22335488 00010000:00010000:0.0:1789760710.878814:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2d63e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22331392->22335487] (req 22331392->22335487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.878830:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2d63e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22331392->22335487] (req 22331392->22335487) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.878840:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2d63e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22331392->22335487] (req 22331392->22335487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.878847:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760710.878854:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2d63e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22331392->22335487] (req 22331392->22335487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.878864:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2d63e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22331392->22335487] (req 22331392->22335487) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.878887:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2d63e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22331392->22335487] (req 22331392->22335487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.878895:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2d63e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22331392->22335487] (req 22331392->22335487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760710.879118:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac3353800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512556160:12345-192.168.203.56@tcp:4:dd.0 Request processed in 66764us (70711us total) trans 4294972771 rc 0/0 00000100:00100000:0.0:1789760710.879190:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5482 00000100:00100000:0.0:1789760710.879203:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 690322836267 time 690376630904 state 0x2 ref 0 00000100:00100000:1.0:1789760710.887909:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760710.888112:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512556288 00000100:00100000:3.0:1789760710.888136:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512556288 00000100:00100000:3.0:1789760710.888212:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 690322836267 time 690376630904 state 0x0 ref 1 00000100:00100000:3.0:1789760710.888242:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 690376891320 time 690376630904 state 0x1 ref 1 00000100:00100000:3.0:1789760710.888317:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 690376891320 time 690452616536 state 0x0 ref 1 00000100:00100000:3.0:1789760710.888331:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac38b0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512556288:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760710.888395:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22335488, mtime: 0x6aad94c6, atime: 0x0, ctime: 0x6aad94c6, blocks: 0xb007 00010000:00010000:3.0:1789760710.888407:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2d645 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.888427:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=690s v=(0 0) 00010000:00010000:3.0:1789760710.888432:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760710.888440:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2d645 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22335488->22339583] (req 22335488->22339583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.888465:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2d645 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22335488->22339583] (req 22335488->22339583) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760711.006313:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972772 is committed 00002000:00010000:2.0:1789760711.006352:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22335488 -> 22339584 00010000:00010000:2.0:1789760711.006375:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2d645 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22335488->22339583] (req 22335488->22339583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760711.006395:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2d645 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22335488->22339583] (req 22335488->22339583) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760711.006406:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2d645 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22335488->22339583] (req 22335488->22339583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760711.006415:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760711.006421:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2d645 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22335488->22339583] (req 22335488->22339583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760711.006431:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2d645 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22335488->22339583] (req 22335488->22339583) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760711.006455:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2d645 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22335488->22339583] (req 22335488->22339583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760711.006464:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2d645 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22335488->22339583] (req 22335488->22339583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760711.006591:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac38b0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512556288:12345-192.168.203.56@tcp:4:dd.0 Request processed in 118255us (118694us total) trans 4294972772 rc 0/0 00000100:00100000:2.0:1789760711.006602:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5483 00000100:00100000:2.0:1789760711.006613:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 690376891320 time 690452616536 state 0x2 ref 0 00000100:00100000:2.0:1789760711.011445:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760711.011597:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512556416 00000100:00100000:0.0:1789760711.011610:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512556416 00000100:00100000:0.0:1789760711.011714:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 690376891320 time 690452616536 state 0x0 ref 1 00000100:00100000:0.0:1789760711.011733:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 690452876952 time 690452616536 state 0x1 ref 1 00000100:00100000:0.0:1789760711.011771:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 690452876952 time 690576070049 state 0x0 ref 1 00000100:00100000:0.0:1789760711.011786:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac3352a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512556416:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760711.011851:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22339584, mtime: 0x6aad94c6, atime: 0x0, ctime: 0x6aad94c6, blocks: 0xb007 00010000:00010000:0.0:1789760711.011861:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2d64c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760711.011881:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=690s v=(0 0) 00010000:00010000:0.0:1789760711.011886:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760711.011894:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2d64c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22339584->22343679] (req 22339584->22343679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760711.011910:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2d64c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22339584->22343679] (req 22339584->22343679) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:1.0:1789760711.013403:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:1.0:1789760711.028022:0:5836:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:1.0:1789760711.063545:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22339584 -> 22343680 00010000:00010000:1.0:1789760711.063581:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2d64c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22339584->22343679] (req 22339584->22343679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.063604:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2d64c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22339584->22343679] (req 22339584->22343679) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.063615:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2d64c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22339584->22343679] (req 22339584->22343679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.063623:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760711.063629:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2d64c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22339584->22343679] (req 22339584->22343679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.063639:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2d64c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22339584->22343679] (req 22339584->22343679) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.063662:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2d64c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22339584->22343679] (req 22339584->22343679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.063669:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2d64c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22339584->22343679] (req 22339584->22343679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760711.063821:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac3352a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2360:x1876699512556416:12345-192.168.203.56@tcp:4:dd.0 Request processed in 52031us (52383us total) trans 4294972773 rc 0/0 00000100:00100000:1.0:1789760711.063833:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5484 00000100:00100000:1.0:1789760711.063845:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 690452876952 time 690576070049 state 0x2 ref 0 00000001:00080000:3.0:1789760711.064592:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972773 is committed 00000100:00100000:0.0:1789760711.068995:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760711.069349:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512556544 00000100:00100000:2.0:1789760711.069363:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512556544 00000100:00100000:2.0:1789760711.069496:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 690452876952 time 690576070049 state 0x0 ref 1 00000100:00100000:2.0:1789760711.069515:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 690576330465 time 690576070049 state 0x1 ref 1 00000100:00100000:2.0:1789760711.069552:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 690576330465 time 690633851012 state 0x0 ref 1 00000100:00100000:2.0:1789760711.070131:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febfb895880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512556544:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760711.070449:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22343680, mtime: 0x6aad94c6, atime: 0x0, ctime: 0x6aad94c6, blocks: 0xb007 00010000:00010000:2.0:1789760711.070462:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2d653 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760711.070482:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=690s v=(0 0) 00010000:00010000:2.0:1789760711.070488:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760711.070496:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2d653 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22343680->22347775] (req 22343680->22347775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760711.070513:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2d653 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22343680->22347775] (req 22343680->22347775) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760711.091048:0:3309:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8feac3353480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3309:1876699540218368:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789760711.091142:0:3309:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760711.091363:0:6706:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540218368 00000100:00100000:1.0:1789760711.091374:0:6706:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540218368 00000100:00100000:1.0:1789760711.091439:0:6706:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 185 00000100:00100000:1.0:1789760711.091451:0:6706:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac3351180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3309:x1876699540218368:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1789760711.091507:0:6706:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3859808256 avail 3857711104 00000100:00100000:1.0:1789760711.091591:0:6706:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac3351180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3309:x1876699540218368:12345-0@lo:13:osp-pre-1-0.0 Request processed in 141us (450us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760711.091600:0:6706:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 185 00000100:00100000:0.0:1789760711.091857:0:3309:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8feac3353480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3309:1876699540218368:0@lo:13:osp-pre-1-0.0 00000001:00080000:3.0:1789760711.102342:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972774 is committed 00002000:00010000:1.0:1789760711.102369:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22343680 -> 22347776 00010000:00010000:1.0:1789760711.102390:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2d653 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22343680->22347775] (req 22343680->22347775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.102406:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2d653 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22343680->22347775] (req 22343680->22347775) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.102416:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2d653 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22343680->22347775] (req 22343680->22347775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.102424:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760711.102430:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2d653 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22343680->22347775] (req 22343680->22347775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.102441:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2d653 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22343680->22347775] (req 22343680->22347775) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.102466:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2d653 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22343680->22347775] (req 22343680->22347775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.102472:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2d653 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22343680->22347775] (req 22343680->22347775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760711.102596:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febfb895880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512556544:12345-192.168.203.56@tcp:4:dd.0 Request processed in 33025us (33609us total) trans 4294972774 rc 0/0 00000100:00100000:1.0:1789760711.102967:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5485 00000100:00100000:1.0:1789760711.102980:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 690576330465 time 690633851012 state 0x2 ref 0 00000100:00100000:1.0:1789760711.107506:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760711.107904:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512556672 00000100:00100000:2.0:1789760711.107918:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512556672 00000100:00100000:2.0:1789760711.108094:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 690576330465 time 690633851012 state 0x0 ref 1 00000100:00100000:2.0:1789760711.108112:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 690634111428 time 690633851012 state 0x1 ref 1 00000100:00100000:2.0:1789760711.108270:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 690634111428 time 690672567547 state 0x0 ref 1 00000100:00100000:2.0:1789760711.108286:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac38b3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512556672:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760711.108372:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22347776, mtime: 0x6aad94c6, atime: 0x0, ctime: 0x6aad94c6, blocks: 0xb007 00010000:00010000:2.0:1789760711.108382:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2d65a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760711.108403:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=690s v=(0 0) 00010000:00010000:2.0:1789760711.108408:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760711.108416:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2d65a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22347776->22351871] (req 22347776->22351871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760711.108432:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2d65a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22347776->22351871] (req 22347776->22351871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760711.170769:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972775 is committed 00002000:00010000:1.0:1789760711.170946:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22347776 -> 22351872 00010000:00010000:1.0:1789760711.170972:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2d65a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22347776->22351871] (req 22347776->22351871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.170986:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2d65a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22347776->22351871] (req 22347776->22351871) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.170996:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2d65a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22347776->22351871] (req 22347776->22351871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.171010:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760711.171015:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2d65a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22347776->22351871] (req 22347776->22351871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.171025:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2d65a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22347776->22351871] (req 22347776->22351871) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.171045:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2d65a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22347776->22351871] (req 22347776->22351871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.171052:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2d65a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22347776->22351871] (req 22347776->22351871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760711.171143:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac38b3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512556672:12345-192.168.203.56@tcp:4:dd.0 Request processed in 62856us (63649us total) trans 4294972775 rc 0/0 00000100:00100000:1.0:1789760711.171153:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5486 00000100:00100000:1.0:1789760711.171163:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 690634111428 time 690672567547 state 0x2 ref 0 00000100:00100000:3.0:1789760711.174266:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760711.175995:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512556800 00000100:00100000:1.0:1789760711.176009:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512556800 00000100:00100000:1.0:1789760711.176080:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 690634111428 time 690672567547 state 0x0 ref 1 00000100:00100000:1.0:1789760711.176097:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 690672827963 time 690672567547 state 0x1 ref 1 00000100:00100000:1.0:1789760711.176131:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 690672827963 time 690740430914 state 0x0 ref 1 00000100:00100000:1.0:1789760711.176147:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2747800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512556800:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760711.176195:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22351872, mtime: 0x6aad94c6, atime: 0x0, ctime: 0x6aad94c6, blocks: 0xb007 00010000:00010000:1.0:1789760711.176203:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2d661 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.176220:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=690s v=(0 0) 00010000:00010000:1.0:1789760711.176225:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760711.176244:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2d661 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22351872->22355967] (req 22351872->22355967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.176259:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2d661 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22351872->22355967] (req 22351872->22355967) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760711.217499:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22351872 -> 22355968 00010000:00010000:1.0:1789760711.217520:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2d661 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22351872->22355967] (req 22351872->22355967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.217533:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2d661 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22351872->22355967] (req 22351872->22355967) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.217543:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2d661 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22351872->22355967] (req 22351872->22355967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.217550:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760711.217555:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2d661 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22351872->22355967] (req 22351872->22355967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.217596:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2d661 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22351872->22355967] (req 22351872->22355967) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.217616:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2d661 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22351872->22355967] (req 22351872->22355967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.217624:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2d661 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22351872->22355967] (req 22351872->22355967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760711.217743:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2747800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2361:x1876699512556800:12345-192.168.203.56@tcp:4:dd.0 Request processed in 41594us (43483us total) trans 4294972776 rc 0/0 00000100:00100000:1.0:1789760711.217754:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5487 00000100:00100000:1.0:1789760711.217763:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 690672827963 time 690740430914 state 0x2 ref 0 00000001:00080000:2.0:1789760711.218403:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972776 is committed 00000100:00100000:0.0:1789760711.225322:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760711.225461:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512556928 00000100:00100000:1.0:1789760711.225471:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512556928 00000100:00100000:1.0:1789760711.225537:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 690672827963 time 690740430914 state 0x0 ref 1 00000100:00100000:1.0:1789760711.225554:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 690740691330 time 690740430914 state 0x1 ref 1 00000100:00100000:1.0:1789760711.225593:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 690740691330 time 690789892246 state 0x0 ref 1 00000100:00100000:1.0:1789760711.225606:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febd1362a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512556928:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760711.225659:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22355968, mtime: 0x6aad94c6, atime: 0x0, ctime: 0x6aad94c6, blocks: 0xb007 00010000:00010000:1.0:1789760711.225668:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2d668 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.225685:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=690s v=(0 0) 00010000:00010000:1.0:1789760711.225689:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760711.225723:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2d668 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22355968->22360063] (req 22355968->22360063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.225738:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2d668 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22355968->22360063] (req 22355968->22360063) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760711.285859:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972777 is committed 00002000:00010000:1.0:1789760711.286032:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22355968 -> 22360064 00010000:00010000:1.0:1789760711.286048:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2d668 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22355968->22360063] (req 22355968->22360063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.286130:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2d668 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22355968->22360063] (req 22355968->22360063) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.286141:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2d668 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22355968->22360063] (req 22355968->22360063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.286148:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760711.286153:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2d668 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22355968->22360063] (req 22355968->22360063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.286162:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2d668 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22355968->22360063] (req 22355968->22360063) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.286181:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2d668 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22355968->22360063] (req 22355968->22360063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.286189:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2d668 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22355968->22360063] (req 22355968->22360063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760711.286382:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febd1362a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512556928:12345-192.168.203.56@tcp:4:dd.0 Request processed in 60774us (61065us total) trans 4294972777 rc 0/0 00000100:00100000:1.0:1789760711.286393:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5488 00000100:00100000:1.0:1789760711.286403:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 690740691330 time 690789892246 state 0x2 ref 0 00000100:00100000:0.0:1789760711.305330:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760711.309643:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512557056 00000100:00100000:1.0:1789760711.309745:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512557056 00000100:00100000:1.0:1789760711.309876:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 690740691330 time 690789892246 state 0x0 ref 1 00000100:00100000:1.0:1789760711.309892:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 690790152662 time 690789892246 state 0x1 ref 1 00000100:00100000:1.0:1789760711.309978:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 690790152662 time 690874277323 state 0x0 ref 1 00000100:00100000:1.0:1789760711.309992:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febd1362300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512557056:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760711.310041:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22360064, mtime: 0x6aad94c6, atime: 0x0, ctime: 0x6aad94c6, blocks: 0xb007 00010000:00010000:1.0:1789760711.310049:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2d66f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.310077:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=690s v=(0 0) 00010000:00010000:1.0:1789760711.310082:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760711.310089:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2d66f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22360064->22364159] (req 22360064->22364159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.310108:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2d66f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22360064->22364159] (req 22360064->22364159) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760711.394910:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972778 is committed 00002000:00010000:1.0:1789760711.395099:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22360064 -> 22364160 00010000:00010000:1.0:1789760711.395119:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2d66f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22360064->22364159] (req 22360064->22364159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.395135:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2d66f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22360064->22364159] (req 22360064->22364159) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.395145:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2d66f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22360064->22364159] (req 22360064->22364159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.395154:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760711.395160:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2d66f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22360064->22364159] (req 22360064->22364159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.395170:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2d66f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22360064->22364159] (req 22360064->22364159) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.395195:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2d66f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22360064->22364159] (req 22360064->22364159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.395204:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2d66f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22360064->22364159] (req 22360064->22364159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760711.395512:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febd1362300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512557056:12345-192.168.203.56@tcp:4:dd.0 Request processed in 85513us (90182us total) trans 4294972778 rc 0/0 00000100:00100000:1.0:1789760711.395527:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5489 00000100:00100000:1.0:1789760711.395539:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 690790152662 time 690874277323 state 0x2 ref 0 00000100:00100000:0.0:1789760711.401799:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760711.402103:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512557184 00000100:00100000:1.0:1789760711.402115:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512557184 00000100:00100000:1.0:1789760711.402187:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 690790152662 time 690874277323 state 0x0 ref 1 00000100:00100000:1.0:1789760711.402205:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 690874537739 time 690874277323 state 0x1 ref 1 00000100:00100000:1.0:1789760711.402238:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 690874537739 time 690966537779 state 0x0 ref 1 00000100:00100000:1.0:1789760711.402252:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac437e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512557184:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760711.402325:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22364160, mtime: 0x6aad94c6, atime: 0x0, ctime: 0x6aad94c6, blocks: 0xb007 00010000:00010000:1.0:1789760711.402336:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2d676 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.402356:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=690s v=(0 0) 00010000:00010000:1.0:1789760711.402362:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760711.402369:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2d676 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22364160->22368255] (req 22364160->22368255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.402384:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2d676 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22364160->22368255] (req 22364160->22368255) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760711.463166:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22364160 -> 22368256 00010000:00010000:1.0:1789760711.463197:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2d676 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22364160->22368255] (req 22364160->22368255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.463215:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2d676 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22364160->22368255] (req 22364160->22368255) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.463226:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2d676 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22364160->22368255] (req 22364160->22368255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.463248:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760711.463254:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2d676 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22364160->22368255] (req 22364160->22368255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.463265:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2d676 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22364160->22368255] (req 22364160->22368255) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.463296:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2d676 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22364160->22368255] (req 22364160->22368255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.463305:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2d676 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22364160->22368255] (req 22364160->22368255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760711.463433:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac437e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2360:x1876699512557184:12345-192.168.203.56@tcp:4:dd.0 Request processed in 61175us (61642us total) trans 4294972779 rc 0/0 00000100:00100000:1.0:1789760711.463450:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5490 00000100:00100000:1.0:1789760711.463462:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 690874537739 time 690966537779 state 0x2 ref 0 00000001:00080000:2.0:1789760711.465064:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972779 is committed 00000100:00100000:0.0:1789760711.466496:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760711.466593:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512557312 00000100:00100000:1.0:1789760711.466606:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512557312 00000100:00100000:1.0:1789760711.466674:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 690874537739 time 690966537779 state 0x0 ref 1 00000100:00100000:1.0:1789760711.466693:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 690966798195 time 690966537779 state 0x1 ref 1 00000100:00100000:1.0:1789760711.466799:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 690966798195 time 691031097906 state 0x0 ref 1 00000100:00100000:1.0:1789760711.466810:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac437e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512557312:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760711.466865:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22368256, mtime: 0x6aad94c6, atime: 0x0, ctime: 0x6aad94c6, blocks: 0xb007 00010000:00010000:1.0:1789760711.466873:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2d67d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.466890:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=691s v=(0 0) 00010000:00010000:1.0:1789760711.466893:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760711.466899:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2d67d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22368256->22372351] (req 22368256->22372351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.466911:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2d67d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22368256->22372351] (req 22368256->22372351) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760711.518021:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22368256 -> 22372352 00010000:00010000:1.0:1789760711.518049:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2d67d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22368256->22372351] (req 22368256->22372351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.518067:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2d67d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22368256->22372351] (req 22368256->22372351) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.518078:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2d67d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22368256->22372351] (req 22368256->22372351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.518087:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760711.518093:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2d67d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22368256->22372351] (req 22368256->22372351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.518104:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2d67d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22368256->22372351] (req 22368256->22372351) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.518128:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2d67d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22368256->22372351] (req 22368256->22372351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.518137:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2d67d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22368256->22372351] (req 22368256->22372351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760711.518382:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac437e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2361:x1876699512557312:12345-192.168.203.56@tcp:4:dd.0 Request processed in 51566us (51888us total) trans 4294972780 rc 0/0 00000100:00100000:1.0:1789760711.518395:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5491 00000100:00100000:1.0:1789760711.518407:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 690966798195 time 691031097906 state 0x2 ref 0 00000001:00080000:3.0:1789760711.518550:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972780 is committed 00000100:00100000:0.0:1789760711.522399:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760711.522536:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512557440 00000100:00100000:1.0:1789760711.522551:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512557440 00000100:00100000:1.0:1789760711.522630:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 690966798195 time 691031097906 state 0x0 ref 1 00000100:00100000:1.0:1789760711.522650:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 691031358322 time 691031097906 state 0x1 ref 1 00000100:00100000:1.0:1789760711.522767:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 691031358322 time 691087065638 state 0x0 ref 1 00000100:00100000:1.0:1789760711.522782:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac437ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512557440:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760711.522846:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22372352, mtime: 0x6aad94c6, atime: 0x0, ctime: 0x6aad94c6, blocks: 0xb007 00010000:00010000:1.0:1789760711.522857:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2d684 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.522875:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=691s v=(0 0) 00010000:00010000:1.0:1789760711.522880:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760711.522887:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2d684 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22372352->22376447] (req 22372352->22376447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.522903:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2d684 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22372352->22376447] (req 22372352->22376447) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760711.552499:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972781 is committed 00002000:00010000:1.0:1789760711.552635:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22372352 -> 22376448 00002000:00010000:1.0:1789760711.552640:0:13063:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760710 -> 1789760711 00002000:00010000:1.0:1789760711.552643:0:13063:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760710 -> 1789760711 00010000:00010000:1.0:1789760711.552660:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2d684 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22372352->22376447] (req 22372352->22376447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.552674:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2d684 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22372352->22376447] (req 22372352->22376447) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.552685:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2d684 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22372352->22376447] (req 22372352->22376447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.552724:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760711.552731:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2d684 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22372352->22376447] (req 22372352->22376447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.552742:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2d684 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22372352->22376447] (req 22372352->22376447) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.552767:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2d684 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22372352->22376447] (req 22372352->22376447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.552774:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2d684 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22372352->22376447] (req 22372352->22376447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760711.552871:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac437ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512557440:12345-192.168.203.56@tcp:4:dd.0 Request processed in 30086us (30481us total) trans 4294972781 rc 0/0 00000100:00100000:1.0:1789760711.552880:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5492 00000100:00100000:1.0:1789760711.552890:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 691031358322 time 691087065638 state 0x2 ref 0 00000100:00100000:0.0:1789760711.556203:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760711.556616:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512557568 00000100:00100000:1.0:1789760711.556629:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512557568 00000100:00100000:1.0:1789760711.556796:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 691031358322 time 691087065638 state 0x0 ref 1 00000100:00100000:1.0:1789760711.556816:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 691087326054 time 691087065638 state 0x1 ref 1 00000100:00100000:1.0:1789760711.556891:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 691087326054 time 691121189877 state 0x0 ref 1 00000100:00100000:1.0:1789760711.556908:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac437df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512557568:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760711.556992:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22376448, mtime: 0x6aad94c7, atime: 0x0, ctime: 0x6aad94c7, blocks: 0xb007 00010000:00010000:1.0:1789760711.557004:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2d68b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.557024:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=691s v=(0 0) 00010000:00010000:1.0:1789760711.557029:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760711.557037:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2d68b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22376448->22380543] (req 22376448->22380543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.557054:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2d68b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22376448->22380543] (req 22376448->22380543) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760711.581199:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972782 is committed 00002000:00010000:1.0:1789760711.581392:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22376448 -> 22380544 00010000:00010000:1.0:1789760711.581415:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2d68b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22376448->22380543] (req 22376448->22380543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.581431:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2d68b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22376448->22380543] (req 22376448->22380543) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.581441:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2d68b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22376448->22380543] (req 22376448->22380543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.581448:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760711.581454:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2d68b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22376448->22380543] (req 22376448->22380543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.581464:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2d68b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22376448->22380543] (req 22376448->22380543) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.581488:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2d68b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22376448->22380543] (req 22376448->22380543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.581495:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2d68b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22376448->22380543] (req 22376448->22380543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760711.581670:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac437df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512557568:12345-192.168.203.56@tcp:4:dd.0 Request processed in 24756us (25471us total) trans 4294972782 rc 0/0 00000100:00100000:1.0:1789760711.581684:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5493 00000100:00100000:1.0:1789760711.581746:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 691087326054 time 691121189877 state 0x2 ref 0 00000100:00100000:0.0:1789760711.585027:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760711.585135:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512557696 00000100:00100000:1.0:1789760711.585148:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512557696 00000100:00100000:1.0:1789760711.585222:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 691087326054 time 691121189877 state 0x0 ref 1 00000100:00100000:1.0:1789760711.585284:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 691121450293 time 691121189877 state 0x1 ref 1 00000100:00100000:1.0:1789760711.585315:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 691121450293 time 691149614608 state 0x0 ref 1 00000100:00100000:1.0:1789760711.585329:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac437fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512557696:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760711.585395:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22380544, mtime: 0x6aad94c7, atime: 0x0, ctime: 0x6aad94c7, blocks: 0xb007 00010000:00010000:1.0:1789760711.585406:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2d692 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.585426:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=691s v=(0 0) 00010000:00010000:1.0:1789760711.585432:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760711.585440:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2d692 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22380544->22384639] (req 22380544->22384639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.585457:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2d692 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22380544->22384639] (req 22380544->22384639) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760711.612552:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22380544 -> 22384640 00010000:00010000:1.0:1789760711.612590:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2d692 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22380544->22384639] (req 22380544->22384639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.612605:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2d692 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22380544->22384639] (req 22380544->22384639) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.612681:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2d692 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22380544->22384639] (req 22380544->22384639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.612688:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760711.612726:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2d692 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22380544->22384639] (req 22380544->22384639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.612738:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2d692 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22380544->22384639] (req 22380544->22384639) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.612778:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2d692 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22380544->22384639] (req 22380544->22384639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.612787:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2d692 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22380544->22384639] (req 22380544->22384639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760711.612901:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac437fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2359:x1876699512557696:12345-192.168.203.56@tcp:4:dd.0 Request processed in 27568us (27942us total) trans 4294972783 rc 0/0 00000100:00100000:1.0:1789760711.612914:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5494 00000100:00100000:1.0:1789760711.612926:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 691121450293 time 691149614608 state 0x2 ref 0 00000001:00080000:3.0:1789760711.613132:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972783 is committed 00000100:00100000:0.0:1789760711.616827:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760711.616936:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512557824 00000100:00100000:1.0:1789760711.616948:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512557824 00000100:00100000:1.0:1789760711.617028:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 691121450293 time 691149614608 state 0x0 ref 1 00000100:00100000:1.0:1789760711.617047:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 691149875024 time 691149614608 state 0x1 ref 1 00000100:00100000:1.0:1789760711.617296:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 691149875024 time 691181371871 state 0x0 ref 1 00000100:00100000:1.0:1789760711.617314:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac437f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512557824:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760711.617379:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22384640, mtime: 0x6aad94c7, atime: 0x0, ctime: 0x6aad94c7, blocks: 0xb007 00010000:00010000:1.0:1789760711.617389:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2d699 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.617405:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=691s v=(0 0) 00010000:00010000:1.0:1789760711.617410:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760711.617416:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2d699 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22384640->22388735] (req 22384640->22388735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.617429:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2d699 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22384640->22388735] (req 22384640->22388735) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760711.686854:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972784 is committed 00002000:00010000:2.0:1789760711.687034:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22384640 -> 22388736 00010000:00010000:2.0:1789760711.687053:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2d699 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22384640->22388735] (req 22384640->22388735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760711.687068:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2d699 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22384640->22388735] (req 22384640->22388735) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760711.687087:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2d699 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22384640->22388735] (req 22384640->22388735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760711.687095:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760711.687100:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2d699 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22384640->22388735] (req 22384640->22388735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760711.687117:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2d699 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22384640->22388735] (req 22384640->22388735) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760711.687138:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2d699 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22384640->22388735] (req 22384640->22388735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760711.687145:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2d699 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22384640->22388735] (req 22384640->22388735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760711.687257:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac437f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512557824:12345-192.168.203.56@tcp:4:dd.0 Request processed in 69941us (70436us total) trans 4294972784 rc 0/0 00000100:00100000:2.0:1789760711.687268:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5495 00000100:00100000:2.0:1789760711.687280:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 691149875024 time 691181371871 state 0x2 ref 0 00000100:00100000:1.0:1789760711.704381:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760711.710299:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512557952 00000100:00100000:0.0:1789760711.710315:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512557952 00000100:00100000:0.0:1789760711.710377:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 691149875024 time 691181371871 state 0x0 ref 1 00000100:00100000:0.0:1789760711.710393:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 691181632287 time 691181371871 state 0x1 ref 1 00000100:00100000:0.0:1789760711.710429:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 691181632287 time 691274728749 state 0x0 ref 1 00000100:00100000:0.0:1789760711.710443:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac38b2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512557952:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760711.710493:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22388736, mtime: 0x6aad94c7, atime: 0x0, ctime: 0x6aad94c7, blocks: 0xb007 00010000:00010000:0.0:1789760711.710502:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2d6a0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760711.710524:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=691s v=(0 0) 00010000:00010000:0.0:1789760711.710529:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760711.710535:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2d6a0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22388736->22392831] (req 22388736->22392831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760711.710549:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2d6a0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22388736->22392831] (req 22388736->22392831) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760711.872683:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972785 is committed 00002000:00010000:3.0:1789760711.873632:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22388736 -> 22392832 00010000:00010000:3.0:1789760711.873660:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2d6a0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22388736->22392831] (req 22388736->22392831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.873673:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2d6a0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22388736->22392831] (req 22388736->22392831) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.873683:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2d6a0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22388736->22392831] (req 22388736->22392831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.873720:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760711.873726:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2d6a0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22388736->22392831] (req 22388736->22392831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.873736:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2d6a0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22388736->22392831] (req 22388736->22392831) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.873754:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2d6a0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22388736->22392831] (req 22388736->22392831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.873762:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2d6a0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22388736->22392831] (req 22388736->22392831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760711.873878:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac38b2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512557952:12345-192.168.203.56@tcp:4:dd.0 Request processed in 163432us (169500us total) trans 4294972785 rc 0/0 00000100:00100000:3.0:1789760711.873889:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5496 00000100:00100000:3.0:1789760711.873898:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 691181632287 time 691274728749 state 0x2 ref 0 00000100:00100000:0.0:1789760711.886278:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760711.886378:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512558080 00000100:00100000:1.0:1789760711.886390:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512558080 00000100:00100000:1.0:1789760711.886444:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 691181632287 time 691274728749 state 0x0 ref 1 00000100:00100000:1.0:1789760711.886460:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 691274989165 time 691274728749 state 0x1 ref 1 00000100:00100000:1.0:1789760711.886491:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 691274989165 time 691450790431 state 0x0 ref 1 00000100:00100000:1.0:1789760711.886504:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac437d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512558080:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760711.886565:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22392832, mtime: 0x6aad94c7, atime: 0x0, ctime: 0x6aad94c7, blocks: 0xb007 00010000:00010000:1.0:1789760711.886575:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2d6a7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.886590:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=691s v=(0 0) 00010000:00010000:1.0:1789760711.886604:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760711.886610:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2d6a7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22392832->22396927] (req 22392832->22396927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.886624:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2d6a7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22392832->22396927] (req 22392832->22396927) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760711.929089:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22392832 -> 22396928 00010000:00010000:1.0:1789760711.929107:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2d6a7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22392832->22396927] (req 22392832->22396927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.929120:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2d6a7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22392832->22396927] (req 22392832->22396927) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.929129:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2d6a7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22392832->22396927] (req 22392832->22396927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.929136:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760711.929141:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2d6a7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22392832->22396927] (req 22392832->22396927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.929151:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2d6a7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22392832->22396927] (req 22392832->22396927) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.929169:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2d6a7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22392832->22396927] (req 22392832->22396927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.929176:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2d6a7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22392832->22396927] (req 22392832->22396927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760711.929321:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972786 is committed 00000100:00100000:1.0:1789760711.929469:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac437d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512558080:12345-192.168.203.56@tcp:4:dd.0 Request processed in 42963us (43198us total) trans 4294972786 rc 0/0 00000100:00100000:1.0:1789760711.929484:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5497 00000100:00100000:1.0:1789760711.929494:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 691274989165 time 691450790431 state 0x2 ref 0 00000100:00100000:0.0:1789760711.932831:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760711.933047:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512558208 00000100:00100000:2.0:1789760711.933057:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512558208 00000100:00100000:2.0:1789760711.933105:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 691274989165 time 691450790431 state 0x0 ref 1 00000100:00100000:2.0:1789760711.933119:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 691451050847 time 691450790431 state 0x1 ref 1 00000100:00100000:2.0:1789760711.933145:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 691451050847 time 691497445685 state 0x0 ref 1 00000100:00100000:2.0:1789760711.933156:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febea4b8050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512558208:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760711.933199:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22396928, mtime: 0x6aad94c7, atime: 0x0, ctime: 0x6aad94c7, blocks: 0xb007 00010000:00010000:2.0:1789760711.933207:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2d6ae lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760711.933223:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=691s v=(0 0) 00010000:00010000:2.0:1789760711.933326:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760711.933332:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2d6ae lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22396928->22401023] (req 22396928->22401023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760711.933347:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2d6ae lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22396928->22401023] (req 22396928->22401023) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760711.985841:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22396928 -> 22401024 00010000:00010000:1.0:1789760711.985860:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2d6ae lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22396928->22401023] (req 22396928->22401023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.985874:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2d6ae lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22396928->22401023] (req 22396928->22401023) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.985885:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2d6ae lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22396928->22401023] (req 22396928->22401023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.985892:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760711.985898:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2d6ae lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22396928->22401023] (req 22396928->22401023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.985909:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2d6ae lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22396928->22401023] (req 22396928->22401023) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.985926:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2d6ae lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22396928->22401023] (req 22396928->22401023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.985935:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2d6ae lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22396928->22401023] (req 22396928->22401023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760711.986046:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febea4b8050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2359:x1876699512558208:12345-192.168.203.56@tcp:4:dd.0 Request processed in 52885us (53220us total) trans 4294972787 rc 0/0 00000100:00100000:1.0:1789760711.986056:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5498 00000100:00100000:1.0:1789760711.991516:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 691451050847 time 691497445685 state 0x2 ref 0 00000001:00080000:3.0:1789760711.992532:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972787 is committed 00000100:00100000:2.0:1789760711.992822:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760711.992917:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512558336 00000100:00100000:3.0:1789760711.992925:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512558336 00000100:00100000:3.0:1789760711.993049:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 691451050847 time 691497445685 state 0x0 ref 1 00000100:00100000:3.0:1789760711.993197:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 691497706101 time 691497445685 state 0x1 ref 1 00000100:00100000:3.0:1789760711.993324:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 691497706101 time 691557621684 state 0x0 ref 1 00000100:00100000:3.0:1789760711.993339:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac3352300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512558336:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760711.993387:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22401024, mtime: 0x6aad94c7, atime: 0x0, ctime: 0x6aad94c7, blocks: 0xb007 00010000:00010000:3.0:1789760711.993395:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2d6b5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.993410:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=691s v=(0 0) 00010000:00010000:3.0:1789760711.993414:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760711.993419:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2d6b5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22401024->22405119] (req 22401024->22405119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.993433:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2d6b5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22401024->22405119] (req 22401024->22405119) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760712.037742:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972788 is committed 00002000:00010000:3.0:1789760712.037842:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22401024 -> 22405120 00010000:00010000:3.0:1789760712.037858:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2d6b5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22401024->22405119] (req 22401024->22405119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.037871:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2d6b5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22401024->22405119] (req 22401024->22405119) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.037882:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2d6b5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22401024->22405119] (req 22401024->22405119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.037889:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760712.037897:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2d6b5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22401024->22405119] (req 22401024->22405119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.037907:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2d6b5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22401024->22405119] (req 22401024->22405119) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.037925:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2d6b5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22401024->22405119] (req 22401024->22405119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.037933:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2d6b5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22401024->22405119] (req 22401024->22405119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760712.038101:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac3352300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512558336:12345-192.168.203.56@tcp:4:dd.0 Request processed in 44762us (45457us total) trans 4294972788 rc 0/0 00000100:00100000:3.0:1789760712.038112:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5499 00000100:00100000:3.0:1789760712.038121:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 691497706101 time 691557621684 state 0x2 ref 0 00000100:00100000:2.0:1789760712.042579:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760712.042789:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512558464 00000100:00100000:1.0:1789760712.042798:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512558464 00000100:00100000:1.0:1789760712.042912:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 691497706101 time 691557621684 state 0x0 ref 1 00000100:00100000:1.0:1789760712.042927:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 691557882100 time 691557621684 state 0x1 ref 1 00000100:00100000:1.0:1789760712.043035:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 691557882100 time 691607333740 state 0x0 ref 1 00000100:00100000:1.0:1789760712.043048:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac3353800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512558464:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760712.043106:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22405120, mtime: 0x6aad94c7, atime: 0x0, ctime: 0x6aad94c7, blocks: 0xb007 00010000:00010000:1.0:1789760712.043116:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2d6bc lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.043133:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=691s v=(0 0) 00010000:00010000:1.0:1789760712.043138:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760712.043144:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2d6bc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22405120->22409215] (req 22405120->22409215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.043159:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2d6bc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22405120->22409215] (req 22405120->22409215) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760712.052868:0:5835:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:3.0:1789760712.130466:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972789 is committed 00002000:00010000:1.0:1789760712.130548:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22405120 -> 22409216 00010000:00010000:1.0:1789760712.130572:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2d6bc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22405120->22409215] (req 22405120->22409215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.130585:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2d6bc lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22405120->22409215] (req 22405120->22409215) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.130594:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2d6bc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22405120->22409215] (req 22405120->22409215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.130601:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760712.130607:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2d6bc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22405120->22409215] (req 22405120->22409215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.130615:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2d6bc lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22405120->22409215] (req 22405120->22409215) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.130634:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2d6bc lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22405120->22409215] (req 22405120->22409215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.130642:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2d6bc lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22405120->22409215] (req 22405120->22409215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760712.130850:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac3353800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512558464:12345-192.168.203.56@tcp:4:dd.0 Request processed in 87798us (88274us total) trans 4294972789 rc 0/0 00000100:00100000:1.0:1789760712.130864:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5500 00000100:00100000:1.0:1789760712.130874:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 691557882100 time 691607333740 state 0x2 ref 0 00000100:00100000:1.0:1789760712.133865:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760712.133946:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512558592 00000100:00100000:2.0:1789760712.133957:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512558592 00000100:00100000:2.0:1789760712.134021:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 691557882100 time 691607333740 state 0x0 ref 1 00000100:00100000:2.0:1789760712.134035:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 691607594156 time 691607333740 state 0x1 ref 1 00000100:00100000:2.0:1789760712.134057:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 691607594156 time 691698357274 state 0x0 ref 1 00000100:00100000:2.0:1789760712.134078:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac38b0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512558592:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760712.134135:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22409216, mtime: 0x6aad94c7, atime: 0x0, ctime: 0x6aad94c7, blocks: 0xb007 00010000:00010000:2.0:1789760712.134145:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2d6c3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.134161:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=691s v=(0 0) 00010000:00010000:2.0:1789760712.134167:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760712.134175:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2d6c3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22409216->22413311] (req 22409216->22413311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.134188:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2d6c3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22409216->22413311] (req 22409216->22413311) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760712.180090:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972790 is committed 00002000:00010000:3.0:1789760712.180244:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22409216 -> 22413312 00010000:00010000:3.0:1789760712.180260:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2d6c3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22409216->22413311] (req 22409216->22413311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.180274:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2d6c3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22409216->22413311] (req 22409216->22413311) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.180285:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2d6c3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22409216->22413311] (req 22409216->22413311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.180300:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760712.180305:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2d6c3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22409216->22413311] (req 22409216->22413311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.180314:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2d6c3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22409216->22413311] (req 22409216->22413311) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.180333:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2d6c3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22409216->22413311] (req 22409216->22413311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.180341:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2d6c3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22409216->22413311] (req 22409216->22413311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760712.180427:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac38b0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512558592:12345-192.168.203.56@tcp:4:dd.0 Request processed in 46347us (46568us total) trans 4294972790 rc 0/0 00000100:00100000:3.0:1789760712.180437:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5501 00000100:00100000:3.0:1789760712.180447:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 691607594156 time 691698357274 state 0x2 ref 0 00000100:00100000:2.0:1789760712.183665:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760712.186484:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512558720 00000100:00100000:3.0:1789760712.186499:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512558720 00000100:00100000:3.0:1789760712.186573:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 691607594156 time 691698357274 state 0x0 ref 1 00000100:00100000:3.0:1789760712.186589:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 691698617690 time 691698357274 state 0x1 ref 1 00000100:00100000:3.0:1789760712.186709:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 691698617690 time 691750995254 state 0x0 ref 1 00000100:00100000:3.0:1789760712.186726:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac3352a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512558720:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760712.186780:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22413312, mtime: 0x6aad94c7, atime: 0x0, ctime: 0x6aad94c7, blocks: 0xb007 00010000:00010000:3.0:1789760712.186790:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2d6ca lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.186806:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=691s v=(0 0) 00010000:00010000:3.0:1789760712.186812:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760712.186818:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2d6ca lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22413312->22417407] (req 22413312->22417407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.186832:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2d6ca lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22413312->22417407] (req 22413312->22417407) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760712.245671:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22413312 -> 22417408 00010000:00010000:0.0:1789760712.245691:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2d6ca lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22413312->22417407] (req 22413312->22417407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760712.245741:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2d6ca lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22413312->22417407] (req 22413312->22417407) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760712.245750:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2d6ca lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22413312->22417407] (req 22413312->22417407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760712.245758:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760712.245763:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2d6ca lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22413312->22417407] (req 22413312->22417407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760712.245777:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2d6ca lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22413312->22417407] (req 22413312->22417407) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760712.245796:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2d6ca lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22413312->22417407] (req 22413312->22417407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760712.245804:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2d6ca lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22413312->22417407] (req 22413312->22417407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760712.245898:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac3352a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2359:x1876699512558720:12345-192.168.203.56@tcp:4:dd.0 Request processed in 59170us (62237us total) trans 4294972791 rc 0/0 00000100:00100000:0.0:1789760712.245909:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5502 00000100:00100000:0.0:1789760712.245918:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 691698617690 time 691750995254 state 0x2 ref 0 00000001:00080000:1.0:1789760712.245994:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972791 is committed 00000100:00100000:2.0:1789760712.267028:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760712.268760:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512558848 00000100:00100000:1.0:1789760712.268775:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512558848 00000100:00100000:1.0:1789760712.268910:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 691698617690 time 691750995254 state 0x0 ref 1 00000100:00100000:1.0:1789760712.268928:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 691751255670 time 691750995254 state 0x1 ref 1 00000100:00100000:1.0:1789760712.268968:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 691751255670 time 691833267529 state 0x0 ref 1 00000100:00100000:1.0:1789760712.268981:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac3353480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512558848:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760712.269027:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22417408, mtime: 0x6aad94c7, atime: 0x0, ctime: 0x6aad94c7, blocks: 0xb007 00010000:00010000:1.0:1789760712.269036:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2d6d1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.269051:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=691s v=(0 0) 00010000:00010000:1.0:1789760712.269056:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760712.269062:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2d6d1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22417408->22421503] (req 22417408->22421503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.269076:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2d6d1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22417408->22421503] (req 22417408->22421503) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760712.340288:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972792 is committed 00002000:00010000:3.0:1789760712.340422:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22417408 -> 22421504 00010000:00010000:3.0:1789760712.340442:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2d6d1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22417408->22421503] (req 22417408->22421503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.340455:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2d6d1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22417408->22421503] (req 22417408->22421503) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.340465:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2d6d1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22417408->22421503] (req 22417408->22421503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.340472:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760712.340477:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2d6d1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22417408->22421503] (req 22417408->22421503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.340485:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2d6d1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22417408->22421503] (req 22417408->22421503) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.340505:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2d6d1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22417408->22421503] (req 22417408->22421503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.340512:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2d6d1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22417408->22421503] (req 22417408->22421503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760712.340622:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac3353480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512558848:12345-192.168.203.56@tcp:4:dd.0 Request processed in 71639us (73598us total) trans 4294972792 rc 0/0 00000100:00100000:3.0:1789760712.340633:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5503 00000100:00100000:3.0:1789760712.341887:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 691751255670 time 691833267529 state 0x2 ref 0 00000100:00100000:2.0:1789760712.347594:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760712.347802:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512558976 00000100:00100000:3.0:1789760712.347818:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512558976 00000100:00100000:2.0:1789760712.348312:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 691751255670 time 691833267529 state 0x0 ref 1 00000100:00100000:2.0:1789760712.348333:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 691833527945 time 691833267529 state 0x1 ref 1 00000100:00100000:2.0:1789760712.348383:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 691833527945 time 691912682452 state 0x0 ref 1 00000100:00100000:2.0:1789760712.348398:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febff933b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512558976:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760712.348466:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22421504, mtime: 0x6aad94c7, atime: 0x0, ctime: 0x6aad94c7, blocks: 0xb007 00010000:00010000:2.0:1789760712.348478:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2d6d8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.348497:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=691s v=(0 0) 00010000:00010000:2.0:1789760712.348503:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760712.348511:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2d6d8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22421504->22425599] (req 22421504->22425599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.348528:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2d6d8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22421504->22425599] (req 22421504->22425599) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760712.413129:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972793 is committed 00002000:00010000:3.0:1789760712.414372:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22421504 -> 22425600 00010000:00010000:3.0:1789760712.414394:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2d6d8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22421504->22425599] (req 22421504->22425599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.414407:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2d6d8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22421504->22425599] (req 22421504->22425599) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.414416:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2d6d8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22421504->22425599] (req 22421504->22425599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.414430:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760712.414435:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2d6d8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22421504->22425599] (req 22421504->22425599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.414444:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2d6d8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22421504->22425599] (req 22421504->22425599) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.414462:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2d6d8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22421504->22425599] (req 22421504->22425599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.414470:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2d6d8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22421504->22425599] (req 22421504->22425599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760712.415354:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febff933b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512558976:12345-192.168.203.56@tcp:4:dd.0 Request processed in 66949us (67766us total) trans 4294972793 rc 0/0 00000100:00100000:3.0:1789760712.415367:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5504 00000100:00100000:3.0:1789760712.415379:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 691833527945 time 691912682452 state 0x2 ref 0 00000100:00100000:2.0:1789760712.420324:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760712.420468:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512559104 00000100:00100000:0.0:1789760712.420480:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512559104 00000100:00100000:0.0:1789760712.420564:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 691833527945 time 691912682452 state 0x0 ref 1 00000100:00100000:0.0:1789760712.420584:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 691912942868 time 691912682452 state 0x1 ref 1 00000100:00100000:0.0:1789760712.420798:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 691912942868 time 691985095222 state 0x0 ref 1 00000100:00100000:0.0:1789760712.420814:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febff930380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512559104:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760712.421063:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22425600, mtime: 0x6aad94c7, atime: 0x0, ctime: 0x6aad94c7, blocks: 0xb007 00010000:00010000:0.0:1789760712.421075:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2d6df lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760712.421096:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=691s v=(0 0) 00010000:00010000:0.0:1789760712.421101:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760712.421108:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2d6df lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22425600->22429695] (req 22425600->22429695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760712.421123:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2d6df lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22425600->22429695] (req 22425600->22429695) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760712.532883:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972794 is committed 00002000:00010000:3.0:1789760712.533156:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22425600 -> 22429696 00010000:00010000:3.0:1789760712.533172:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2d6df lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22425600->22429695] (req 22425600->22429695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.533184:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2d6df lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22425600->22429695] (req 22425600->22429695) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.533198:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2d6df lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22425600->22429695] (req 22425600->22429695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.533205:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760712.533210:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2d6df lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22425600->22429695] (req 22425600->22429695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.533218:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2d6df lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22425600->22429695] (req 22425600->22429695) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.533251:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2d6df lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22425600->22429695] (req 22425600->22429695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.533259:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2d6df lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22425600->22429695] (req 22425600->22429695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760712.533361:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febff930380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512559104:12345-192.168.203.56@tcp:4:dd.0 Request processed in 112537us (113041us total) trans 4294972794 rc 0/0 00000100:00100000:3.0:1789760712.533372:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5505 00000100:00100000:3.0:1789760712.533382:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 691912942868 time 691985095222 state 0x2 ref 0 00000100:00100000:2.0:1789760712.548779:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760712.553729:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512559232 00000100:00100000:1.0:1789760712.553745:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512559232 00000100:00100000:1.0:1789760712.553806:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 691912942868 time 691985095222 state 0x0 ref 1 00000100:00100000:1.0:1789760712.554010:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 691985355638 time 691985095222 state 0x1 ref 1 00000100:00100000:1.0:1789760712.554101:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 691985355638 time 692118400099 state 0x0 ref 1 00000100:00100000:1.0:1789760712.554117:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febff932a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512559232:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760712.554166:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22429696, mtime: 0x6aad94c7, atime: 0x0, ctime: 0x6aad94c7, blocks: 0xb007 00010000:00010000:1.0:1789760712.554176:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2d6e6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.554191:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=692s v=(0 0) 00010000:00010000:1.0:1789760712.554196:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760712.554203:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2d6e6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22429696->22433791] (req 22429696->22433791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.554217:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2d6e6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22429696->22433791] (req 22429696->22433791) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760712.653786:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972795 is committed 00002000:00010000:3.0:1789760712.653895:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22429696 -> 22433792 00002000:00010000:3.0:1789760712.653900:0:6709:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760711 -> 1789760712 00002000:00010000:3.0:1789760712.653904:0:6709:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760711 -> 1789760712 00010000:00010000:3.0:1789760712.653916:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2d6e6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22429696->22433791] (req 22429696->22433791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.654059:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2d6e6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22429696->22433791] (req 22429696->22433791) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.654070:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2d6e6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22429696->22433791] (req 22429696->22433791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.654077:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760712.654082:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2d6e6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22429696->22433791] (req 22429696->22433791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.654091:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2d6e6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22429696->22433791] (req 22429696->22433791) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.654110:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2d6e6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22429696->22433791] (req 22429696->22433791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.654117:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2d6e6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22429696->22433791] (req 22429696->22433791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760712.654318:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febff932a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512559232:12345-192.168.203.56@tcp:4:dd.0 Request processed in 100194us (105538us total) trans 4294972795 rc 0/0 00000100:00100000:3.0:1789760712.654327:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5506 00000100:00100000:3.0:1789760712.654342:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 691985355638 time 692118400099 state 0x2 ref 0 00000100:00100000:1.0:1789760712.661157:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760712.661244:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512559360 00000100:00100000:3.0:1789760712.661254:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512559360 00000100:00100000:3.0:1789760712.661319:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 691985355638 time 692118400099 state 0x0 ref 1 00000100:00100000:3.0:1789760712.661336:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 692118660515 time 692118400099 state 0x1 ref 1 00000100:00100000:3.0:1789760712.661365:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 692118660515 time 692225664606 state 0x0 ref 1 00000100:00100000:3.0:1789760712.661378:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac38b3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512559360:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760712.661430:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22433792, mtime: 0x6aad94c8, atime: 0x0, ctime: 0x6aad94c8, blocks: 0xb007 00010000:00010000:3.0:1789760712.661440:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2d6ed lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.661455:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=692s v=(0 0) 00010000:00010000:3.0:1789760712.661460:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760712.661466:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2d6ed lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22433792->22437887] (req 22433792->22437887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.661479:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2d6ed lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22433792->22437887] (req 22433792->22437887) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760712.762455:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972796 is committed 00002000:00010000:2.0:1789760712.764025:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22433792 -> 22437888 00010000:00010000:2.0:1789760712.764044:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2d6ed lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22433792->22437887] (req 22433792->22437887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.764059:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2d6ed lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22433792->22437887] (req 22433792->22437887) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.764069:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2d6ed lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22433792->22437887] (req 22433792->22437887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.764078:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760712.764082:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2d6ed lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22433792->22437887] (req 22433792->22437887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.764091:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2d6ed lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22433792->22437887] (req 22433792->22437887) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.764111:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2d6ed lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22433792->22437887] (req 22433792->22437887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.764119:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2d6ed lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22433792->22437887] (req 22433792->22437887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760712.773213:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac38b3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512559360:12345-192.168.203.56@tcp:4:dd.0 Request processed in 111824us (112053us total) trans 4294972796 rc 0/0 00000100:00100000:2.0:1789760712.773229:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5507 00000100:00100000:2.0:1789760712.773240:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 692118660515 time 692225664606 state 0x2 ref 0 00000100:00100000:3.0:1789760712.793022:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760712.793116:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512559488 00000100:00100000:0.0:1789760712.793126:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512559488 00000100:00100000:0.0:1789760712.793184:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 692118660515 time 692225664606 state 0x0 ref 1 00000100:00100000:0.0:1789760712.793199:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 692225925022 time 692225664606 state 0x1 ref 1 00000100:00100000:0.0:1789760712.793234:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 692225925022 time 692357533674 state 0x0 ref 1 00000100:00100000:0.0:1789760712.793247:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2747480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512559488:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760712.793295:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22437888, mtime: 0x6aad94c8, atime: 0x0, ctime: 0x6aad94c8, blocks: 0xb007 00010000:00010000:0.0:1789760712.793305:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2d6f4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760712.793321:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=692s v=(0 0) 00010000:00010000:0.0:1789760712.793325:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760712.793341:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2d6f4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22437888->22441983] (req 22437888->22441983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760712.793354:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2d6f4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22437888->22441983] (req 22437888->22441983) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760712.887609:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972797 is committed 00002000:00010000:3.0:1789760712.892650:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22437888 -> 22441984 00010000:00010000:3.0:1789760712.892673:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2d6f4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22437888->22441983] (req 22437888->22441983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.892687:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2d6f4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22437888->22441983] (req 22437888->22441983) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.892712:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2d6f4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22437888->22441983] (req 22437888->22441983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.892720:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760712.892724:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2d6f4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22437888->22441983] (req 22437888->22441983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.892733:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2d6f4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22437888->22441983] (req 22437888->22441983) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.892754:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2d6f4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22437888->22441983] (req 22437888->22441983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.892762:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2d6f4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22437888->22441983] (req 22437888->22441983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760712.892877:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2747480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512559488:12345-192.168.203.56@tcp:4:dd.0 Request processed in 99627us (99860us total) trans 4294972797 rc 0/0 00000100:00100000:3.0:1789760712.892886:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5508 00000100:00100000:3.0:1789760712.892896:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 692225925022 time 692357533674 state 0x2 ref 0 00000100:00100000:1.0:1789760712.897731:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760712.897823:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512559616 00000100:00100000:3.0:1789760712.897832:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512559616 00000100:00100000:3.0:1789760712.897898:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 692225925022 time 692357533674 state 0x0 ref 1 00000100:00100000:3.0:1789760712.897913:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 692357794090 time 692357533674 state 0x1 ref 1 00000100:00100000:3.0:1789760712.897946:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 692357794090 time 692462246222 state 0x0 ref 1 00000100:00100000:3.0:1789760712.897960:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac38b2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512559616:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760712.898004:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22441984, mtime: 0x6aad94c8, atime: 0x0, ctime: 0x6aad94c8, blocks: 0xb007 00010000:00010000:3.0:1789760712.898013:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2d6fb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.898028:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=692s v=(0 0) 00010000:00010000:3.0:1789760712.898032:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760712.898038:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2d6fb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22441984->22446079] (req 22441984->22446079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.898052:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2d6fb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22441984->22446079] (req 22441984->22446079) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760712.936285:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972798 is committed 00002000:00010000:1.0:1789760712.939289:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22441984 -> 22446080 00010000:00010000:1.0:1789760712.939311:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2d6fb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22441984->22446079] (req 22441984->22446079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.939325:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2d6fb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22441984->22446079] (req 22441984->22446079) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.939334:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2d6fb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22441984->22446079] (req 22441984->22446079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.939341:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760712.939347:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2d6fb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22441984->22446079] (req 22441984->22446079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.939356:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2d6fb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22441984->22446079] (req 22441984->22446079) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.939377:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2d6fb lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22441984->22446079] (req 22441984->22446079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.939385:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2d6fb lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22441984->22446079] (req 22441984->22446079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760712.939484:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac38b2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512559616:12345-192.168.203.56@tcp:4:dd.0 Request processed in 41521us (41793us total) trans 4294972798 rc 0/0 00000100:00100000:1.0:1789760712.939497:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5509 00000100:00100000:1.0:1789760712.939507:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 692357794090 time 692462246222 state 0x2 ref 0 00000100:00100000:1.0:1789760712.942651:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760712.942747:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512559744 00000100:00100000:2.0:1789760712.942757:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512559744 00000100:00100000:2.0:1789760712.942810:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 692357794090 time 692462246222 state 0x0 ref 1 00000100:00100000:2.0:1789760712.942825:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 692462506638 time 692462246222 state 0x1 ref 1 00000100:00100000:2.0:1789760712.942854:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 692462506638 time 692507154012 state 0x0 ref 1 00000100:00100000:2.0:1789760712.942875:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac38b1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512559744:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760712.942921:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22446080, mtime: 0x6aad94c8, atime: 0x0, ctime: 0x6aad94c8, blocks: 0xb007 00010000:00010000:2.0:1789760712.942930:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2d702 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.942957:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=692s v=(0 0) 00010000:00010000:2.0:1789760712.942962:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760712.942969:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2d702 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22446080->22450175] (req 22446080->22450175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.942982:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2d702 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22446080->22450175] (req 22446080->22450175) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760712.997283:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972799 is committed 00002000:00010000:3.0:1789760712.997429:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22446080 -> 22450176 00010000:00010000:3.0:1789760712.997444:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2d702 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22446080->22450175] (req 22446080->22450175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.997457:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2d702 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22446080->22450175] (req 22446080->22450175) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.997466:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2d702 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22446080->22450175] (req 22446080->22450175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.997474:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760712.997479:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2d702 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22446080->22450175] (req 22446080->22450175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.997489:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2d702 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22446080->22450175] (req 22446080->22450175) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.997508:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2d702 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22446080->22450175] (req 22446080->22450175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.997515:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2d702 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22446080->22450175] (req 22446080->22450175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760712.997756:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac38b1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512559744:12345-192.168.203.56@tcp:4:dd.0 Request processed in 54878us (55115us total) trans 4294972799 rc 0/0 00000100:00100000:3.0:1789760712.997767:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5510 00000100:00100000:3.0:1789760712.997777:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 692462506638 time 692507154012 state 0x2 ref 0 00000100:00100000:1.0:1789760713.002076:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760713.002146:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512559872 00000100:00100000:3.0:1789760713.002155:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512559872 00000100:00100000:3.0:1789760713.002233:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 692462506638 time 692507154012 state 0x0 ref 1 00000100:00100000:3.0:1789760713.002248:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 692507414428 time 692507154012 state 0x1 ref 1 00000100:00100000:3.0:1789760713.002276:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 692507414428 time 692566576659 state 0x0 ref 1 00000100:00100000:3.0:1789760713.002290:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac38b1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512559872:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760713.002341:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22450176, mtime: 0x6aad94c8, atime: 0x0, ctime: 0x6aad94c8, blocks: 0xb007 00010000:00010000:3.0:1789760713.002351:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2d709 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.002366:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=692s v=(0 0) 00010000:00010000:3.0:1789760713.002370:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760713.002375:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2d709 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22450176->22454271] (req 22450176->22454271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.002389:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2d709 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22450176->22454271] (req 22450176->22454271) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:1.0:1789760713.003155:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:2.0:1789760713.051529:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972800 is committed 00002000:00010000:3.0:1789760713.051738:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22450176 -> 22454272 00010000:00010000:3.0:1789760713.051753:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2d709 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22450176->22454271] (req 22450176->22454271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.051765:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2d709 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22450176->22454271] (req 22450176->22454271) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.051774:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2d709 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22450176->22454271] (req 22450176->22454271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.051781:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760713.051786:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2d709 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22450176->22454271] (req 22450176->22454271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.051795:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2d709 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22450176->22454271] (req 22450176->22454271) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.051820:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2d709 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22450176->22454271] (req 22450176->22454271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.051829:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2d709 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22450176->22454271] (req 22450176->22454271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760713.051920:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac38b1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512559872:12345-192.168.203.56@tcp:4:dd.0 Request processed in 49629us (49850us total) trans 4294972800 rc 0/0 00000100:00100000:3.0:1789760713.051930:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5511 00000100:00100000:3.0:1789760713.053774:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 692507414428 time 692566576659 state 0x2 ref 0 00000100:00100000:1.0:1789760713.055872:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760713.056431:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512560000 00000100:00100000:2.0:1789760713.056442:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512560000 00000100:00100000:2.0:1789760713.056495:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 692507414428 time 692566576659 state 0x0 ref 1 00000100:00100000:2.0:1789760713.056512:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 692566837075 time 692566576659 state 0x1 ref 1 00000100:00100000:2.0:1789760713.056542:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 692566837075 time 692620842017 state 0x0 ref 1 00000100:00100000:2.0:1789760713.056565:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4058380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512560000:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760713.056614:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22454272, mtime: 0x6aad94c8, atime: 0x0, ctime: 0x6aad94c8, blocks: 0xb007 00010000:00010000:2.0:1789760713.056623:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2d710 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760713.056638:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=692s v=(0 0) 00010000:00010000:2.0:1789760713.056642:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760713.056649:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2d710 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22454272->22458367] (req 22454272->22458367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760713.056662:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2d710 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22454272->22458367] (req 22454272->22458367) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.077037:0:7647:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:0.0:1789760713.109777:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972801 is committed 00002000:00010000:3.0:1789760713.109909:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22454272 -> 22458368 00010000:00010000:3.0:1789760713.109927:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2d710 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22454272->22458367] (req 22454272->22458367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.110068:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2d710 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22454272->22458367] (req 22454272->22458367) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.110078:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2d710 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22454272->22458367] (req 22454272->22458367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.110085:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760713.110090:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2d710 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22454272->22458367] (req 22454272->22458367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.110103:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2d710 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22454272->22458367] (req 22454272->22458367) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.110123:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2d710 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22454272->22458367] (req 22454272->22458367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.110130:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2d710 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22454272->22458367] (req 22454272->22458367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760713.110321:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4058380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512560000:12345-192.168.203.56@tcp:4:dd.0 Request processed in 53752us (54459us total) trans 4294972801 rc 0/0 00000100:00100000:3.0:1789760713.110333:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5512 00000100:00100000:3.0:1789760713.110343:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 692566837075 time 692620842017 state 0x2 ref 0 00000100:00100000:1.0:1789760713.114839:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760713.115072:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512560128 00000100:00100000:3.0:1789760713.115082:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512560128 00000100:00100000:3.0:1789760713.115137:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 692566837075 time 692620842017 state 0x0 ref 1 00000100:00100000:3.0:1789760713.115153:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 692621102433 time 692620842017 state 0x1 ref 1 00000100:00100000:3.0:1789760713.115299:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 692621102433 time 692679597094 state 0x0 ref 1 00000100:00100000:3.0:1789760713.115313:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4059180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512560128:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760713.115732:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22458368, mtime: 0x6aad94c8, atime: 0x0, ctime: 0x6aad94c8, blocks: 0xb007 00010000:00010000:3.0:1789760713.115742:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2d717 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.115759:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=692s v=(0 0) 00010000:00010000:3.0:1789760713.115764:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760713.115771:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2d717 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22458368->22462463] (req 22458368->22462463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.115786:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2d717 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22458368->22462463] (req 22458368->22462463) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760713.159852:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22458368 -> 22462464 00010000:00010000:3.0:1789760713.159884:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2d717 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22458368->22462463] (req 22458368->22462463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.159897:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2d717 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22458368->22462463] (req 22458368->22462463) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.159906:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2d717 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22458368->22462463] (req 22458368->22462463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.159913:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760713.159919:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2d717 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22458368->22462463] (req 22458368->22462463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.159929:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2d717 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22458368->22462463] (req 22458368->22462463) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.159957:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2d717 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22458368->22462463] (req 22458368->22462463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.159966:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2d717 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22458368->22462463] (req 22458368->22462463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760713.160060:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4059180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2361:x1876699512560128:12345-192.168.203.56@tcp:4:dd.0 Request processed in 44744us (45226us total) trans 4294972802 rc 0/0 00000100:00100000:3.0:1789760713.160071:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5513 00000100:00100000:3.0:1789760713.160081:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 692621102433 time 692679597094 state 0x2 ref 0 00000001:00080000:0.0:1789760713.163260:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972802 is committed 00000100:00100000:0.0:1789760713.164247:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760713.164359:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512560256 00000100:00100000:3.0:1789760713.164369:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512560256 00000100:00100000:3.0:1789760713.164422:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 692621102433 time 692679597094 state 0x0 ref 1 00000100:00100000:3.0:1789760713.164438:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 692679857510 time 692679597094 state 0x1 ref 1 00000100:00100000:3.0:1789760713.164470:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 692679857510 time 692728769764 state 0x0 ref 1 00000100:00100000:3.0:1789760713.164483:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac437e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512560256:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760713.164529:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22462464, mtime: 0x6aad94c8, atime: 0x0, ctime: 0x6aad94c8, blocks: 0xb007 00010000:00010000:3.0:1789760713.164537:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2d71e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.164562:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=692s v=(0 0) 00010000:00010000:3.0:1789760713.164567:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760713.164572:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2d71e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22462464->22466559] (req 22462464->22466559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.164591:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2d71e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22462464->22466559] (req 22462464->22466559) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760713.212624:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22462464 -> 22466560 00010000:00010000:3.0:1789760713.212644:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2d71e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22462464->22466559] (req 22462464->22466559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.212658:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2d71e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22462464->22466559] (req 22462464->22466559) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.212667:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2d71e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22462464->22466559] (req 22462464->22466559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.212675:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760713.212680:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2d71e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22462464->22466559] (req 22462464->22466559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.212688:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2d71e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22462464->22466559] (req 22462464->22466559) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.212739:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2d71e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22462464->22466559] (req 22462464->22466559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.212747:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2d71e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22462464->22466559] (req 22462464->22466559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760713.212842:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac437e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2361:x1876699512560256:12345-192.168.203.56@tcp:4:dd.0 Request processed in 48355us (48593us total) trans 4294972803 rc 0/0 00000100:00100000:3.0:1789760713.212865:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5514 00000100:00100000:3.0:1789760713.212875:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 692679857510 time 692728769764 state 0x2 ref 0 00000001:00080000:1.0:1789760713.215400:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972803 is committed 00000100:00100000:2.0:1789760713.217210:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760713.218578:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512560384 00000100:00100000:3.0:1789760713.218591:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512560384 00000100:00100000:3.0:1789760713.218649:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 692679857510 time 692728769764 state 0x0 ref 1 00000100:00100000:3.0:1789760713.218665:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 692729030180 time 692728769764 state 0x1 ref 1 00000100:00100000:3.0:1789760713.218728:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 692729030180 time 692782996426 state 0x0 ref 1 00000100:00100000:3.0:1789760713.218741:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febff930e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512560384:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760713.218789:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22466560, mtime: 0x6aad94c8, atime: 0x0, ctime: 0x6aad94c8, blocks: 0xb007 00010000:00010000:3.0:1789760713.218798:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2d725 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.218813:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=692s v=(0 0) 00010000:00010000:3.0:1789760713.218818:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760713.218825:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2d725 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22466560->22470655] (req 22466560->22470655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.218838:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2d725 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22466560->22470655] (req 22466560->22470655) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760713.287487:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972804 is committed 00002000:00010000:0.0:1789760713.287689:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22466560 -> 22470656 00010000:00010000:0.0:1789760713.288199:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2d725 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22466560->22470655] (req 22466560->22470655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.288284:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2d725 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22466560->22470655] (req 22466560->22470655) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.288295:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2d725 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22466560->22470655] (req 22466560->22470655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.288302:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760713.288308:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2d725 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22466560->22470655] (req 22466560->22470655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.288317:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2d725 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22466560->22470655] (req 22466560->22470655) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.288344:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2d725 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22466560->22470655] (req 22466560->22470655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.288352:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2d725 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22466560->22470655] (req 22466560->22470655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760713.288570:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febff930e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512560384:12345-192.168.203.56@tcp:4:dd.0 Request processed in 69827us (71364us total) trans 4294972804 rc 0/0 00000100:00100000:2.0:1789760713.288584:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5515 00000100:00100000:2.0:1789760713.288594:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 692729030180 time 692782996426 state 0x2 ref 0 00000100:00100000:2.0:1789760713.292341:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760713.292440:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512560512 00000100:00100000:3.0:1789760713.292449:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512560512 00000100:00100000:3.0:1789760713.292505:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 692729030180 time 692782996426 state 0x0 ref 1 00000100:00100000:3.0:1789760713.292520:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 692783256842 time 692782996426 state 0x1 ref 1 00000100:00100000:3.0:1789760713.292687:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 692783256842 time 692856985162 state 0x0 ref 1 00000100:00100000:3.0:1789760713.292724:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febfc975180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512560512:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760713.292783:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22470656, mtime: 0x6aad94c8, atime: 0x0, ctime: 0x6aad94c8, blocks: 0xb007 00010000:00010000:3.0:1789760713.292793:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2d72c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.292808:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=692s v=(0 0) 00010000:00010000:3.0:1789760713.292813:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760713.292819:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2d72c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22470656->22474751] (req 22470656->22474751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.292833:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2d72c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22470656->22474751] (req 22470656->22474751) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760713.360599:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22470656 -> 22474752 00010000:00010000:1.0:1789760713.360620:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2d72c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22470656->22474751] (req 22470656->22474751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.360634:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2d72c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22470656->22474751] (req 22470656->22474751) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.360645:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2d72c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22470656->22474751] (req 22470656->22474751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.360652:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760713.360657:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2d72c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22470656->22474751] (req 22470656->22474751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.360666:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2d72c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22470656->22474751] (req 22470656->22474751) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.360690:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2d72c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22470656->22474751] (req 22470656->22474751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.360719:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2d72c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22470656->22474751] (req 22470656->22474751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760713.360827:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febfc975180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2361:x1876699512560512:12345-192.168.203.56@tcp:4:dd.0 Request processed in 68123us (68490us total) trans 4294972805 rc 0/0 00000100:00100000:1.0:1789760713.360840:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5516 00000100:00100000:1.0:1789760713.360849:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 692783256842 time 692856985162 state 0x2 ref 0 00000001:00080000:2.0:1789760713.363738:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972805 is committed 00000100:00100000:1.0:1789760713.365841:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760713.365984:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512560640 00000100:00100000:3.0:1789760713.365995:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512560640 00000100:00100000:3.0:1789760713.366063:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 692783256842 time 692856985162 state 0x0 ref 1 00000100:00100000:3.0:1789760713.366079:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 692857245578 time 692856985162 state 0x1 ref 1 00000100:00100000:3.0:1789760713.366113:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 692857245578 time 692930412497 state 0x0 ref 1 00000100:00100000:3.0:1789760713.366125:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4059500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512560640:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760713.366175:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22474752, mtime: 0x6aad94c8, atime: 0x0, ctime: 0x6aad94c8, blocks: 0xb007 00010000:00010000:3.0:1789760713.366184:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2d733 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.366199:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=692s v=(0 0) 00010000:00010000:3.0:1789760713.366204:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760713.366211:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2d733 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22474752->22478847] (req 22474752->22478847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.366235:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2d733 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22474752->22478847] (req 22474752->22478847) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760713.435424:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22474752 -> 22478848 00010000:00010000:3.0:1789760713.435444:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2d733 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22474752->22478847] (req 22474752->22478847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.435457:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2d733 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22474752->22478847] (req 22474752->22478847) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.435466:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2d733 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22474752->22478847] (req 22474752->22478847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.435473:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760713.435479:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2d733 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22474752->22478847] (req 22474752->22478847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.435488:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2d733 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22474752->22478847] (req 22474752->22478847) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.435508:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2d733 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22474752->22478847] (req 22474752->22478847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.435517:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2d733 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22474752->22478847] (req 22474752->22478847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760713.435617:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4059500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2360:x1876699512560640:12345-192.168.203.56@tcp:4:dd.0 Request processed in 69489us (69780us total) trans 4294972806 rc 0/0 00000100:00100000:3.0:1789760713.435628:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5517 00000100:00100000:3.0:1789760713.435637:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 692857245578 time 692930412497 state 0x2 ref 0 00000001:00080000:3.0:1789760713.435675:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972806 is committed 00000100:00100000:3.0:1789760713.440634:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760713.440911:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512560768 00000100:00100000:1.0:1789760713.440922:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512560768 00000100:00100000:1.0:1789760713.440996:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 692857245578 time 692930412497 state 0x0 ref 1 00000100:00100000:1.0:1789760713.441013:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 692930672913 time 692930412497 state 0x1 ref 1 00000100:00100000:1.0:1789760713.441055:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 692930672913 time 693005354663 state 0x0 ref 1 00000100:00100000:1.0:1789760713.441069:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2744700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512560768:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760713.441118:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22478848, mtime: 0x6aad94c8, atime: 0x0, ctime: 0x6aad94c8, blocks: 0xb007 00010000:00010000:1.0:1789760713.441127:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2d73a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.441148:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=693s v=(0 0) 00010000:00010000:1.0:1789760713.441152:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760713.441158:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2d73a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22478848->22482943] (req 22478848->22482943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.441174:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2d73a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22478848->22482943] (req 22478848->22482943) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760713.534278:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22478848 -> 22482944 00010000:00010000:1.0:1789760713.534299:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2d73a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22478848->22482943] (req 22478848->22482943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.534328:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2d73a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22478848->22482943] (req 22478848->22482943) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.534338:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2d73a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22478848->22482943] (req 22478848->22482943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.534346:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760713.534351:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2d73a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22478848->22482943] (req 22478848->22482943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.534361:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2d73a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22478848->22482943] (req 22478848->22482943) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.534380:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2d73a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22478848->22482943] (req 22478848->22482943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.534389:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2d73a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22478848->22482943] (req 22478848->22482943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760713.534685:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972807 is committed 00000100:00100000:1.0:1789760713.535232:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2744700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512560768:12345-192.168.203.56@tcp:4:dd.0 Request processed in 94159us (94602us total) trans 4294972807 rc 0/0 00000100:00100000:1.0:1789760713.535244:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5518 00000100:00100000:1.0:1789760713.535254:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 692930672913 time 693005354663 state 0x2 ref 0 00000100:00100000:3.0:1789760713.543856:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760713.543972:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512560896 00000100:00100000:1.0:1789760713.543982:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512560896 00000100:00100000:1.0:1789760713.544236:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 692930672913 time 693005354663 state 0x0 ref 1 00000100:00100000:1.0:1789760713.544255:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 693005615079 time 693005354663 state 0x1 ref 1 00000100:00100000:1.0:1789760713.544289:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 693005615079 time 693108588734 state 0x0 ref 1 00000100:00100000:1.0:1789760713.544302:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2746a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512560896:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760713.544353:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22482944, mtime: 0x6aad94c8, atime: 0x0, ctime: 0x6aad94c8, blocks: 0xb007 00010000:00010000:1.0:1789760713.544362:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2d741 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.544376:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=693s v=(0 0) 00010000:00010000:1.0:1789760713.544380:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760713.544386:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2d741 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22482944->22487039] (req 22482944->22487039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.544402:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2d741 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22482944->22487039] (req 22482944->22487039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760713.638651:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22482944 -> 22487040 00002000:00010000:0.0:1789760713.638660:0:13063:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760712 -> 1789760713 00002000:00010000:0.0:1789760713.638663:0:13063:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760712 -> 1789760713 00010000:00010000:0.0:1789760713.638676:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2d741 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22482944->22487039] (req 22482944->22487039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.638689:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2d741 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22482944->22487039] (req 22482944->22487039) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.638718:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2d741 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22482944->22487039] (req 22482944->22487039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.638726:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760713.638731:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2d741 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22482944->22487039] (req 22482944->22487039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.638741:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2d741 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22482944->22487039] (req 22482944->22487039) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.638766:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2d741 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22482944->22487039] (req 22482944->22487039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.638774:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2d741 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22482944->22487039] (req 22482944->22487039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760713.638855:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2746a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2360:x1876699512560896:12345-192.168.203.56@tcp:4:dd.0 Request processed in 94550us (95017us total) trans 4294972808 rc 0/0 00000100:00100000:0.0:1789760713.638865:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5519 00000100:00100000:0.0:1789760713.638874:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 693005615079 time 693108588734 state 0x2 ref 0 00000001:00080000:2.0:1789760713.639186:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972808 is committed 00000100:00100000:1.0:1789760713.641855:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760713.641930:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512561024 00000100:00100000:3.0:1789760713.641999:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512561024 00000100:00100000:3.0:1789760713.642055:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 693005615079 time 693108588734 state 0x0 ref 1 00000100:00100000:3.0:1789760713.642070:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 693108849150 time 693108588734 state 0x1 ref 1 00000100:00100000:3.0:1789760713.642135:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 693108849150 time 693206434843 state 0x0 ref 1 00000100:00100000:3.0:1789760713.642148:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4058a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512561024:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760713.642194:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22487040, mtime: 0x6aad94c9, atime: 0x0, ctime: 0x6aad94c9, blocks: 0xb007 00010000:00010000:3.0:1789760713.642203:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2d748 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.642229:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=693s v=(0 0) 00010000:00010000:3.0:1789760713.642234:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760713.642240:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2d748 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22487040->22491135] (req 22487040->22491135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.642253:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2d748 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22487040->22491135] (req 22487040->22491135) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760713.674385:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22487040 -> 22491136 00010000:00010000:3.0:1789760713.674404:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2d748 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22487040->22491135] (req 22487040->22491135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.674419:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2d748 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22487040->22491135] (req 22487040->22491135) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.674429:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2d748 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22487040->22491135] (req 22487040->22491135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.674437:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760713.674442:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2d748 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22487040->22491135] (req 22487040->22491135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.674453:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2d748 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22487040->22491135] (req 22487040->22491135) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.674470:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2d748 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22487040->22491135] (req 22487040->22491135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.674478:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2d748 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22487040->22491135] (req 22487040->22491135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760713.674640:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4058a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2361:x1876699512561024:12345-192.168.203.56@tcp:4:dd.0 Request processed in 32488us (32788us total) trans 4294972809 rc 0/0 00000100:00100000:3.0:1789760713.674650:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5520 00000100:00100000:3.0:1789760713.674659:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 693108849150 time 693206434843 state 0x2 ref 0 00000001:00080000:3.0:1789760713.676606:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972809 is committed 00000100:00100000:3.0:1789760713.677336:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760713.677727:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512561152 00000100:00100000:0.0:1789760713.677737:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512561152 00000100:00100000:0.0:1789760713.677786:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 693108849150 time 693206434843 state 0x0 ref 1 00000100:00100000:0.0:1789760713.677801:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 693206695259 time 693206434843 state 0x1 ref 1 00000100:00100000:0.0:1789760713.677828:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 693206695259 time 693242127610 state 0x0 ref 1 00000100:00100000:0.0:1789760713.677840:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2745880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512561152:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760713.677886:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22491136, mtime: 0x6aad94c9, atime: 0x0, ctime: 0x6aad94c9, blocks: 0xb007 00010000:00010000:0.0:1789760713.677896:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2d74f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.677911:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=693s v=(0 0) 00010000:00010000:0.0:1789760713.677917:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760713.677924:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2d74f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22491136->22495231] (req 22491136->22495231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.677948:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2d74f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22491136->22495231] (req 22491136->22495231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760713.733262:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22491136 -> 22495232 00010000:00010000:0.0:1789760713.733281:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2d74f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22491136->22495231] (req 22491136->22495231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.733292:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2d74f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22491136->22495231] (req 22491136->22495231) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.733302:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2d74f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22491136->22495231] (req 22491136->22495231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.733308:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760713.733313:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2d74f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22491136->22495231] (req 22491136->22495231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.733322:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2d74f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22491136->22495231] (req 22491136->22495231) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.733341:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2d74f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22491136->22495231] (req 22491136->22495231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.733348:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2d74f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22491136->22495231] (req 22491136->22495231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760713.733447:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2745880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2360:x1876699512561152:12345-192.168.203.56@tcp:4:dd.0 Request processed in 55604us (56110us total) trans 4294972810 rc 0/0 00000100:00100000:0.0:1789760713.733457:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5521 00000100:00100000:0.0:1789760713.733466:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 693206695259 time 693242127610 state 0x2 ref 0 00000100:00100000:0.0:1789760713.737243:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760713.737368:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512561280 00000100:00100000:3.0:1789760713.737378:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512561280 00000100:00100000:3.0:1789760713.737442:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 693206695259 time 693242127610 state 0x0 ref 1 00000100:00100000:3.0:1789760713.737458:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 693242388026 time 693242127610 state 0x1 ref 1 00000100:00100000:3.0:1789760713.737496:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 693242388026 time 693301791670 state 0x0 ref 1 00000100:00100000:3.0:1789760713.737510:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac437e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2361:x1876699512561280:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760713.737575:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22495232, mtime: 0x6aad94c9, atime: 0x0, ctime: 0x6aad94c9, blocks: 0xb007 00010000:00010000:3.0:1789760713.737585:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2d756 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.737601:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=693s v=(0 0) 00010000:00010000:3.0:1789760713.737606:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760713.737612:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2d756 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22495232->22499327] (req 22495232->22499327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.737628:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2d756 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22495232->22499327] (req 22495232->22499327) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760713.741307:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972810 is committed 00000001:00080000:1.0:1789760713.783520:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972811 is committed 00002000:00010000:3.0:1789760713.783629:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22495232 -> 22499328 00010000:00010000:3.0:1789760713.783646:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2d756 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22495232->22499327] (req 22495232->22499327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.783657:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2d756 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22495232->22499327] (req 22495232->22499327) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.783667:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2d756 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22495232->22499327] (req 22495232->22499327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.783675:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760713.783680:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2d756 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22495232->22499327] (req 22495232->22499327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.783689:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2d756 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22495232->22499327] (req 22495232->22499327) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.783728:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2d756 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22495232->22499327] (req 22495232->22499327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.783737:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2d756 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22495232->22499327] (req 22495232->22499327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760713.783834:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac437e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512561280:12345-192.168.203.56@tcp:4:dd.0 Request processed in 46322us (46595us total) trans 4294972811 rc 0/0 00000100:00100000:3.0:1789760713.783844:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5522 00000100:00100000:3.0:1789760713.783854:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 693242388026 time 693301791670 state 0x2 ref 0 00000100:00100000:3.0:1789760713.787151:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760713.790840:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512561408 00000100:00100000:0.0:1789760713.790854:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512561408 00000100:00100000:0.0:1789760713.790914:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 693242388026 time 693301791670 state 0x0 ref 1 00000100:00100000:0.0:1789760713.790930:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 693302052086 time 693301791670 state 0x1 ref 1 00000100:00100000:0.0:1789760713.790990:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 693302052086 time 693355289783 state 0x0 ref 1 00000100:00100000:0.0:1789760713.791004:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2746300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512561408:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760713.791052:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22499328, mtime: 0x6aad94c9, atime: 0x0, ctime: 0x6aad94c9, blocks: 0xb007 00010000:00010000:0.0:1789760713.791061:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2d75d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.791077:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=693s v=(0 0) 00010000:00010000:0.0:1789760713.791082:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760713.791088:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2d75d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22499328->22503423] (req 22499328->22503423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.791101:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2d75d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22499328->22503423] (req 22499328->22503423) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760713.857049:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22499328 -> 22503424 00010000:00010000:1.0:1789760713.857069:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2d75d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22499328->22503423] (req 22499328->22503423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.857081:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2d75d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22499328->22503423] (req 22499328->22503423) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.857091:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2d75d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22499328->22503423] (req 22499328->22503423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.857097:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760713.857102:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2d75d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22499328->22503423] (req 22499328->22503423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.857122:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2d75d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22499328->22503423] (req 22499328->22503423) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.857140:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2d75d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22499328->22503423] (req 22499328->22503423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.857147:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2d75d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22499328->22503423] (req 22499328->22503423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760713.857242:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2746300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2360:x1876699512561408:12345-192.168.203.56@tcp:4:dd.0 Request processed in 66236us (70094us total) trans 4294972812 rc 0/0 00000100:00100000:1.0:1789760713.857253:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5523 00000100:00100000:1.0:1789760713.857263:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 693302052086 time 693355289783 state 0x2 ref 0 00000001:00080000:0.0:1789760713.857899:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972812 is committed 00000100:00100000:0.0:1789760713.862999:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760713.864633:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512561536 00000100:00100000:1.0:1789760713.864647:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512561536 00000100:00100000:1.0:1789760713.864735:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 693302052086 time 693355289783 state 0x0 ref 1 00000100:00100000:1.0:1789760713.864749:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 693355550199 time 693355289783 state 0x1 ref 1 00000100:00100000:1.0:1789760713.865645:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 693355550199 time 693429937788 state 0x0 ref 1 00000100:00100000:1.0:1789760713.865667:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac437ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512561536:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760713.865742:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22503424, mtime: 0x6aad94c9, atime: 0x0, ctime: 0x6aad94c9, blocks: 0xb007 00010000:00010000:1.0:1789760713.865752:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2d764 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.865767:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=693s v=(0 0) 00010000:00010000:1.0:1789760713.865771:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760713.865777:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2d764 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22503424->22507519] (req 22503424->22507519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.865792:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2d764 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22503424->22507519] (req 22503424->22507519) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760713.914024:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22503424 -> 22507520 00010000:00010000:1.0:1789760713.914043:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2d764 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22503424->22507519] (req 22503424->22507519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.914056:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2d764 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22503424->22507519] (req 22503424->22507519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.914066:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2d764 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22503424->22507519] (req 22503424->22507519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.914073:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760713.914078:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2d764 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22503424->22507519] (req 22503424->22507519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.914088:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2d764 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22503424->22507519] (req 22503424->22507519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.914106:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2d764 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22503424->22507519] (req 22503424->22507519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.914123:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2d764 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22503424->22507519] (req 22503424->22507519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760713.914226:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac437ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2361:x1876699512561536:12345-192.168.203.56@tcp:4:dd.0 Request processed in 48561us (51231us total) trans 4294972813 rc 0/0 00000100:00100000:1.0:1789760713.914240:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5524 00000100:00100000:1.0:1789760713.914250:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 693355550199 time 693429937788 state 0x2 ref 0 00000001:00080000:0.0:1789760713.914966:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972813 is committed 00000100:00100000:2.0:1789760713.918253:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760713.918373:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512561664 00000100:00100000:3.0:1789760713.918383:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512561664 00000100:00100000:3.0:1789760713.918433:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 693355550199 time 693429937788 state 0x0 ref 1 00000100:00100000:3.0:1789760713.918448:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 693430198204 time 693429937788 state 0x1 ref 1 00000100:00100000:3.0:1789760713.918477:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 693430198204 time 693482776727 state 0x0 ref 1 00000100:00100000:3.0:1789760713.918490:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac3302680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512561664:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760713.918551:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22507520, mtime: 0x6aad94c9, atime: 0x0, ctime: 0x6aad94c9, blocks: 0xb007 00010000:00010000:3.0:1789760713.918561:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2d76b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.918582:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=693s v=(0 0) 00010000:00010000:3.0:1789760713.918587:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760713.918593:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2d76b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22507520->22511615] (req 22507520->22511615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.918607:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2d76b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22507520->22511615] (req 22507520->22511615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760713.956651:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972814 is committed 00002000:00010000:1.0:1789760713.956803:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22507520 -> 22511616 00010000:00010000:1.0:1789760713.956819:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2d76b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22507520->22511615] (req 22507520->22511615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.957172:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2d76b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22507520->22511615] (req 22507520->22511615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.957184:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2d76b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22507520->22511615] (req 22507520->22511615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.957192:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760713.957196:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2d76b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22507520->22511615] (req 22507520->22511615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.957205:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2d76b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22507520->22511615] (req 22507520->22511615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.957283:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2d76b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22507520->22511615] (req 22507520->22511615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.957292:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2d76b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22507520->22511615] (req 22507520->22511615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760713.957424:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac3302680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512561664:12345-192.168.203.56@tcp:4:dd.0 Request processed in 38932us (39175us total) trans 4294972814 rc 0/0 00000100:00100000:1.0:1789760713.957435:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5525 00000100:00100000:1.0:1789760713.957445:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 693430198204 time 693482776727 state 0x2 ref 0 00000100:00100000:2.0:1789760713.960889:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760713.961023:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512561792 00000100:00100000:1.0:1789760713.961033:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512561792 00000100:00100000:1.0:1789760713.961084:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 693430198204 time 693482776727 state 0x0 ref 1 00000100:00100000:1.0:1789760713.961099:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 693483037143 time 693482776727 state 0x1 ref 1 00000100:00100000:1.0:1789760713.961234:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 693483037143 time 693525531596 state 0x0 ref 1 00000100:00100000:1.0:1789760713.961247:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac3300e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512561792:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760713.961292:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22511616, mtime: 0x6aad94c9, atime: 0x0, ctime: 0x6aad94c9, blocks: 0xb007 00010000:00010000:1.0:1789760713.961302:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2d772 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.961318:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=693s v=(0 0) 00010000:00010000:1.0:1789760713.961322:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760713.961328:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2d772 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22511616->22515711] (req 22511616->22515711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.961343:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2d772 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22511616->22515711] (req 22511616->22515711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760713.990082:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760713.990118:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760713.990138:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760713.990164:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512562048 00000100:00100000:2.0:1789760713.990173:0:5847:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512562048 00000100:00100000:2.0:1789760713.990225:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512562176 00000100:00100000:2.0:1789760713.990227:0:5847:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512562176 00000100:00100000:2.0:1789760713.990248:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 444 00000100:00100000:2.0:1789760713.990257:0:5847:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac3302d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+8:2361:x1876699512562048:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:2.0:1789760713.990321:0:5847:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac3302d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+8:2361:x1876699512562048:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 65us (249us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760713.990329:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 444 00000100:00100000:2.0:1789760713.990343:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 445 00000100:00100000:2.0:1789760713.990348:0:5847:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac3303100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:d2c40150-e97b-40da-952b-cb9f18bf7e96+5:2361:x1876699512562176:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:2.0:1789760713.990368:0:5847:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac3303100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:d2c40150-e97b-40da-952b-cb9f18bf7e96+5:2361:x1876699512562176:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 20us (229us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760713.990377:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 445 00000100:00100000:2.0:1789760713.990392:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512561920 00000100:00100000:2.0:1789760713.990395:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512561920 00000100:00100000:2.0:1789760713.990414:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 386 00000100:00100000:2.0:1789760713.990420:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febc3ebc850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:e6cf7267-9b19-462a-a263-14115df15dab+13:2361:x1876699512561920:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:2.0:1789760713.990441:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febc3ebc850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:e6cf7267-9b19-462a-a263-14115df15dab+13:2361:x1876699512561920:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 21us (323us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760713.990447:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 386 00000100:00080000:2.0:1789760713.990469:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000001:00080000:2.0:1789760714.001121:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972815 is committed 00002000:00010000:1.0:1789760714.001355:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22511616 -> 22515712 00010000:00010000:1.0:1789760714.001371:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2d772 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22511616->22515711] (req 22511616->22515711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.001385:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2d772 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22511616->22515711] (req 22511616->22515711) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.001395:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2d772 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22511616->22515711] (req 22511616->22515711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.001402:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760714.001407:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2d772 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22511616->22515711] (req 22511616->22515711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.001418:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2d772 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22511616->22515711] (req 22511616->22515711) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.001436:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2d772 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22511616->22515711] (req 22511616->22515711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.001445:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2d772 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22511616->22515711] (req 22511616->22515711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760714.001606:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac3300e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512561792:12345-192.168.203.56@tcp:4:dd.0 Request processed in 40281us (40646us total) trans 4294972815 rc 0/0 00000100:00100000:1.0:1789760714.001621:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5526 00000100:00100000:1.0:1789760714.001631:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 693483037143 time 693525531596 state 0x2 ref 0 00000100:00100000:2.0:1789760714.004943:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760714.005516:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512562304 00000100:00100000:1.0:1789760714.005596:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512562304 00000100:00100000:1.0:1789760714.005656:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 693483037143 time 693525531596 state 0x0 ref 1 00000100:00100000:1.0:1789760714.005672:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 693525792012 time 693525531596 state 0x1 ref 1 00000100:00100000:2.0:1789760714.005881:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 693525792012 time 693570178154 state 0x0 ref 1 00000100:00100000:2.0:1789760714.005896:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac3301c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512562304:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760714.006008:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22515712, mtime: 0x6aad94c9, atime: 0x0, ctime: 0x6aad94c9, blocks: 0xb007 00010000:00010000:2.0:1789760714.006017:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2d779 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.006034:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=693s v=(0 0) 00010000:00010000:2.0:1789760714.006038:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760714.006045:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2d779 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22515712->22519807] (req 22515712->22519807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.006058:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2d779 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22515712->22519807] (req 22515712->22519807) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760714.060532:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972816 is committed 00002000:00010000:3.0:1789760714.060633:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22515712 -> 22519808 00010000:00010000:3.0:1789760714.060647:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2d779 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22515712->22519807] (req 22515712->22519807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.060660:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2d779 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22515712->22519807] (req 22515712->22519807) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.060671:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2d779 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22515712->22519807] (req 22515712->22519807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.060679:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760714.060684:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2d779 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22515712->22519807] (req 22515712->22519807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.060693:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2d779 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22515712->22519807] (req 22515712->22519807) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.060738:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2d779 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22515712->22519807] (req 22515712->22519807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.060747:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2d779 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22515712->22519807] (req 22515712->22519807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760714.060826:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac3301c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512562304:12345-192.168.203.56@tcp:4:dd.0 Request processed in 54929us (55898us total) trans 4294972816 rc 0/0 00000100:00100000:3.0:1789760714.060836:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5527 00000100:00100000:3.0:1789760714.060846:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 693525792012 time 693570178154 state 0x2 ref 0 00000100:00100000:2.0:1789760714.065158:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760714.065312:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512562432 00000100:00100000:1.0:1789760714.065322:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512562432 00000100:00100000:1.0:1789760714.065375:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 693525792012 time 693570178154 state 0x0 ref 1 00000100:00100000:1.0:1789760714.065389:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 693570438570 time 693570178154 state 0x1 ref 1 00000100:00100000:1.0:1789760714.065456:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 693570438570 time 693629754694 state 0x0 ref 1 00000100:00100000:1.0:1789760714.065469:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febfb374380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512562432:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760714.065515:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22519808, mtime: 0x6aad94c9, atime: 0x0, ctime: 0x6aad94c9, blocks: 0xb007 00010000:00010000:1.0:1789760714.065525:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2d780 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.065548:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=693s v=(0 0) 00010000:00010000:1.0:1789760714.065553:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760714.065560:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2d780 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22519808->22523903] (req 22519808->22523903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.065581:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2d780 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22519808->22523903] (req 22519808->22523903) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.099263:0:5836:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:3.0:1789760714.106094:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972817 is committed 00002000:00010000:1.0:1789760714.106170:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22519808 -> 22523904 00010000:00010000:1.0:1789760714.106186:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2d780 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22519808->22523903] (req 22519808->22523903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.106199:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2d780 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22519808->22523903] (req 22519808->22523903) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.106209:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2d780 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22519808->22523903] (req 22519808->22523903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.106678:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760714.106683:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2d780 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22519808->22523903] (req 22519808->22523903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.106692:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2d780 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22519808->22523903] (req 22519808->22523903) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.109158:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2d780 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22519808->22523903] (req 22519808->22523903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.109171:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2d780 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22519808->22523903] (req 22519808->22523903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760714.109288:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febfb374380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512562432:12345-192.168.203.56@tcp:4:dd.0 Request processed in 43817us (44137us total) trans 4294972817 rc 0/0 00000100:00100000:1.0:1789760714.109298:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5528 00000100:00100000:1.0:1789760714.109308:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 693570438570 time 693629754694 state 0x2 ref 0 00000100:00100000:0.0:1789760714.111980:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760714.112046:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512562560 00000100:00100000:1.0:1789760714.112055:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512562560 00000100:00100000:1.0:1789760714.112102:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 693570438570 time 693629754694 state 0x0 ref 1 00000100:00100000:1.0:1789760714.112117:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 693630015110 time 693629754694 state 0x1 ref 1 00000100:00100000:1.0:1789760714.112145:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 693630015110 time 693676444649 state 0x0 ref 1 00000100:00100000:1.0:1789760714.112156:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac437df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512562560:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760714.112203:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22523904, mtime: 0x6aad94c9, atime: 0x0, ctime: 0x6aad94c9, blocks: 0xb007 00010000:00010000:1.0:1789760714.112211:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2d787 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.112237:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=693s v=(0 0) 00010000:00010000:1.0:1789760714.112241:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760714.112248:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2d787 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22523904->22527999] (req 22523904->22527999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.112262:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2d787 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22523904->22527999] (req 22523904->22527999) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760714.166678:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972818 is committed 00002000:00010000:1.0:1789760714.166756:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22523904 -> 22528000 00010000:00010000:1.0:1789760714.166773:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2d787 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22523904->22527999] (req 22523904->22527999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.166786:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2d787 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22523904->22527999] (req 22523904->22527999) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.166799:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2d787 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22523904->22527999] (req 22523904->22527999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.166806:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760714.166811:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2d787 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22523904->22527999] (req 22523904->22527999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.166820:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2d787 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22523904->22527999] (req 22523904->22527999) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.166838:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2d787 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22523904->22527999] (req 22523904->22527999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.166846:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2d787 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22523904->22527999] (req 22523904->22527999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760714.167107:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac437df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512562560:12345-192.168.203.56@tcp:4:dd.0 Request processed in 54777us (54960us total) trans 4294972818 rc 0/0 00000100:00100000:1.0:1789760714.167122:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5529 00000100:00100000:1.0:1789760714.167131:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 693630015110 time 693676444649 state 0x2 ref 0 00000100:00100000:0.0:1789760714.174614:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760714.174734:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512562688 00000100:00100000:3.0:1789760714.174744:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512562688 00000100:00100000:3.0:1789760714.174798:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 693630015110 time 693676444649 state 0x0 ref 1 00000100:00100000:3.0:1789760714.174812:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 693676705065 time 693676444649 state 0x1 ref 1 00000100:00100000:3.0:1789760714.174831:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 693676705065 time 693739132169 state 0x0 ref 1 00000100:00100000:3.0:1789760714.174843:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac437fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512562688:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760714.174888:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22528000, mtime: 0x6aad94c9, atime: 0x0, ctime: 0x6aad94c9, blocks: 0xb007 00010000:00010000:3.0:1789760714.174896:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2d78e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.174911:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=693s v=(0 0) 00010000:00010000:3.0:1789760714.174916:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760714.174922:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2d78e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22528000->22532095] (req 22528000->22532095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.174935:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2d78e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22528000->22532095] (req 22528000->22532095) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760714.246022:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972819 is committed 00002000:00010000:1.0:1789760714.249671:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22528000 -> 22532096 00010000:00010000:1.0:1789760714.249691:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2d78e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22528000->22532095] (req 22528000->22532095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.249718:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2d78e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22528000->22532095] (req 22528000->22532095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.249728:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2d78e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22528000->22532095] (req 22528000->22532095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.249737:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760714.249742:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2d78e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22528000->22532095] (req 22528000->22532095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.249751:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2d78e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22528000->22532095] (req 22528000->22532095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.249771:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2d78e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22528000->22532095] (req 22528000->22532095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.249778:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2d78e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22528000->22532095] (req 22528000->22532095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760714.249888:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac437fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512562688:12345-192.168.203.56@tcp:4:dd.0 Request processed in 75041us (75277us total) trans 4294972819 rc 0/0 00000100:00100000:1.0:1789760714.249899:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5530 00000100:00100000:1.0:1789760714.249909:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 693676705065 time 693739132169 state 0x2 ref 0 00000100:00100000:2.0:1789760714.254621:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760714.254805:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512562816 00000100:00100000:1.0:1789760714.254815:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512562816 00000100:00100000:2.0:1789760714.255344:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 693676705065 time 693739132169 state 0x0 ref 1 00000100:00100000:2.0:1789760714.255358:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 693739392585 time 693739132169 state 0x1 ref 1 00000100:00100000:2.0:1789760714.255427:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 693739392585 time 693819726166 state 0x0 ref 1 00000100:00100000:2.0:1789760714.255439:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4074700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512562816:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760714.255494:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22532096, mtime: 0x6aad94c9, atime: 0x0, ctime: 0x6aad94c9, blocks: 0xb007 00010000:00010000:2.0:1789760714.255503:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2d795 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.255516:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=693s v=(0 0) 00010000:00010000:2.0:1789760714.255520:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760714.255526:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2d795 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22532096->22536191] (req 22532096->22536191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.255548:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2d795 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22532096->22536191] (req 22532096->22536191) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760714.329552:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972820 is committed 00002000:00010000:3.0:1789760714.329654:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22532096 -> 22536192 00010000:00010000:3.0:1789760714.329668:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2d795 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22532096->22536191] (req 22532096->22536191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.329681:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2d795 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22532096->22536191] (req 22532096->22536191) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.329691:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2d795 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22532096->22536191] (req 22532096->22536191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.329721:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760714.329726:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2d795 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22532096->22536191] (req 22532096->22536191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.329736:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2d795 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22532096->22536191] (req 22532096->22536191) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.329753:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2d795 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22532096->22536191] (req 22532096->22536191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.329761:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2d795 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22532096->22536191] (req 22532096->22536191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760714.329857:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4074700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512562816:12345-192.168.203.56@tcp:4:dd.0 Request processed in 74415us (75241us total) trans 4294972820 rc 0/0 00000100:00100000:3.0:1789760714.329867:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5531 00000100:00100000:3.0:1789760714.329876:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 693739392585 time 693819726166 state 0x2 ref 0 00000100:00100000:0.0:1789760714.336103:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760714.339816:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512562944 00000100:00100000:1.0:1789760714.339830:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512562944 00000100:00100000:1.0:1789760714.339891:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 693739392585 time 693819726166 state 0x0 ref 1 00000100:00100000:1.0:1789760714.339906:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 693819986582 time 693819726166 state 0x1 ref 1 00000100:00100000:1.0:1789760714.339938:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 693819986582 time 693904227446 state 0x0 ref 1 00000100:00100000:1.0:1789760714.339951:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac437f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512562944:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760714.339995:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22536192, mtime: 0x6aad94c9, atime: 0x0, ctime: 0x6aad94c9, blocks: 0xb007 00010000:00010000:1.0:1789760714.340003:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2d79c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.340312:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=693s v=(0 0) 00010000:00010000:1.0:1789760714.340316:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760714.340322:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2d79c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22536192->22540287] (req 22536192->22540287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.340341:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2d79c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22536192->22540287] (req 22536192->22540287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760714.394909:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972821 is committed 00002000:00010000:1.0:1789760714.395332:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22536192 -> 22540288 00010000:00010000:1.0:1789760714.395346:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2d79c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22536192->22540287] (req 22536192->22540287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.395357:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2d79c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22536192->22540287] (req 22536192->22540287) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.395365:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2d79c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22536192->22540287] (req 22536192->22540287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.395372:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760714.395376:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2d79c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22536192->22540287] (req 22536192->22540287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.395386:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2d79c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22536192->22540287] (req 22536192->22540287) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.395405:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2d79c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22536192->22540287] (req 22536192->22540287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.395413:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2d79c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22536192->22540287] (req 22536192->22540287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760714.395504:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac437f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512562944:12345-192.168.203.56@tcp:4:dd.0 Request processed in 55551us (59405us total) trans 4294972821 rc 0/0 00000100:00100000:1.0:1789760714.395514:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5532 00000100:00100000:1.0:1789760714.395522:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 693819986582 time 693904227446 state 0x2 ref 0 00000100:00100000:2.0:1789760714.400873:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760714.403971:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512563072 00000100:00100000:1.0:1789760714.403984:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512563072 00000100:00100000:2.0:1789760714.404206:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 693819986582 time 693904227446 state 0x0 ref 1 00000100:00100000:2.0:1789760714.404283:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 693904487862 time 693904227446 state 0x1 ref 1 00000100:00100000:2.0:1789760714.404360:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 693904487862 time 693968659393 state 0x0 ref 1 00000100:00100000:2.0:1789760714.404374:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4074a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512563072:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760714.404425:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22540288, mtime: 0x6aad94c9, atime: 0x0, ctime: 0x6aad94c9, blocks: 0xb007 00010000:00010000:2.0:1789760714.404434:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2d7a3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.404449:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=693s v=(0 0) 00010000:00010000:2.0:1789760714.404454:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760714.404460:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2d7a3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22540288->22544383] (req 22540288->22544383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.404473:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2d7a3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22540288->22544383] (req 22540288->22544383) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760714.466526:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22540288 -> 22544384 00010000:00010000:3.0:1789760714.466552:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2d7a3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22540288->22544383] (req 22540288->22544383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.466566:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2d7a3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22540288->22544383] (req 22540288->22544383) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.466575:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2d7a3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22540288->22544383] (req 22540288->22544383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.466581:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760714.466586:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2d7a3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22540288->22544383] (req 22540288->22544383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.466596:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2d7a3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22540288->22544383] (req 22540288->22544383) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.466617:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2d7a3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22540288->22544383] (req 22540288->22544383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.466630:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2d7a3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22540288->22544383] (req 22540288->22544383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760714.466744:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4074a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2360:x1876699512563072:12345-192.168.203.56@tcp:4:dd.0 Request processed in 62368us (65874us total) trans 4294972822 rc 0/0 00000100:00100000:3.0:1789760714.466753:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5533 00000100:00100000:3.0:1789760714.466763:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 693904487862 time 693968659393 state 0x2 ref 0 00000001:00080000:3.0:1789760714.466798:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972822 is committed 00000100:00100000:2.0:1789760714.470581:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760714.470666:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512563200 00000100:00100000:3.0:1789760714.470676:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512563200 00000100:00100000:3.0:1789760714.470771:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 693904487862 time 693968659393 state 0x0 ref 1 00000100:00100000:3.0:1789760714.470787:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 693968919809 time 693968659393 state 0x1 ref 1 00000100:00100000:3.0:1789760714.470807:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 693968919809 time 694035107038 state 0x0 ref 1 00000100:00100000:3.0:1789760714.470820:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5307100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512563200:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760714.470869:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22544384, mtime: 0x6aad94c9, atime: 0x0, ctime: 0x6aad94c9, blocks: 0xb007 00010000:00010000:3.0:1789760714.470878:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2d7aa lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.470893:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=694s v=(0 0) 00010000:00010000:3.0:1789760714.470897:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760714.470903:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2d7aa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22544384->22548479] (req 22544384->22548479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.470915:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2d7aa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22544384->22548479] (req 22544384->22548479) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760714.512127:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972823 is committed 00002000:00010000:3.0:1789760714.512637:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22544384 -> 22548480 00010000:00010000:3.0:1789760714.512651:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2d7aa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22544384->22548479] (req 22544384->22548479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.512661:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2d7aa lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22544384->22548479] (req 22544384->22548479) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.512669:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2d7aa lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22544384->22548479] (req 22544384->22548479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.512676:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760714.512681:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2d7aa lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22544384->22548479] (req 22544384->22548479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.512688:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2d7aa lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22544384->22548479] (req 22544384->22548479) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.512722:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2d7aa lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22544384->22548479] (req 22544384->22548479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.512730:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2d7aa lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22544384->22548479] (req 22544384->22548479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760714.512826:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5307100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512563200:12345-192.168.203.56@tcp:4:dd.0 Request processed in 42005us (42251us total) trans 4294972823 rc 0/0 00000100:00100000:3.0:1789760714.512836:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5534 00000100:00100000:3.0:1789760714.512845:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 693968919809 time 694035107038 state 0x2 ref 0 00000100:00100000:0.0:1789760714.515140:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760714.515359:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512563328 00000100:00100000:1.0:1789760714.515370:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512563328 00000100:00100000:1.0:1789760714.515423:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 693968919809 time 694035107038 state 0x0 ref 1 00000100:00100000:1.0:1789760714.515439:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 694035367454 time 694035107038 state 0x1 ref 1 00000100:00100000:1.0:1789760714.515469:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 694035367454 time 694079769396 state 0x0 ref 1 00000100:00100000:1.0:1789760714.515482:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac437d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512563328:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760714.515525:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22548480, mtime: 0x6aad94c9, atime: 0x0, ctime: 0x6aad94c9, blocks: 0xb007 00010000:00010000:1.0:1789760714.515534:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2d7b1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.515557:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=694s v=(0 0) 00010000:00010000:1.0:1789760714.515562:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760714.515568:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2d7b1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22548480->22552575] (req 22548480->22552575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.515581:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2d7b1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22548480->22552575] (req 22548480->22552575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760714.562734:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972824 is committed 00002000:00010000:3.0:1789760714.562908:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22548480 -> 22552576 00002000:00010000:3.0:1789760714.562913:0:6710:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760713 -> 1789760714 00002000:00010000:3.0:1789760714.562916:0:6710:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760713 -> 1789760714 00010000:00010000:3.0:1789760714.562940:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2d7b1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22548480->22552575] (req 22548480->22552575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.562952:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2d7b1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22548480->22552575] (req 22548480->22552575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.562961:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2d7b1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22548480->22552575] (req 22548480->22552575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.562969:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760714.562973:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2d7b1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22548480->22552575] (req 22548480->22552575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.562983:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2d7b1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22548480->22552575] (req 22548480->22552575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.563002:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2d7b1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22548480->22552575] (req 22548480->22552575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.563011:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2d7b1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22548480->22552575] (req 22548480->22552575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760714.563107:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac437d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512563328:12345-192.168.203.56@tcp:4:dd.0 Request processed in 47623us (47970us total) trans 4294972824 rc 0/0 00000100:00100000:3.0:1789760714.563117:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5535 00000100:00100000:3.0:1789760714.563127:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 694035367454 time 694079769396 state 0x2 ref 0 00000100:00100000:2.0:1789760714.568078:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760714.568387:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512563456 00000100:00100000:3.0:1789760714.568397:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512563456 00000100:00100000:3.0:1789760714.568466:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 694035367454 time 694079769396 state 0x0 ref 1 00000100:00100000:3.0:1789760714.568482:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 694080029812 time 694079769396 state 0x1 ref 1 00000100:00100000:3.0:1789760714.568516:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 694080029812 time 694132816004 state 0x0 ref 1 00000100:00100000:3.0:1789760714.568530:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5305f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512563456:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760714.568593:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22552576, mtime: 0x6aad94ca, atime: 0x0, ctime: 0x6aad94ca, blocks: 0xb007 00010000:00010000:3.0:1789760714.568602:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2d7b8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.568619:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=694s v=(0 0) 00010000:00010000:3.0:1789760714.568624:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760714.568630:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2d7b8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22552576->22556671] (req 22552576->22556671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.568649:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2d7b8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22552576->22556671] (req 22552576->22556671) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760714.623213:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22552576 -> 22556672 00010000:00010000:3.0:1789760714.623235:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2d7b8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22552576->22556671] (req 22552576->22556671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.623249:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2d7b8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22552576->22556671] (req 22552576->22556671) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.623258:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2d7b8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22552576->22556671] (req 22552576->22556671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.623264:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760714.623270:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2d7b8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22552576->22556671] (req 22552576->22556671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.623278:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2d7b8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22552576->22556671] (req 22552576->22556671) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.623297:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2d7b8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22552576->22556671] (req 22552576->22556671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.623304:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2d7b8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22552576->22556671] (req 22552576->22556671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760714.623410:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5305f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2359:x1876699512563456:12345-192.168.203.56@tcp:4:dd.0 Request processed in 54878us (55346us total) trans 4294972825 rc 0/0 00000100:00100000:3.0:1789760714.623421:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5536 00000100:00100000:3.0:1789760714.623431:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 694080029812 time 694132816004 state 0x2 ref 0 00000001:00080000:0.0:1789760714.624232:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972825 is committed 00000100:00100000:2.0:1789760714.628755:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760714.633951:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512563584 00000100:00100000:3.0:1789760714.633967:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512563584 00000100:00100000:3.0:1789760714.634033:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 694080029812 time 694132816004 state 0x0 ref 1 00000100:00100000:3.0:1789760714.634049:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 694133076420 time 694132816004 state 0x1 ref 1 00000100:00100000:3.0:1789760714.634081:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 694133076420 time 694198381078 state 0x0 ref 1 00000100:00100000:3.0:1789760714.634095:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5307480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512563584:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760714.634140:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22556672, mtime: 0x6aad94ca, atime: 0x0, ctime: 0x6aad94ca, blocks: 0xb007 00010000:00010000:3.0:1789760714.634149:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2d7bf lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.634165:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=694s v=(0 0) 00010000:00010000:3.0:1789760714.634170:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760714.634176:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2d7bf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22556672->22560767] (req 22556672->22560767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.634190:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2d7bf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22556672->22560767] (req 22556672->22560767) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760714.679939:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972826 is committed 00002000:00010000:2.0:1789760714.681800:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22556672 -> 22560768 00010000:00010000:2.0:1789760714.681818:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2d7bf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22556672->22560767] (req 22556672->22560767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.681828:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2d7bf lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22556672->22560767] (req 22556672->22560767) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.681838:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2d7bf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22556672->22560767] (req 22556672->22560767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.681844:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760714.681849:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2d7bf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22556672->22560767] (req 22556672->22560767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.681858:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2d7bf lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22556672->22560767] (req 22556672->22560767) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.681875:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2d7bf lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22556672->22560767] (req 22556672->22560767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.681882:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2d7bf lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22556672->22560767] (req 22556672->22560767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760714.681962:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5307480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512563584:12345-192.168.203.56@tcp:4:dd.0 Request processed in 47866us (53211us total) trans 4294972826 rc 0/0 00000100:00100000:2.0:1789760714.681973:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5537 00000100:00100000:2.0:1789760714.681982:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 694133076420 time 694198381078 state 0x2 ref 0 00000100:00100000:1.0:1789760714.691111:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760714.691212:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512563712 00000100:00100000:3.0:1789760714.691222:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512563712 00000100:00100000:3.0:1789760714.691277:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 694133076420 time 694198381078 state 0x0 ref 1 00000100:00100000:3.0:1789760714.691293:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 694198641494 time 694198381078 state 0x1 ref 1 00000100:00100000:3.0:1789760714.691325:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 694198641494 time 694255624332 state 0x0 ref 1 00000100:00100000:3.0:1789760714.691337:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac405ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512563712:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760714.691384:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22560768, mtime: 0x6aad94ca, atime: 0x0, ctime: 0x6aad94ca, blocks: 0xb007 00010000:00010000:3.0:1789760714.691393:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2d7c6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.691409:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=694s v=(0 0) 00010000:00010000:3.0:1789760714.691414:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760714.691420:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2d7c6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22560768->22564863] (req 22560768->22564863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.691433:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2d7c6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22560768->22564863] (req 22560768->22564863) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760714.741422:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972827 is committed 00002000:00010000:3.0:1789760714.741648:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22560768 -> 22564864 00010000:00010000:3.0:1789760714.741663:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2d7c6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22560768->22564863] (req 22560768->22564863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.741681:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2d7c6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22560768->22564863] (req 22560768->22564863) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.741689:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2d7c6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22560768->22564863] (req 22560768->22564863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.741728:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760714.741734:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2d7c6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22560768->22564863] (req 22560768->22564863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.741743:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2d7c6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22560768->22564863] (req 22560768->22564863) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.741764:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2d7c6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22560768->22564863] (req 22560768->22564863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.741772:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2d7c6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22560768->22564863] (req 22560768->22564863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760714.741901:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac405ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512563712:12345-192.168.203.56@tcp:4:dd.0 Request processed in 50560us (50794us total) trans 4294972827 rc 0/0 00000100:00100000:3.0:1789760714.741910:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5538 00000100:00100000:3.0:1789760714.741918:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 694198641494 time 694255624332 state 0x2 ref 0 00000100:00100000:2.0:1789760714.749827:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760714.750040:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512563840 00000100:00100000:3.0:1789760714.750054:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512563840 00000100:00100000:3.0:1789760714.750142:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 694198641494 time 694255624332 state 0x0 ref 1 00000100:00100000:3.0:1789760714.750162:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 694255884748 time 694255624332 state 0x1 ref 1 00000100:00100000:3.0:1789760714.750199:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 694255884748 time 694314498095 state 0x0 ref 1 00000100:00100000:3.0:1789760714.750222:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5304700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512563840:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760714.750292:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22564864, mtime: 0x6aad94ca, atime: 0x0, ctime: 0x6aad94ca, blocks: 0xb007 00010000:00010000:3.0:1789760714.750304:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2d7cd lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.750322:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=694s v=(0 0) 00010000:00010000:3.0:1789760714.750329:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760714.750336:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2d7cd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22564864->22568959] (req 22564864->22568959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.750354:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2d7cd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22564864->22568959] (req 22564864->22568959) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760714.805978:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972828 is committed 00002000:00010000:3.0:1789760714.806407:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22564864 -> 22568960 00010000:00010000:3.0:1789760714.806422:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2d7cd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22564864->22568959] (req 22564864->22568959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.806434:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2d7cd lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22564864->22568959] (req 22564864->22568959) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.806449:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2d7cd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22564864->22568959] (req 22564864->22568959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.806456:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760714.806460:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2d7cd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22564864->22568959] (req 22564864->22568959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.806468:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2d7cd lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22564864->22568959] (req 22564864->22568959) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.806484:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2d7cd lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22564864->22568959] (req 22564864->22568959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.806490:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2d7cd lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22564864->22568959] (req 22564864->22568959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760714.806685:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5304700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512563840:12345-192.168.203.56@tcp:4:dd.0 Request processed in 56468us (56866us total) trans 4294972828 rc 0/0 00000100:00100000:3.0:1789760714.806731:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5539 00000100:00100000:3.0:1789760714.806740:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 694255884748 time 694314498095 state 0x2 ref 0 00000100:00100000:3.0:1789760714.818206:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760714.818322:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512563968 00000100:00100000:2.0:1789760714.818332:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512563968 00000100:00100000:2.0:1789760714.818383:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 694255884748 time 694314498095 state 0x0 ref 1 00000100:00100000:2.0:1789760714.818402:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 694314758511 time 694314498095 state 0x1 ref 1 00000100:00100000:2.0:1789760714.818437:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 694314758511 time 694382736308 state 0x0 ref 1 00000100:00100000:2.0:1789760714.818449:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2745180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512563968:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760714.818495:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22568960, mtime: 0x6aad94ca, atime: 0x0, ctime: 0x6aad94ca, blocks: 0xb007 00010000:00010000:2.0:1789760714.818503:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2d7d4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.818518:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=694s v=(0 0) 00010000:00010000:2.0:1789760714.818522:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760714.818528:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2d7d4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22568960->22573055] (req 22568960->22573055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.818550:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2d7d4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22568960->22573055] (req 22568960->22573055) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760714.870142:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972829 is committed 00002000:00010000:1.0:1789760714.870285:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22568960 -> 22573056 00010000:00010000:1.0:1789760714.870299:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2d7d4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22568960->22573055] (req 22568960->22573055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.870312:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2d7d4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22568960->22573055] (req 22568960->22573055) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.870321:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2d7d4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22568960->22573055] (req 22568960->22573055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.870328:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760714.870332:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2d7d4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22568960->22573055] (req 22568960->22573055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.870341:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2d7d4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22568960->22573055] (req 22568960->22573055) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.870360:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2d7d4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22568960->22573055] (req 22568960->22573055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.870368:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2d7d4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22568960->22573055] (req 22568960->22573055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760714.870463:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2745180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512563968:12345-192.168.203.56@tcp:4:dd.0 Request processed in 52010us (52260us total) trans 4294972829 rc 0/0 00000100:00100000:1.0:1789760714.870472:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5540 00000100:00100000:1.0:1789760714.870481:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 694314758511 time 694382736308 state 0x2 ref 0 00000100:00100000:1.0:1789760714.874517:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760714.874599:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512564096 00000100:00100000:3.0:1789760714.874609:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512564096 00000100:00100000:3.0:1789760714.874662:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 694314758511 time 694382736308 state 0x0 ref 1 00000100:00100000:3.0:1789760714.874680:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 694382996724 time 694382736308 state 0x1 ref 1 00000100:00100000:3.0:1789760714.874743:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 694382996724 time 694439042658 state 0x0 ref 1 00000100:00100000:3.0:1789760714.874756:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4058e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512564096:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760714.874806:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22573056, mtime: 0x6aad94ca, atime: 0x0, ctime: 0x6aad94ca, blocks: 0xb007 00010000:00010000:3.0:1789760714.874815:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2d7db lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.874830:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=694s v=(0 0) 00010000:00010000:3.0:1789760714.874834:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760714.874840:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2d7db lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22573056->22577151] (req 22573056->22577151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.874854:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2d7db lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22573056->22577151] (req 22573056->22577151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760714.931913:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972830 is committed 00002000:00010000:1.0:1789760714.932545:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22573056 -> 22577152 00010000:00010000:1.0:1789760714.932560:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2d7db lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22573056->22577151] (req 22573056->22577151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.932572:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2d7db lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22573056->22577151] (req 22573056->22577151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.932584:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2d7db lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22573056->22577151] (req 22573056->22577151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.932591:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760714.932596:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2d7db lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22573056->22577151] (req 22573056->22577151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.932607:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2d7db lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22573056->22577151] (req 22573056->22577151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.932626:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2d7db lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22573056->22577151] (req 22573056->22577151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.932637:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2d7db lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22573056->22577151] (req 22573056->22577151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760714.932764:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4058e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512564096:12345-192.168.203.56@tcp:4:dd.0 Request processed in 58006us (58251us total) trans 4294972830 rc 0/0 00000100:00100000:1.0:1789760714.932774:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5541 00000100:00100000:1.0:1789760714.932783:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 694382996724 time 694439042658 state 0x2 ref 0 00000100:00100000:2.0:1789760714.938758:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760714.938953:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512564224 00000100:00100000:3.0:1789760714.938962:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512564224 00000100:00100000:3.0:1789760714.939013:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 694382996724 time 694439042658 state 0x0 ref 1 00000100:00100000:3.0:1789760714.939028:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 694439303074 time 694439042658 state 0x1 ref 1 00000100:00100000:3.0:1789760714.939058:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 694439303074 time 694503357607 state 0x0 ref 1 00000100:00100000:3.0:1789760714.939081:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5306a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512564224:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760714.939129:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22577152, mtime: 0x6aad94ca, atime: 0x0, ctime: 0x6aad94ca, blocks: 0xb007 00010000:00010000:3.0:1789760714.939138:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2d7e2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.939153:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=694s v=(0 0) 00010000:00010000:3.0:1789760714.939157:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760714.939164:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2d7e2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22577152->22581247] (req 22577152->22581247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.939177:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2d7e2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22577152->22581247] (req 22577152->22581247) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760714.995140:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972831 is committed 00002000:00010000:1.0:1789760714.996082:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22577152 -> 22581248 00010000:00010000:1.0:1789760714.996098:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2d7e2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22577152->22581247] (req 22577152->22581247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.996111:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2d7e2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22577152->22581247] (req 22577152->22581247) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.996120:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2d7e2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22577152->22581247] (req 22577152->22581247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.996128:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760714.996133:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2d7e2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22577152->22581247] (req 22577152->22581247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.996142:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2d7e2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22577152->22581247] (req 22577152->22581247) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.996165:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2d7e2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22577152->22581247] (req 22577152->22581247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.996173:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2d7e2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22577152->22581247] (req 22577152->22581247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760714.996279:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5306a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512564224:12345-192.168.203.56@tcp:4:dd.0 Request processed in 57196us (57525us total) trans 4294972831 rc 0/0 00000100:00100000:1.0:1789760714.996289:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5542 00000100:00100000:1.0:1789760714.996299:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 694439303074 time 694503357607 state 0x2 ref 0 00000100:00100000:2.0:1789760715.002467:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760715.007116:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512564352 00000100:00100000:1.0:1789760715.007132:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512564352 00000100:00100000:1.0:1789760715.007225:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 694439303074 time 694503357607 state 0x0 ref 1 00000100:00100000:1.0:1789760715.007239:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 694503618023 time 694503357607 state 0x1 ref 1 00000100:00100000:1.0:1789760715.007259:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 694503618023 time 694571559724 state 0x0 ref 1 00000100:00100000:1.0:1789760715.007272:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5305880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512564352:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760715.007322:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22581248, mtime: 0x6aad94ca, atime: 0x0, ctime: 0x6aad94ca, blocks: 0xb007 00010000:00010000:1.0:1789760715.007330:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2d7e9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.007346:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=694s v=(0 0) 00010000:00010000:1.0:1789760715.007351:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760715.007358:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2d7e9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22581248->22585343] (req 22581248->22585343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.007371:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2d7e9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22581248->22585343] (req 22581248->22585343) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1789760715.008115:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00010000:2.0:1789760715.078652:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22581248 -> 22585344 00010000:00010000:2.0:1789760715.078670:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2d7e9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22581248->22585343] (req 22581248->22585343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.078688:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2d7e9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22581248->22585343] (req 22581248->22585343) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.078714:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2d7e9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22581248->22585343] (req 22581248->22585343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.078723:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760715.078728:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2d7e9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22581248->22585343] (req 22581248->22585343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.078736:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2d7e9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22581248->22585343] (req 22581248->22585343) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.078753:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2d7e9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22581248->22585343] (req 22581248->22585343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.078762:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2d7e9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22581248->22585343] (req 22581248->22585343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760715.078853:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5305880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2360:x1876699512564352:12345-192.168.203.56@tcp:4:dd.0 Request processed in 71580us (76391us total) trans 4294972832 rc 0/0 00000100:00100000:2.0:1789760715.078863:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5543 00000100:00100000:2.0:1789760715.078872:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 694503618023 time 694571559724 state 0x2 ref 0 00000001:00080000:2.0:1789760715.078910:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972832 is committed 00000100:00100000:0.0:1789760715.085106:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760715.085219:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512564480 00000100:00100000:1.0:1789760715.085229:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512564480 00000100:00100000:1.0:1789760715.085281:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 694503618023 time 694571559724 state 0x0 ref 1 00000100:00100000:1.0:1789760715.085298:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 694571820140 time 694571559724 state 0x1 ref 1 00000100:00100000:1.0:1789760715.085331:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 694571820140 time 694649629303 state 0x0 ref 1 00000100:00100000:1.0:1789760715.085344:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febffb9f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512564480:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760715.085394:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22585344, mtime: 0x6aad94ca, atime: 0x0, ctime: 0x6aad94ca, blocks: 0xb007 00010000:00010000:1.0:1789760715.085405:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2d7f0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.085421:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=694s v=(0 0) 00010000:00010000:1.0:1789760715.085426:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760715.085433:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2d7f0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22585344->22589439] (req 22585344->22589439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.085447:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2d7f0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22585344->22589439] (req 22585344->22589439) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.127440:0:5835:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:3.0:1789760715.157692:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972833 is committed 00002000:00010000:0.0:1789760715.157884:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22585344 -> 22589440 00010000:00010000:0.0:1789760715.157900:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2d7f0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22585344->22589439] (req 22585344->22589439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.157912:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2d7f0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22585344->22589439] (req 22585344->22589439) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.157920:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2d7f0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22585344->22589439] (req 22585344->22589439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.157927:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760715.157932:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2d7f0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22585344->22589439] (req 22585344->22589439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.157946:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2d7f0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22585344->22589439] (req 22585344->22589439) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.157965:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2d7f0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22585344->22589439] (req 22585344->22589439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.157973:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2d7f0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22585344->22589439] (req 22585344->22589439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760715.158074:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febffb9f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512564480:12345-192.168.203.56@tcp:4:dd.0 Request processed in 72718us (72962us total) trans 4294972833 rc 0/0 00000100:00100000:0.0:1789760715.158084:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5544 00000100:00100000:0.0:1789760715.158093:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 694571820140 time 694649629303 state 0x2 ref 0 00000100:00100000:0.0:1789760715.165597:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760715.165693:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512564608 00000100:00100000:1.0:1789760715.165736:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512564608 00000100:00100000:1.0:1789760715.165792:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 694571820140 time 694649629303 state 0x0 ref 1 00000100:00100000:1.0:1789760715.165807:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 694649889719 time 694649629303 state 0x1 ref 1 00000100:00100000:1.0:1789760715.165827:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 694649889719 time 694730127668 state 0x0 ref 1 00000100:00100000:1.0:1789760715.165839:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febffb9e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512564608:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760715.165885:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22589440, mtime: 0x6aad94ca, atime: 0x0, ctime: 0x6aad94ca, blocks: 0xb007 00010000:00010000:1.0:1789760715.165894:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2d7f7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.165921:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=694s v=(0 0) 00010000:00010000:1.0:1789760715.165926:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760715.165940:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2d7f7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22589440->22593535] (req 22589440->22593535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.165954:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2d7f7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22589440->22593535] (req 22589440->22593535) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760715.211673:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972834 is committed 00002000:00010000:0.0:1789760715.212355:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22589440 -> 22593536 00010000:00010000:0.0:1789760715.212375:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2d7f7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22589440->22593535] (req 22589440->22593535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.212388:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2d7f7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22589440->22593535] (req 22589440->22593535) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.212397:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2d7f7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22589440->22593535] (req 22589440->22593535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.212422:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760715.212427:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2d7f7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22589440->22593535] (req 22589440->22593535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.212436:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2d7f7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22589440->22593535] (req 22589440->22593535) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.212457:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2d7f7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22589440->22593535] (req 22589440->22593535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.212464:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2d7f7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22589440->22593535] (req 22589440->22593535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760715.212575:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febffb9e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512564608:12345-192.168.203.56@tcp:4:dd.0 Request processed in 46732us (46981us total) trans 4294972834 rc 0/0 00000100:00100000:0.0:1789760715.212588:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5545 00000100:00100000:0.0:1789760715.212597:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 694649889719 time 694730127668 state 0x2 ref 0 00000100:00100000:3.0:1789760715.216522:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760715.216750:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512564736 00000100:00100000:2.0:1789760715.216760:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512564736 00000100:00100000:2.0:1789760715.216812:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 694649889719 time 694730127668 state 0x0 ref 1 00000100:00100000:2.0:1789760715.216828:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 694730388084 time 694730127668 state 0x1 ref 1 00000100:00100000:2.0:1789760715.216862:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 694730388084 time 694781161729 state 0x0 ref 1 00000100:00100000:2.0:1789760715.216873:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2744a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512564736:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760715.216920:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22593536, mtime: 0x6aad94ca, atime: 0x0, ctime: 0x6aad94ca, blocks: 0xb007 00010000:00010000:2.0:1789760715.216929:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2d7fe lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.216945:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=694s v=(0 0) 00010000:00010000:2.0:1789760715.216949:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760715.216957:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2d7fe lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22593536->22597631] (req 22593536->22597631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.216970:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2d7fe lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22593536->22597631] (req 22593536->22597631) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760715.267483:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22593536 -> 22597632 00010000:00010000:2.0:1789760715.267503:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2d7fe lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22593536->22597631] (req 22593536->22597631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.267516:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2d7fe lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22593536->22597631] (req 22593536->22597631) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.267525:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2d7fe lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22593536->22597631] (req 22593536->22597631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.267542:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760715.267547:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2d7fe lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22593536->22597631] (req 22593536->22597631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.267556:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2d7fe lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22593536->22597631] (req 22593536->22597631) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.267575:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2d7fe lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22593536->22597631] (req 22593536->22597631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.267582:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2d7fe lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22593536->22597631] (req 22593536->22597631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760715.267660:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2744a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2358:x1876699512564736:12345-192.168.203.56@tcp:4:dd.0 Request processed in 50783us (51146us total) trans 4294972835 rc 0/0 00000100:00100000:2.0:1789760715.267670:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5546 00000100:00100000:2.0:1789760715.267679:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 694730388084 time 694781161729 state 0x2 ref 0 00000001:00080000:3.0:1789760715.268576:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972835 is committed 00000100:00100000:1.0:1789760715.270911:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760715.271011:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512564864 00000100:00100000:0.0:1789760715.271022:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512564864 00000100:00100000:0.0:1789760715.271091:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 694730388084 time 694781161729 state 0x0 ref 1 00000100:00100000:0.0:1789760715.271107:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 694781422145 time 694781161729 state 0x1 ref 1 00000100:00100000:0.0:1789760715.271139:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 694781422145 time 694835438427 state 0x0 ref 1 00000100:00100000:0.0:1789760715.271156:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febfb89ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512564864:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760715.271335:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22597632, mtime: 0x6aad94ca, atime: 0x0, ctime: 0x6aad94ca, blocks: 0xb007 00010000:00010000:0.0:1789760715.271344:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2d805 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.271359:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=694s v=(0 0) 00010000:00010000:0.0:1789760715.271363:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760715.271370:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2d805 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22597632->22601727] (req 22597632->22601727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.271383:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2d805 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22597632->22601727] (req 22597632->22601727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760715.336588:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972836 is committed 00002000:00010000:2.0:1789760715.336716:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22597632 -> 22601728 00010000:00010000:2.0:1789760715.336730:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2d805 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22597632->22601727] (req 22597632->22601727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.336749:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2d805 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22597632->22601727] (req 22597632->22601727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.336759:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2d805 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22597632->22601727] (req 22597632->22601727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.336767:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760715.336772:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2d805 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22597632->22601727] (req 22597632->22601727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.336780:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2d805 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22597632->22601727] (req 22597632->22601727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.336799:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2d805 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22597632->22601727] (req 22597632->22601727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.336808:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2d805 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22597632->22601727] (req 22597632->22601727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760715.336905:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febfb89ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512564864:12345-192.168.203.56@tcp:4:dd.0 Request processed in 65748us (65998us total) trans 4294972836 rc 0/0 00000100:00100000:2.0:1789760715.336916:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5547 00000100:00100000:2.0:1789760715.336936:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 694781422145 time 694835438427 state 0x2 ref 0 00000100:00100000:0.0:1789760715.344414:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760715.344752:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512564992 00000100:00100000:1.0:1789760715.344763:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512564992 00000100:00100000:1.0:1789760715.344821:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 694781422145 time 694835438427 state 0x0 ref 1 00000100:00100000:1.0:1789760715.344837:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 694835698843 time 694835438427 state 0x1 ref 1 00000100:00100000:1.0:1789760715.344866:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 694835698843 time 694909165952 state 0x0 ref 1 00000100:00100000:1.0:1789760715.344879:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febffb9ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512564992:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760715.344927:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22601728, mtime: 0x6aad94ca, atime: 0x0, ctime: 0x6aad94ca, blocks: 0xb007 00010000:00010000:1.0:1789760715.344937:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2d80c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.344952:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=694s v=(0 0) 00010000:00010000:1.0:1789760715.344956:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760715.344962:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2d80c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22601728->22605823] (req 22601728->22605823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.344976:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2d80c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22601728->22605823] (req 22601728->22605823) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760715.419589:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972837 is committed 00002000:00010000:2.0:1789760715.419842:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22601728 -> 22605824 00010000:00010000:2.0:1789760715.419859:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2d80c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22601728->22605823] (req 22601728->22605823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.419871:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2d80c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22601728->22605823] (req 22601728->22605823) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.419880:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2d80c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22601728->22605823] (req 22601728->22605823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.419887:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760715.419892:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2d80c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22601728->22605823] (req 22601728->22605823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.419902:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2d80c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22601728->22605823] (req 22601728->22605823) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.419932:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2d80c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22601728->22605823] (req 22601728->22605823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.419941:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2d80c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22601728->22605823] (req 22601728->22605823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760715.420042:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febffb9ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512564992:12345-192.168.203.56@tcp:4:dd.0 Request processed in 75159us (75643us total) trans 4294972837 rc 0/0 00000100:00100000:2.0:1789760715.420052:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5548 00000100:00100000:2.0:1789760715.420062:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 694835698843 time 694909165952 state 0x2 ref 0 00000100:00100000:1.0:1789760715.422994:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760715.423119:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512565120 00000100:00100000:2.0:1789760715.423129:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512565120 00000100:00100000:2.0:1789760715.423182:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 694835698843 time 694909165952 state 0x0 ref 1 00000100:00100000:2.0:1789760715.423207:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 694909426368 time 694909165952 state 0x1 ref 1 00000100:00100000:2.0:1789760715.423235:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 694909426368 time 694987534222 state 0x0 ref 1 00000100:00100000:2.0:1789760715.423246:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febfb89fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512565120:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760715.423294:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22605824, mtime: 0x6aad94ca, atime: 0x0, ctime: 0x6aad94ca, blocks: 0xb007 00010000:00010000:2.0:1789760715.423303:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2d813 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.423318:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=694s v=(0 0) 00010000:00010000:2.0:1789760715.423322:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760715.423328:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2d813 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22605824->22609919] (req 22605824->22609919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.423341:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2d813 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22605824->22609919] (req 22605824->22609919) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760715.476884:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972838 is committed 00002000:00010000:1.0:1789760715.477025:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22605824 -> 22609920 00010000:00010000:1.0:1789760715.477040:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2d813 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22605824->22609919] (req 22605824->22609919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.477051:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2d813 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22605824->22609919] (req 22605824->22609919) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.477059:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2d813 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22605824->22609919] (req 22605824->22609919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.477066:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760715.477071:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2d813 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22605824->22609919] (req 22605824->22609919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.477079:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2d813 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22605824->22609919] (req 22605824->22609919) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.477100:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2d813 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22605824->22609919] (req 22605824->22609919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.477107:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2d813 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22605824->22609919] (req 22605824->22609919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760715.477260:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febfb89fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512565120:12345-192.168.203.56@tcp:4:dd.0 Request processed in 53949us (54209us total) trans 4294972838 rc 0/0 00000100:00100000:1.0:1789760715.477272:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5549 00000100:00100000:1.0:1789760715.477282:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 694909426368 time 694987534222 state 0x2 ref 0 00000100:00100000:1.0:1789760715.480526:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760715.482843:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512565248 00000100:00100000:2.0:1789760715.482855:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512565248 00000100:00100000:2.0:1789760715.482913:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 694909426368 time 694987534222 state 0x0 ref 1 00000100:00100000:2.0:1789760715.482996:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 694987794638 time 694987534222 state 0x1 ref 1 00000100:00100000:2.0:1789760715.483075:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 694987794638 time 695047374297 state 0x0 ref 1 00000100:00100000:2.0:1789760715.483089:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac28a1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512565248:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760715.483147:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22609920, mtime: 0x6aad94ca, atime: 0x0, ctime: 0x6aad94ca, blocks: 0xb007 00010000:00010000:2.0:1789760715.483156:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2d81a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.483171:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=695s v=(0 0) 00010000:00010000:2.0:1789760715.483175:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760715.483181:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2d81a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22609920->22614015] (req 22609920->22614015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.483194:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2d81a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22609920->22614015] (req 22609920->22614015) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760715.539288:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972839 is committed 00002000:00010000:2.0:1789760715.539405:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22609920 -> 22614016 00010000:00010000:2.0:1789760715.539423:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2d81a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22609920->22614015] (req 22609920->22614015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.539436:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2d81a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22609920->22614015] (req 22609920->22614015) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.539444:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2d81a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22609920->22614015] (req 22609920->22614015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.539452:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760715.539456:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2d81a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22609920->22614015] (req 22609920->22614015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.539466:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2d81a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22609920->22614015] (req 22609920->22614015) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.539486:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2d81a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22609920->22614015] (req 22609920->22614015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.539493:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2d81a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22609920->22614015] (req 22609920->22614015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760715.539601:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac28a1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512565248:12345-192.168.203.56@tcp:4:dd.0 Request processed in 56510us (59079us total) trans 4294972839 rc 0/0 00000100:00100000:2.0:1789760715.539611:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5550 00000100:00100000:2.0:1789760715.539620:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 694987794638 time 695047374297 state 0x2 ref 0 00000100:00100000:0.0:1789760715.543847:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760715.543984:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512565376 00000100:00100000:2.0:1789760715.543996:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512565376 00000100:00100000:2.0:1789760715.544062:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 694987794638 time 695047374297 state 0x0 ref 1 00000100:00100000:2.0:1789760715.544080:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 695047634713 time 695047374297 state 0x1 ref 1 00000100:00100000:2.0:1789760715.544110:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 695047634713 time 695108409817 state 0x0 ref 1 00000100:00100000:2.0:1789760715.544123:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febffb9c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512565376:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760715.544186:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22614016, mtime: 0x6aad94ca, atime: 0x0, ctime: 0x6aad94ca, blocks: 0xb007 00010000:00010000:2.0:1789760715.544205:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2d821 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.544222:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=695s v=(0 0) 00010000:00010000:2.0:1789760715.544227:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760715.544233:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2d821 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22614016->22618111] (req 22614016->22618111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.544247:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2d821 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22614016->22618111] (req 22614016->22618111) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760715.608712:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972840 is committed 00002000:00010000:1.0:1789760715.608798:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22614016 -> 22618112 00002000:00010000:1.0:1789760715.608803:0:13062:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760714 -> 1789760715 00002000:00010000:1.0:1789760715.608805:0:13062:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760714 -> 1789760715 00010000:00010000:1.0:1789760715.608819:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2d821 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22614016->22618111] (req 22614016->22618111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.608833:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2d821 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22614016->22618111] (req 22614016->22618111) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.608843:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2d821 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22614016->22618111] (req 22614016->22618111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.608850:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760715.608855:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2d821 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22614016->22618111] (req 22614016->22618111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.608865:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2d821 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22614016->22618111] (req 22614016->22618111) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.608883:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2d821 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22614016->22618111] (req 22614016->22618111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.608890:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2d821 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22614016->22618111] (req 22614016->22618111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760715.608981:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febffb9c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512565376:12345-192.168.203.56@tcp:4:dd.0 Request processed in 64856us (65144us total) trans 4294972840 rc 0/0 00000100:00100000:1.0:1789760715.608991:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5551 00000100:00100000:1.0:1789760715.609001:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 695047634713 time 695108409817 state 0x2 ref 0 00000100:00100000:1.0:1789760715.619194:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760715.619408:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512565504 00000100:00100000:2.0:1789760715.619419:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512565504 00000100:00100000:2.0:1789760715.619476:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 695047634713 time 695108409817 state 0x0 ref 1 00000100:00100000:2.0:1789760715.619493:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 695108670233 time 695108409817 state 0x1 ref 1 00000100:00100000:2.0:1789760715.619523:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 695108670233 time 695183822924 state 0x0 ref 1 00000100:00100000:2.0:1789760715.619548:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac28a2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512565504:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760715.619597:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22618112, mtime: 0x6aad94cb, atime: 0x0, ctime: 0x6aad94cb, blocks: 0xb007 00010000:00010000:2.0:1789760715.619605:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2d828 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.619620:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=695s v=(0 0) 00010000:00010000:2.0:1789760715.619625:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760715.619630:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2d828 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22618112->22622207] (req 22618112->22622207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.619643:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2d828 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22618112->22622207] (req 22618112->22622207) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760715.656944:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972841 is committed 00002000:00010000:2.0:1789760715.657076:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22618112 -> 22622208 00010000:00010000:2.0:1789760715.657092:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2d828 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22618112->22622207] (req 22618112->22622207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.657104:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2d828 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22618112->22622207] (req 22618112->22622207) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.657114:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2d828 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22618112->22622207] (req 22618112->22622207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.657120:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760715.657125:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2d828 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22618112->22622207] (req 22618112->22622207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.657133:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2d828 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22618112->22622207] (req 22618112->22622207) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.657151:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2d828 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22618112->22622207] (req 22618112->22622207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.657280:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2d828 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22618112->22622207] (req 22618112->22622207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760715.657383:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac28a2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512565504:12345-192.168.203.56@tcp:4:dd.0 Request processed in 37833us (38193us total) trans 4294972841 rc 0/0 00000100:00100000:2.0:1789760715.657393:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5552 00000100:00100000:2.0:1789760715.657402:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 695108670233 time 695183822924 state 0x2 ref 0 00000100:00100000:0.0:1789760715.662192:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760715.662292:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512565632 00000100:00100000:2.0:1789760715.662306:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512565632 00000100:00100000:2.0:1789760715.662377:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 695108670233 time 695183822924 state 0x0 ref 1 00000100:00100000:2.0:1789760715.662396:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 695184083340 time 695183822924 state 0x1 ref 1 00000100:00100000:2.0:1789760715.662437:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 695184083340 time 695226735752 state 0x0 ref 1 00000100:00100000:2.0:1789760715.662452:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febffb9c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512565632:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760715.662509:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22622208, mtime: 0x6aad94cb, atime: 0x0, ctime: 0x6aad94cb, blocks: 0xb007 00010000:00010000:2.0:1789760715.662517:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2d82f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.662550:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=695s v=(0 0) 00010000:00010000:2.0:1789760715.662555:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760715.662561:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2d82f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22622208->22626303] (req 22622208->22626303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.662576:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2d82f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22622208->22626303] (req 22622208->22626303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760715.699463:0:3309:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8feac4362d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3309:1876699540218880:0@lo:400:kworker.0 00000100:00100000:1.0:1789760715.699543:0:3309:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760715.699604:0:3309:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:1.0:1789760715.699611:0:3309:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8feac28a0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:6df98d54-eb52-4c5c-8fc9-ee9d74f90f98:3309:1876699540218496:0@lo:400:kworker.0 00000100:00100000:1.0:1789760715.699638:0:3309:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760715.699661:0:3309:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8febc7b33480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3309:1876699540219008:0@lo:400:kworker.0 00000100:00100000:1.0:1789760715.699685:0:3309:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760715.699730:0:3311:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8feac28a3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3311:1876699540218624:0@lo:400:kworker.0 00000100:00100000:1.0:1789760715.699753:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760715.699769:0:3311:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8febc7b32680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3311:1876699540219136:0@lo:400:kworker.0 00000100:00100000:1.0:1789760715.699788:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760715.699805:0:3311:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1789760715.699811:0:3311:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8febfbeae300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3311:1876699540218752:0@lo:400:kworker.0 00000100:00100000:1.0:1789760715.699832:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760715.699863:0:6704:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540218880 00000100:00100000:1.0:1789760715.699870:0:6704:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540218880 00000100:00100000:1.0:1789760715.699919:0:6704:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540219136 00000100:00100000:1.0:1789760715.699923:0:6704:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540219136 00000100:00100000:1.0:1789760715.699946:0:6704:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 171 00000100:00100000:1.0:1789760715.699955:0:6704:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac38b7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3309:x1876699540218880:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760715.700006:0:6704:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac38b7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3309:x1876699540218880:12345-0@lo:400:kworker.0 Request processed in 52us (476us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760715.700013:0:6704:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 171 00000100:00100000:1.0:1789760715.700031:0:6704:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 172 00000100:00100000:1.0:1789760715.700036:0:6704:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac38b4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3311:x1876699540219136:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760715.700069:0:6704:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac38b4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3311:x1876699540219136:12345-0@lo:400:kworker.0 Request processed in 33us (279us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760715.700074:0:6704:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 172 00000100:00100000:1.0:1789760715.700091:0:5842:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540218496 00000100:00100000:1.0:1789760715.700094:0:5842:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540218496 00000100:00100000:1.0:1789760715.700117:0:5842:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 387 00000100:00100000:1.0:1789760715.700123:0:5842:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febe0502850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:6df98d54-eb52-4c5c-8fc9-ee9d74f90f98+19:3309:x1876699540218496:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760715.700167:0:5842:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febe0502850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:6df98d54-eb52-4c5c-8fc9-ee9d74f90f98+19:3309:x1876699540218496:12345-0@lo:400:kworker.0 Request processed in 29us (513us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760715.700173:0:5842:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 387 00000100:00100000:1.0:1789760715.700236:0:3311:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8febc7b32680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3311:1876699540219136:0@lo:400:kworker.0 00000100:00100000:1.0:1789760715.700263:0:3309:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8feac28a0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:6df98d54-eb52-4c5c-8fc9-ee9d74f90f98:3309:1876699540218496:0@lo:400:kworker.0 00000100:00100000:1.0:1789760715.700274:0:3309:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8feac4362d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3309:1876699540218880:0@lo:400:kworker.0 00000100:00100000:2.0:1789760715.700570:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540219008 00000100:00100000:2.0:1789760715.700575:0:5847:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540219008 00000100:00100000:2.0:1789760715.700611:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540218624 00000100:00100000:2.0:1789760715.700614:0:5847:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540218624 00000100:00100000:2.0:1789760715.700626:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540218752 00000100:00100000:2.0:1789760715.700628:0:5847:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540218752 00000100:00100000:2.0:1789760715.700641:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 446 00000100:00100000:2.0:1789760715.700647:0:5847:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac38b4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3309:x1876699540219008:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760715.700687:0:5847:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac38b4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3309:x1876699540219008:12345-0@lo:400:kworker.0 Request processed in 41us (1001us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760715.700693:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 446 00000100:00100000:2.0:1789760715.700724:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 447 00000100:00100000:2.0:1789760715.700729:0:5847:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac38b7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3311:x1876699540218624:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760715.700757:0:5847:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac38b7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3311:x1876699540218624:12345-0@lo:400:kworker.0 Request processed in 28us (1003us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760715.700762:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 447 00000100:00100000:2.0:1789760715.700772:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 448 00000100:00100000:2.0:1789760715.700777:0:5847:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac38b4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3311:x1876699540218752:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760715.700799:0:5847:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac38b4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3311:x1876699540218752:12345-0@lo:400:kworker.0 Request processed in 22us (966us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760715.700804:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 448 00000100:00080000:2.0:1789760715.700822:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:2.0:1789760715.700826:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:2.0:1789760715.700828:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1789760715.700846:0:3311:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8febfbeae300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3311:1876699540218752:0@lo:400:kworker.0 00000100:00100000:2.0:1789760715.700857:0:3311:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8feac28a3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3311:1876699540218624:0@lo:400:kworker.0 00000100:00100000:2.0:1789760715.700879:0:3309:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8febc7b33480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3309:1876699540219008:0@lo:400:kworker.0 00000100:00100000:1.0:1789760715.702688:0:3312:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8febffb9f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3312:1876699540219264:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1789760715.702766:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760715.702874:0:6707:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540219264 00000100:00100000:0.0:1789760715.702882:0:6707:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540219264 00000100:00100000:0.0:1789760715.702926:0:6707:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 186 00000100:00100000:0.0:1789760715.702934:0:6707:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac38b5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3312:x1876699540219264:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1789760715.702956:0:6707:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 528285696 pending 1703936 free 3835691008 avail 3813670912 00000100:00100000:0.0:1789760715.702996:0:6707:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac38b5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3312:x1876699540219264:12345-0@lo:13:osp-pre-0-0.0 Request processed in 63us (230us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760715.703002:0:6707:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 186 00000100:00100000:1.0:1789760715.703093:0:3312:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8febffb9f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3312:1876699540219264:0@lo:13:osp-pre-0-0.0 00002000:00010000:0.0:1789760715.731338:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22622208 -> 22626304 00010000:00010000:0.0:1789760715.731357:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2d82f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22622208->22626303] (req 22622208->22626303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.731368:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2d82f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22622208->22626303] (req 22622208->22626303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.731376:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2d82f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22622208->22626303] (req 22622208->22626303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.731383:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760715.731387:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2d82f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22622208->22626303] (req 22622208->22626303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.731396:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2d82f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22622208->22626303] (req 22622208->22626303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.731426:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2d82f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22622208->22626303] (req 22622208->22626303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.731434:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2d82f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22622208->22626303] (req 22622208->22626303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760715.731524:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febffb9c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2359:x1876699512565632:12345-192.168.203.56@tcp:4:dd.0 Request processed in 69072us (69339us total) trans 4294972842 rc 0/0 00000100:00100000:0.0:1789760715.731546:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5553 00000100:00100000:0.0:1789760715.731555:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 695184083340 time 695226735752 state 0x2 ref 0 00000001:00080000:0.0:1789760715.734869:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972842 is committed 00000100:00100000:0.0:1789760715.743238:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760715.743403:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512565760 00000100:00100000:1.0:1789760715.743413:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512565760 00000100:00100000:1.0:1789760715.743465:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 695184083340 time 695226735752 state 0x0 ref 1 00000100:00100000:1.0:1789760715.743479:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 695226996168 time 695226735752 state 0x1 ref 1 00000100:00100000:1.0:1789760715.743615:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 695226996168 time 695307914021 state 0x0 ref 1 00000100:00100000:1.0:1789760715.743630:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febffb9d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512565760:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760715.743675:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22626304, mtime: 0x6aad94cb, atime: 0x0, ctime: 0x6aad94cb, blocks: 0xb007 00010000:00010000:1.0:1789760715.743684:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2d836 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.743740:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=695s v=(0 0) 00010000:00010000:1.0:1789760715.743746:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760715.743752:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2d836 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22626304->22630399] (req 22626304->22630399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.743764:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2d836 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22626304->22630399] (req 22626304->22630399) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760715.814369:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972843 is committed 00002000:00010000:2.0:1789760715.814486:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22626304 -> 22630400 00010000:00010000:2.0:1789760715.814501:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2d836 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22626304->22630399] (req 22626304->22630399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.814514:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2d836 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22626304->22630399] (req 22626304->22630399) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.814522:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2d836 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22626304->22630399] (req 22626304->22630399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.814610:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760715.814614:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2d836 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22626304->22630399] (req 22626304->22630399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.814623:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2d836 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22626304->22630399] (req 22626304->22630399) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.814641:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2d836 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22626304->22630399] (req 22626304->22630399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.814649:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2d836 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22626304->22630399] (req 22626304->22630399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760715.814814:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febffb9d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512565760:12345-192.168.203.56@tcp:4:dd.0 Request processed in 71182us (71582us total) trans 4294972843 rc 0/0 00000100:00100000:2.0:1789760715.814826:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5554 00000100:00100000:2.0:1789760715.814836:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 695226996168 time 695307914021 state 0x2 ref 0 00000100:00100000:3.0:1789760715.818140:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760715.818845:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512565888 00000100:00100000:2.0:1789760715.818857:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512565888 00000100:00100000:2.0:1789760715.819274:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 695226996168 time 695307914021 state 0x0 ref 1 00000100:00100000:2.0:1789760715.819292:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 695308174437 time 695307914021 state 0x1 ref 1 00000100:00100000:2.0:1789760715.819325:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 695308174437 time 695383625052 state 0x0 ref 1 00000100:00100000:2.0:1789760715.819339:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febea4ba450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512565888:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760715.819387:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22630400, mtime: 0x6aad94cb, atime: 0x0, ctime: 0x6aad94cb, blocks: 0xb007 00010000:00010000:2.0:1789760715.819396:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2d83d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.819410:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=695s v=(0 0) 00010000:00010000:2.0:1789760715.819415:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760715.819421:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2d83d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22630400->22634495] (req 22630400->22634495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.819434:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2d83d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22630400->22634495] (req 22630400->22634495) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760715.878572:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22630400 -> 22634496 00010000:00010000:0.0:1789760715.878595:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2d83d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22630400->22634495] (req 22630400->22634495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.878608:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2d83d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22630400->22634495] (req 22630400->22634495) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.878617:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2d83d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22630400->22634495] (req 22630400->22634495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.878626:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760715.878631:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2d83d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22630400->22634495] (req 22630400->22634495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.878640:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2d83d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22630400->22634495] (req 22630400->22634495) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.878660:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2d83d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22630400->22634495] (req 22630400->22634495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.878669:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2d83d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22630400->22634495] (req 22630400->22634495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760715.878800:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febea4ba450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2359:x1876699512565888:12345-192.168.203.56@tcp:4:dd.0 Request processed in 59459us (60664us total) trans 4294972844 rc 0/0 00000100:00100000:0.0:1789760715.878811:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5555 00000100:00100000:0.0:1789760715.878820:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 695308174437 time 695383625052 state 0x2 ref 0 00000001:00080000:3.0:1789760715.879487:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972844 is committed 00000100:00100000:1.0:1789760715.886199:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760715.886287:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512566016 00000100:00100000:0.0:1789760715.886297:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512566016 00000100:00100000:0.0:1789760715.886352:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 695308174437 time 695383625052 state 0x0 ref 1 00000100:00100000:0.0:1789760715.886367:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 695383885468 time 695383625052 state 0x1 ref 1 00000100:00100000:0.0:1789760715.886408:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 695383885468 time 695450699215 state 0x0 ref 1 00000100:00100000:0.0:1789760715.886423:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac38b6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512566016:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760715.886475:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22634496, mtime: 0x6aad94cb, atime: 0x0, ctime: 0x6aad94cb, blocks: 0xb007 00010000:00010000:0.0:1789760715.886484:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2d844 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.886501:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=695s v=(0 0) 00010000:00010000:0.0:1789760715.886506:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760715.886513:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2d844 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22634496->22638591] (req 22634496->22638591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.886536:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2d844 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22634496->22638591] (req 22634496->22638591) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760715.933816:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972845 is committed 00002000:00010000:2.0:1789760715.933985:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22634496 -> 22638592 00010000:00010000:2.0:1789760715.934001:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2d844 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22634496->22638591] (req 22634496->22638591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.934014:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2d844 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22634496->22638591] (req 22634496->22638591) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.934022:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2d844 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22634496->22638591] (req 22634496->22638591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.934029:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760715.934033:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2d844 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22634496->22638591] (req 22634496->22638591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.934041:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2d844 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22634496->22638591] (req 22634496->22638591) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.934058:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2d844 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22634496->22638591] (req 22634496->22638591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.934067:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2d844 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22634496->22638591] (req 22634496->22638591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760715.934160:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac38b6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512566016:12345-192.168.203.56@tcp:4:dd.0 Request processed in 47735us (47965us total) trans 4294972845 rc 0/0 00000100:00100000:2.0:1789760715.934171:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5556 00000100:00100000:2.0:1789760715.934181:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 695383885468 time 695450699215 state 0x2 ref 0 00000100:00100000:2.0:1789760715.940770:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760715.940869:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512566144 00000100:00100000:1.0:1789760715.940879:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512566144 00000100:00100000:1.0:1789760715.940948:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 695383885468 time 695450699215 state 0x0 ref 1 00000100:00100000:1.0:1789760715.940962:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 695450959631 time 695450699215 state 0x1 ref 1 00000100:00100000:1.0:1789760715.940992:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 695450959631 time 695505291945 state 0x0 ref 1 00000100:00100000:1.0:1789760715.941004:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5305180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512566144:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760715.941051:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22638592, mtime: 0x6aad94cb, atime: 0x0, ctime: 0x6aad94cb, blocks: 0xb007 00010000:00010000:1.0:1789760715.941061:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2d84b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.941076:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=695s v=(0 0) 00010000:00010000:1.0:1789760715.941081:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760715.941087:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2d84b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22638592->22642687] (req 22638592->22642687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.941101:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2d84b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22638592->22642687] (req 22638592->22642687) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760716.007683:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972846 is committed 00002000:00010000:3.0:1789760716.007902:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22638592 -> 22642688 00010000:00010000:3.0:1789760716.007919:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2d84b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22638592->22642687] (req 22638592->22642687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.007932:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2d84b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22638592->22642687] (req 22638592->22642687) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.007942:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2d84b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22638592->22642687] (req 22638592->22642687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.007956:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760716.007961:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2d84b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22638592->22642687] (req 22638592->22642687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.007970:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2d84b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22638592->22642687] (req 22638592->22642687) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.007990:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2d84b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22638592->22642687] (req 22638592->22642687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.007997:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2d84b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22638592->22642687] (req 22638592->22642687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760716.008098:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5305180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512566144:12345-192.168.203.56@tcp:4:dd.0 Request processed in 67091us (67331us total) trans 4294972846 rc 0/0 00000100:00100000:3.0:1789760716.008108:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5557 00000100:00100000:3.0:1789760716.008118:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 695450959631 time 695505291945 state 0x2 ref 0 00000100:00100000:1.0:1789760716.013163:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760716.013238:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512566272 00000100:00100000:3.0:1789760716.013247:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512566272 00000100:00100000:3.0:1789760716.013301:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 695450959631 time 695505291945 state 0x0 ref 1 00000100:00100000:3.0:1789760716.013317:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 695505552361 time 695505291945 state 0x1 ref 1 00000100:00100000:3.0:1789760716.013346:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 695505552361 time 695577645163 state 0x0 ref 1 00000100:00100000:3.0:1789760716.013362:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac38b4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512566272:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760716.013409:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22642688, mtime: 0x6aad94cb, atime: 0x0, ctime: 0x6aad94cb, blocks: 0xb007 00010000:00010000:3.0:1789760716.013418:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2d852 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.013433:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=695s v=(0 0) 00010000:00010000:3.0:1789760716.013438:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760716.013444:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2d852 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22642688->22646783] (req 22642688->22646783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.013457:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2d852 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22642688->22646783] (req 22642688->22646783) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760716.084460:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972847 is committed 00002000:00010000:1.0:1789760716.086756:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22642688 -> 22646784 00010000:00010000:1.0:1789760716.086777:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2d852 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22642688->22646783] (req 22642688->22646783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.086790:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2d852 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22642688->22646783] (req 22642688->22646783) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.086799:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2d852 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22642688->22646783] (req 22642688->22646783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.086805:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760716.086810:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2d852 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22642688->22646783] (req 22642688->22646783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.086823:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2d852 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22642688->22646783] (req 22642688->22646783) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.086842:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2d852 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22642688->22646783] (req 22642688->22646783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.086850:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2d852 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22642688->22646783] (req 22642688->22646783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760716.087152:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac38b4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512566272:12345-192.168.203.56@tcp:4:dd.0 Request processed in 73788us (73993us total) trans 4294972847 rc 0/0 00000100:00100000:1.0:1789760716.087162:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5558 00000100:00100000:1.0:1789760716.087182:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 695505552361 time 695577645163 state 0x2 ref 0 00000100:00100000:1.0:1789760716.095168:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760716.095294:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512566400 00000100:00100000:0.0:1789760716.095305:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512566400 00000100:00100000:0.0:1789760716.095357:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 695505552361 time 695577645163 state 0x0 ref 1 00000100:00100000:0.0:1789760716.095371:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 695577905579 time 695577645163 state 0x1 ref 1 00000100:00100000:0.0:1789760716.095404:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 695577905579 time 695659703407 state 0x0 ref 1 00000100:00100000:0.0:1789760716.095417:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac38b6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512566400:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760716.095464:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22646784, mtime: 0x6aad94cb, atime: 0x0, ctime: 0x6aad94cb, blocks: 0xb007 00010000:00010000:0.0:1789760716.095474:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2d859 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.095488:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=695s v=(0 0) 00010000:00010000:0.0:1789760716.095492:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760716.095498:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2d859 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22646784->22650879] (req 22646784->22650879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.095510:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2d859 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22646784->22650879] (req 22646784->22650879) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.147041:0:7647:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:3.0:1789760716.154681:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972848 is committed 00002000:00010000:1.0:1789760716.155577:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22646784 -> 22650880 00010000:00010000:1.0:1789760716.155594:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2d859 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22646784->22650879] (req 22646784->22650879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.155608:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2d859 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22646784->22650879] (req 22646784->22650879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.155616:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2d859 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22646784->22650879] (req 22646784->22650879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.155627:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760716.155631:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2d859 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22646784->22650879] (req 22646784->22650879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.155639:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2d859 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22646784->22650879] (req 22646784->22650879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.155655:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2d859 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22646784->22650879] (req 22646784->22650879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.155661:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2d859 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22646784->22650879] (req 22646784->22650879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760716.155915:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac38b6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512566400:12345-192.168.203.56@tcp:4:dd.0 Request processed in 60495us (60751us total) trans 4294972848 rc 0/0 00000100:00100000:1.0:1789760716.155926:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5559 00000100:00100000:1.0:1789760716.155936:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 695577905579 time 695659703407 state 0x2 ref 0 00000100:00100000:2.0:1789760716.162923:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760716.163084:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512566528 00000100:00100000:3.0:1789760716.163095:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512566528 00000100:00100000:3.0:1789760716.163160:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 695577905579 time 695659703407 state 0x0 ref 1 00000100:00100000:3.0:1789760716.163185:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 695659963823 time 695659703407 state 0x1 ref 1 00000100:00100000:3.0:1789760716.163322:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 695659963823 time 695727620108 state 0x0 ref 1 00000100:00100000:3.0:1789760716.163336:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5305500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512566528:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760716.163388:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22650880, mtime: 0x6aad94cb, atime: 0x0, ctime: 0x6aad94cb, blocks: 0xb007 00010000:00010000:3.0:1789760716.163408:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2d860 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.163425:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=695s v=(0 0) 00010000:00010000:3.0:1789760716.163429:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760716.163435:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2d860 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22650880->22654975] (req 22650880->22654975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.163449:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2d860 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22650880->22654975] (req 22650880->22654975) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760716.211437:0:3309:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8febffb9f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3309:1876699540219392:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1789760716.211506:0:3309:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760716.211596:0:6707:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540219392 00000100:00100000:0.0:1789760716.211604:0:6707:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540219392 00000100:00100000:0.0:1789760716.211649:0:6707:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 187 00000100:00100000:0.0:1789760716.211658:0:6707:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febffb9e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3309:x1876699540219392:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1789760716.213553:0:6707:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3859808256 avail 3857711104 00000100:00100000:0.0:1789760716.213640:0:6707:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febffb9e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3309:x1876699540219392:12345-0@lo:13:osp-pre-1-0.0 Request processed in 1980us (2130us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760716.213649:0:6707:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 187 00000100:00100000:0.0:1789760716.213728:0:3309:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8febffb9f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3309:1876699540219392:0@lo:13:osp-pre-1-0.0 00000001:00080000:2.0:1789760716.236754:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972849 is committed 00002000:00010000:1.0:1789760716.236852:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22650880 -> 22654976 00010000:00010000:1.0:1789760716.236867:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2d860 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22650880->22654975] (req 22650880->22654975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.236879:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2d860 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22650880->22654975] (req 22650880->22654975) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.236892:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2d860 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22650880->22654975] (req 22650880->22654975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.236898:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760716.236903:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2d860 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22650880->22654975] (req 22650880->22654975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.236911:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2d860 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22650880->22654975] (req 22650880->22654975) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.236943:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2d860 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22650880->22654975] (req 22650880->22654975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.236951:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2d860 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22650880->22654975] (req 22650880->22654975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760716.237056:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5305500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512566528:12345-192.168.203.56@tcp:4:dd.0 Request processed in 73719us (74139us total) trans 4294972849 rc 0/0 00000100:00100000:1.0:1789760716.237065:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5560 00000100:00100000:1.0:1789760716.237075:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 695659963823 time 695727620108 state 0x2 ref 0 00000100:00100000:3.0:1789760716.245435:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760716.254922:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512566656 00000100:00100000:1.0:1789760716.254937:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512566656 00000100:00100000:1.0:1789760716.255006:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 695659963823 time 695727620108 state 0x0 ref 1 00000100:00100000:1.0:1789760716.255030:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 695727880524 time 695727620108 state 0x1 ref 1 00000100:00100000:1.0:1789760716.255064:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 695727880524 time 695819363689 state 0x0 ref 1 00000100:00100000:1.0:1789760716.255076:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2745500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512566656:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760716.255122:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22654976, mtime: 0x6aad94cb, atime: 0x0, ctime: 0x6aad94cb, blocks: 0xb007 00010000:00010000:1.0:1789760716.255131:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2d867 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.255146:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=695s v=(0 0) 00010000:00010000:1.0:1789760716.255150:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760716.255156:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2d867 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22654976->22659071] (req 22654976->22659071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.255478:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2d867 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22654976->22659071] (req 22654976->22659071) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760716.322541:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972850 is committed 00002000:00010000:1.0:1789760716.322761:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22654976 -> 22659072 00010000:00010000:1.0:1789760716.322777:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2d867 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22654976->22659071] (req 22654976->22659071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.322790:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2d867 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22654976->22659071] (req 22654976->22659071) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.322800:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2d867 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22654976->22659071] (req 22654976->22659071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.322808:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760716.322813:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2d867 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22654976->22659071] (req 22654976->22659071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.322822:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2d867 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22654976->22659071] (req 22654976->22659071) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.322841:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2d867 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22654976->22659071] (req 22654976->22659071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.322849:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2d867 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22654976->22659071] (req 22654976->22659071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760716.322940:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2745500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512566656:12345-192.168.203.56@tcp:4:dd.0 Request processed in 67861us (77509us total) trans 4294972850 rc 0/0 00000100:00100000:1.0:1789760716.322950:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5561 00000100:00100000:1.0:1789760716.322959:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 695727880524 time 695819363689 state 0x2 ref 0 00000100:00100000:0.0:1789760716.325774:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760716.325832:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512566784 00000100:00100000:1.0:1789760716.325843:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512566784 00000100:00100000:1.0:1789760716.325893:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 695727880524 time 695819363689 state 0x0 ref 1 00000100:00100000:1.0:1789760716.325908:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 695819624105 time 695819363689 state 0x1 ref 1 00000100:00100000:1.0:1789760716.325940:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 695819624105 time 695890239701 state 0x0 ref 1 00000100:00100000:1.0:1789760716.325951:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febffb9f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512566784:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760716.325995:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22659072, mtime: 0x6aad94cb, atime: 0x0, ctime: 0x6aad94cb, blocks: 0xb007 00010000:00010000:1.0:1789760716.326004:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2d86e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.326019:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=695s v=(0 0) 00010000:00010000:1.0:1789760716.326024:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760716.326030:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2d86e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22659072->22663167] (req 22659072->22663167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.326044:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2d86e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22659072->22663167] (req 22659072->22663167) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760716.400249:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972851 is committed 00002000:00010000:1.0:1789760716.400318:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22659072 -> 22663168 00010000:00010000:1.0:1789760716.400333:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2d86e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22659072->22663167] (req 22659072->22663167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.400346:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2d86e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22659072->22663167] (req 22659072->22663167) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.400355:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2d86e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22659072->22663167] (req 22659072->22663167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.400362:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760716.400367:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2d86e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22659072->22663167] (req 22659072->22663167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.400375:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2d86e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22659072->22663167] (req 22659072->22663167) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.400393:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2d86e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22659072->22663167] (req 22659072->22663167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.400401:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2d86e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22659072->22663167] (req 22659072->22663167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760716.400477:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febffb9f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512566784:12345-192.168.203.56@tcp:4:dd.0 Request processed in 74524us (74709us total) trans 4294972851 rc 0/0 00000100:00100000:1.0:1789760716.400487:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5562 00000100:00100000:1.0:1789760716.400497:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 695819624105 time 695890239701 state 0x2 ref 0 00000100:00100000:3.0:1789760716.403799:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760716.403939:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512566912 00000100:00100000:1.0:1789760716.403950:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512566912 00000100:00100000:1.0:1789760716.404004:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 695819624105 time 695890239701 state 0x0 ref 1 00000100:00100000:1.0:1789760716.404018:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 695890500117 time 695890239701 state 0x1 ref 1 00000100:00100000:1.0:1789760716.404057:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 695890500117 time 695968356713 state 0x0 ref 1 00000100:00100000:1.0:1789760716.404069:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2747480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512566912:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760716.404115:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22663168, mtime: 0x6aad94cb, atime: 0x0, ctime: 0x6aad94cb, blocks: 0xb007 00010000:00010000:1.0:1789760716.404124:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2d875 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.404140:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=695s v=(0 0) 00010000:00010000:1.0:1789760716.404145:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760716.404151:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2d875 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22663168->22667263] (req 22663168->22667263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.404163:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2d875 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22663168->22667263] (req 22663168->22667263) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760716.503906:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972852 is committed 00002000:00010000:1.0:1789760716.504320:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22663168 -> 22667264 00010000:00010000:1.0:1789760716.504336:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2d875 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22663168->22667263] (req 22663168->22667263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.504349:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2d875 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22663168->22667263] (req 22663168->22667263) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.504357:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2d875 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22663168->22667263] (req 22663168->22667263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.504365:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760716.504370:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2d875 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22663168->22667263] (req 22663168->22667263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.504379:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2d875 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22663168->22667263] (req 22663168->22667263) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.504400:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2d875 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22663168->22667263] (req 22663168->22667263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.504413:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2d875 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22663168->22667263] (req 22663168->22667263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760716.504501:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2747480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512566912:12345-192.168.203.56@tcp:4:dd.0 Request processed in 100430us (100710us total) trans 4294972852 rc 0/0 00000100:00100000:1.0:1789760716.504511:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5563 00000100:00100000:1.0:1789760716.504532:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 695890500117 time 695968356713 state 0x2 ref 0 00000100:00100000:1.0:1789760716.510907:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760716.511058:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512567040 00000100:00100000:3.0:1789760716.511069:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512567040 00000100:00100000:3.0:1789760716.511137:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 695890500117 time 695968356713 state 0x0 ref 1 00000100:00100000:3.0:1789760716.511153:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 695968617129 time 695968356713 state 0x1 ref 1 00000100:00100000:3.0:1789760716.511178:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 695968617129 time 696075478177 state 0x0 ref 1 00000100:00100000:3.0:1789760716.511200:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac38b5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512567040:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760716.511250:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22667264, mtime: 0x6aad94cb, atime: 0x0, ctime: 0x6aad94cb, blocks: 0xb007 00010000:00010000:3.0:1789760716.511258:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2d87c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.511275:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=696s v=(0 0) 00010000:00010000:3.0:1789760716.511279:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760716.511286:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2d87c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22667264->22671359] (req 22667264->22671359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.511300:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2d87c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22667264->22671359] (req 22667264->22671359) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760716.553069:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22667264 -> 22671360 00002000:00010000:0.0:1789760716.553079:0:13063:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760715 -> 1789760716 00002000:00010000:0.0:1789760716.553082:0:13063:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760715 -> 1789760716 00010000:00010000:0.0:1789760716.553104:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2d87c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22667264->22671359] (req 22667264->22671359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.553116:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2d87c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22667264->22671359] (req 22667264->22671359) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.553126:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2d87c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22667264->22671359] (req 22667264->22671359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.553133:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760716.553138:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2d87c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22667264->22671359] (req 22667264->22671359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.553148:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2d87c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22667264->22671359] (req 22667264->22671359) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.553166:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2d87c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22667264->22671359] (req 22667264->22671359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.553174:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2d87c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22667264->22671359] (req 22667264->22671359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760716.553279:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac38b5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2361:x1876699512567040:12345-192.168.203.56@tcp:4:dd.0 Request processed in 42085us (42377us total) trans 4294972853 rc 0/0 00000100:00100000:0.0:1789760716.553288:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5564 00000100:00100000:0.0:1789760716.553298:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 695968617129 time 696075478177 state 0x2 ref 0 00000001:00080000:2.0:1789760716.553543:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972853 is committed 00000100:00100000:2.0:1789760716.557399:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760716.557899:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512567168 00000100:00100000:1.0:1789760716.557910:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512567168 00000100:00100000:1.0:1789760716.557975:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 695968617129 time 696075478177 state 0x0 ref 1 00000100:00100000:1.0:1789760716.557990:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 696075738593 time 696075478177 state 0x1 ref 1 00000100:00100000:1.0:1789760716.558022:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 696075738593 time 696122321612 state 0x0 ref 1 00000100:00100000:1.0:1789760716.558150:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5304a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512567168:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760716.558210:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22671360, mtime: 0x6aad94cc, atime: 0x0, ctime: 0x6aad94cc, blocks: 0xb007 00010000:00010000:1.0:1789760716.558218:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2d883 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.558233:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=696s v=(0 0) 00010000:00010000:1.0:1789760716.558237:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760716.558243:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2d883 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22671360->22675455] (req 22671360->22675455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.558257:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2d883 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22671360->22675455] (req 22671360->22675455) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760716.599382:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972854 is committed 00002000:00010000:1.0:1789760716.599543:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22671360 -> 22675456 00010000:00010000:1.0:1789760716.599558:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2d883 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22671360->22675455] (req 22671360->22675455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.599571:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2d883 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22671360->22675455] (req 22671360->22675455) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.599580:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2d883 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22671360->22675455] (req 22671360->22675455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.599588:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760716.599593:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2d883 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22671360->22675455] (req 22671360->22675455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.599800:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2d883 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22671360->22675455] (req 22671360->22675455) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.599821:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2d883 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22671360->22675455] (req 22671360->22675455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.599828:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2d883 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22671360->22675455] (req 22671360->22675455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760716.599910:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5304a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512567168:12345-192.168.203.56@tcp:4:dd.0 Request processed in 41759us (42516us total) trans 4294972854 rc 0/0 00000100:00100000:1.0:1789760716.599929:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5565 00000100:00100000:1.0:1789760716.599938:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 696075738593 time 696122321612 state 0x2 ref 0 00000100:00100000:3.0:1789760716.603630:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760716.603795:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512567296 00000100:00100000:1.0:1789760716.603805:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512567296 00000100:00100000:1.0:1789760716.603856:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 696075738593 time 696122321612 state 0x0 ref 1 00000100:00100000:1.0:1789760716.603870:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 696122582028 time 696122321612 state 0x1 ref 1 00000100:00100000:1.0:1789760716.603891:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 696122582028 time 696168190722 state 0x0 ref 1 00000100:00100000:1.0:1789760716.603902:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2744700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512567296:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760716.603960:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22675456, mtime: 0x6aad94cc, atime: 0x0, ctime: 0x6aad94cc, blocks: 0xb007 00010000:00010000:1.0:1789760716.603968:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2d88a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.603985:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=696s v=(0 0) 00010000:00010000:1.0:1789760716.603989:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760716.603996:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2d88a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22675456->22679551] (req 22675456->22679551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.604008:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2d88a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22675456->22679551] (req 22675456->22679551) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760716.652636:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972855 is committed 00002000:00010000:1.0:1789760716.652741:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22675456 -> 22679552 00010000:00010000:1.0:1789760716.652755:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2d88a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22675456->22679551] (req 22675456->22679551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.652766:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2d88a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22675456->22679551] (req 22675456->22679551) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.652774:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2d88a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22675456->22679551] (req 22675456->22679551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.652780:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760716.652786:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2d88a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22675456->22679551] (req 22675456->22679551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.652795:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2d88a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22675456->22679551] (req 22675456->22679551) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.652812:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2d88a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22675456->22679551] (req 22675456->22679551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.652819:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2d88a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22675456->22679551] (req 22675456->22679551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760716.652920:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2744700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512567296:12345-192.168.203.56@tcp:4:dd.0 Request processed in 49007us (49287us total) trans 4294972855 rc 0/0 00000100:00100000:1.0:1789760716.652932:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5566 00000100:00100000:1.0:1789760716.652942:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 696122582028 time 696168190722 state 0x2 ref 0 00000100:00100000:3.0:1789760716.655757:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760716.656056:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512567424 00000100:00100000:2.0:1789760716.656066:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512567424 00000100:00100000:2.0:1789760716.656129:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 696122582028 time 696168190722 state 0x0 ref 1 00000100:00100000:2.0:1789760716.656147:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 696168451138 time 696168190722 state 0x1 ref 1 00000100:00100000:2.0:1789760716.656170:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 696168451138 time 696220469553 state 0x0 ref 1 00000100:00100000:2.0:1789760716.656181:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2746a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512567424:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760716.656246:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22679552, mtime: 0x6aad94cc, atime: 0x0, ctime: 0x6aad94cc, blocks: 0xb007 00010000:00010000:2.0:1789760716.656255:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2d891 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.656271:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=696s v=(0 0) 00010000:00010000:2.0:1789760716.656275:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760716.656282:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2d891 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22679552->22683647] (req 22679552->22683647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.656296:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2d891 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22679552->22683647] (req 22679552->22683647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760716.710157:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972856 is committed 00002000:00010000:1.0:1789760716.710349:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22679552 -> 22683648 00010000:00010000:1.0:1789760716.710363:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2d891 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22679552->22683647] (req 22679552->22683647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.710376:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2d891 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22679552->22683647] (req 22679552->22683647) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.710386:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2d891 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22679552->22683647] (req 22679552->22683647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.710393:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760716.710398:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2d891 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22679552->22683647] (req 22679552->22683647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.710407:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2d891 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22679552->22683647] (req 22679552->22683647) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.710425:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2d891 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22679552->22683647] (req 22679552->22683647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.710433:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2d891 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22679552->22683647] (req 22679552->22683647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760716.710532:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2746a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512567424:12345-192.168.203.56@tcp:4:dd.0 Request processed in 54338us (54770us total) trans 4294972856 rc 0/0 00000100:00100000:1.0:1789760716.710543:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5567 00000100:00100000:1.0:1789760716.710552:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 696168451138 time 696220469553 state 0x2 ref 0 00000100:00100000:3.0:1789760716.713419:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760716.713561:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512567552 00000100:00100000:1.0:1789760716.713570:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512567552 00000100:00100000:1.0:1789760716.713822:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 696168451138 time 696220469553 state 0x0 ref 1 00000100:00100000:1.0:1789760716.713837:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 696220729969 time 696220469553 state 0x1 ref 1 00000100:00100000:1.0:1789760716.713901:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 696220729969 time 696278201010 state 0x0 ref 1 00000100:00100000:1.0:1789760716.713926:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2745880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512567552:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760716.713973:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22683648, mtime: 0x6aad94cc, atime: 0x0, ctime: 0x6aad94cc, blocks: 0xb007 00010000:00010000:1.0:1789760716.713980:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2d898 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.713996:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=696s v=(0 0) 00010000:00010000:1.0:1789760716.714000:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760716.714005:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2d898 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22683648->22687743] (req 22683648->22687743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.714017:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2d898 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22683648->22687743] (req 22683648->22687743) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760716.763393:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972857 is committed 00002000:00010000:1.0:1789760716.763492:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22683648 -> 22687744 00010000:00010000:1.0:1789760716.763508:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2d898 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22683648->22687743] (req 22683648->22687743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.763529:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2d898 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22683648->22687743] (req 22683648->22687743) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.763538:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2d898 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22683648->22687743] (req 22683648->22687743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.763546:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760716.763550:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2d898 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22683648->22687743] (req 22683648->22687743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.763560:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2d898 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22683648->22687743] (req 22683648->22687743) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.763579:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2d898 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22683648->22687743] (req 22683648->22687743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.763587:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2d898 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22683648->22687743] (req 22683648->22687743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760716.763732:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2745880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512567552:12345-192.168.203.56@tcp:4:dd.0 Request processed in 49780us (50281us total) trans 4294972857 rc 0/0 00000100:00100000:1.0:1789760716.763746:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5568 00000100:00100000:1.0:1789760716.763755:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 696220729969 time 696278201010 state 0x2 ref 0 00000100:00100000:1.0:1789760716.767145:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760716.767276:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512567680 00000100:00100000:3.0:1789760716.767285:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512567680 00000100:00100000:3.0:1789760716.767337:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 696220729969 time 696278201010 state 0x0 ref 1 00000100:00100000:3.0:1789760716.767352:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 696278461426 time 696278201010 state 0x1 ref 1 00000100:00100000:3.0:1789760716.767385:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 696278461426 time 696331685025 state 0x0 ref 1 00000100:00100000:3.0:1789760716.767397:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4362d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512567680:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760716.767442:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22687744, mtime: 0x6aad94cc, atime: 0x0, ctime: 0x6aad94cc, blocks: 0xb007 00010000:00010000:3.0:1789760716.767451:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2d89f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.767466:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=696s v=(0 0) 00010000:00010000:3.0:1789760716.767469:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760716.767476:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2d89f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22687744->22691839] (req 22687744->22691839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.767493:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2d89f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22687744->22691839] (req 22687744->22691839) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760716.800662:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972858 is committed 00002000:00010000:1.0:1789760716.800791:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22687744 -> 22691840 00010000:00010000:1.0:1789760716.800806:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2d89f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22687744->22691839] (req 22687744->22691839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.800818:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2d89f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22687744->22691839] (req 22687744->22691839) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.800826:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2d89f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22687744->22691839] (req 22687744->22691839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.800834:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760716.800838:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2d89f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22687744->22691839] (req 22687744->22691839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.800847:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2d89f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22687744->22691839] (req 22687744->22691839) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.800866:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2d89f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22687744->22691839] (req 22687744->22691839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.800873:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2d89f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22687744->22691839] (req 22687744->22691839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760716.800972:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4362d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512567680:12345-192.168.203.56@tcp:4:dd.0 Request processed in 33573us (33833us total) trans 4294972858 rc 0/0 00000100:00100000:1.0:1789760716.800981:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5569 00000100:00100000:1.0:1789760716.800990:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 696278461426 time 696331685025 state 0x2 ref 0 00000100:00100000:2.0:1789760716.805640:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760716.805771:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512567808 00000100:00100000:1.0:1789760716.805780:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512567808 00000100:00100000:1.0:1789760716.805832:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 696278461426 time 696331685025 state 0x0 ref 1 00000100:00100000:1.0:1789760716.805847:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 696331945441 time 696331685025 state 0x1 ref 1 00000100:00100000:1.0:1789760716.805881:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 696331945441 time 696370180601 state 0x0 ref 1 00000100:00100000:1.0:1789760716.805893:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5306d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512567808:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760716.805941:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22691840, mtime: 0x6aad94cc, atime: 0x0, ctime: 0x6aad94cc, blocks: 0xb007 00010000:00010000:1.0:1789760716.805951:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2d8a6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.805966:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=696s v=(0 0) 00010000:00010000:1.0:1789760716.805971:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760716.805977:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2d8a6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22691840->22695935] (req 22691840->22695935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.805990:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2d8a6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22691840->22695935] (req 22691840->22695935) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760716.900891:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972859 is committed 00002000:00010000:1.0:1789760716.901060:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22691840 -> 22695936 00010000:00010000:1.0:1789760716.901081:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2d8a6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22691840->22695935] (req 22691840->22695935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.901098:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2d8a6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22691840->22695935] (req 22691840->22695935) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.901109:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2d8a6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22691840->22695935] (req 22691840->22695935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.901118:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760716.901124:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2d8a6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22691840->22695935] (req 22691840->22695935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.901134:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2d8a6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22691840->22695935] (req 22691840->22695935) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.901160:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2d8a6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22691840->22695935] (req 22691840->22695935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.901168:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2d8a6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22691840->22695935] (req 22691840->22695935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760716.901396:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5306d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512567808:12345-192.168.203.56@tcp:4:dd.0 Request processed in 95498us (95758us total) trans 4294972859 rc 0/0 00000100:00100000:1.0:1789760716.901413:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5570 00000100:00100000:1.0:1789760716.901425:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 696331945441 time 696370180601 state 0x2 ref 0 00000100:00100000:0.0:1789760716.905868:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760716.905911:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512567936 00000100:00100000:1.0:1789760716.905924:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512567936 00000100:00100000:1.0:1789760716.905999:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 696331945441 time 696370180601 state 0x0 ref 1 00000100:00100000:1.0:1789760716.906019:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 696370441017 time 696370180601 state 0x1 ref 1 00000100:00100000:1.0:1789760716.906069:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 696370441017 time 696470358512 state 0x0 ref 1 00000100:00100000:1.0:1789760716.906084:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febffb9ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512567936:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760716.906154:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22695936, mtime: 0x6aad94cc, atime: 0x0, ctime: 0x6aad94cc, blocks: 0xb007 00010000:00010000:1.0:1789760716.906166:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2d8ad lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.906185:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=696s v=(0 0) 00010000:00010000:1.0:1789760716.906190:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760716.906197:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2d8ad lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22695936->22700031] (req 22695936->22700031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.906214:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2d8ad lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22695936->22700031] (req 22695936->22700031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760716.954842:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972860 is committed 00002000:00010000:1.0:1789760716.955022:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22695936 -> 22700032 00010000:00010000:1.0:1789760716.955120:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2d8ad lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22695936->22700031] (req 22695936->22700031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.955139:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2d8ad lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22695936->22700031] (req 22695936->22700031) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.955151:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2d8ad lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22695936->22700031] (req 22695936->22700031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.955160:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760716.955166:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2d8ad lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22695936->22700031] (req 22695936->22700031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.955176:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2d8ad lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22695936->22700031] (req 22695936->22700031) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.957179:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2d8ad lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22695936->22700031] (req 22695936->22700031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.957209:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2d8ad lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22695936->22700031] (req 22695936->22700031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760716.958620:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febffb9ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512567936:12345-192.168.203.56@tcp:4:dd.0 Request processed in 52523us (52751us total) trans 4294972860 rc 0/0 00000100:00100000:0.0:1789760716.958637:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5571 00000100:00100000:0.0:1789760716.958649:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 696370441017 time 696470358512 state 0x2 ref 0 00000100:00100000:3.0:1789760716.964585:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760716.964679:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512568064 00000100:00100000:1.0:1789760716.964737:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512568064 00000100:00100000:1.0:1789760716.965282:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 696370441017 time 696470358512 state 0x0 ref 1 00000100:00100000:1.0:1789760716.965301:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 696470618928 time 696470358512 state 0x1 ref 1 00000100:00100000:1.0:1789760716.965338:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 696470618928 time 696529636935 state 0x0 ref 1 00000100:00100000:1.0:1789760716.965355:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2746300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512568064:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760716.965419:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22700032, mtime: 0x6aad94cc, atime: 0x0, ctime: 0x6aad94cc, blocks: 0xb007 00010000:00010000:1.0:1789760716.965491:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2d8b4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.965512:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=696s v=(0 0) 00010000:00010000:1.0:1789760716.965525:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760716.965534:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2d8b4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22700032->22704127] (req 22700032->22704127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.965551:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2d8b4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22700032->22704127] (req 22700032->22704127) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760717.030533:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972861 is committed 00002000:00010000:2.0:1789760717.030731:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22700032 -> 22704128 00010000:00010000:2.0:1789760717.030749:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2d8b4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22700032->22704127] (req 22700032->22704127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.030761:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2d8b4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22700032->22704127] (req 22700032->22704127) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.030771:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2d8b4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22700032->22704127] (req 22700032->22704127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.030779:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760717.030785:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2d8b4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22700032->22704127] (req 22700032->22704127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.030794:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2d8b4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22700032->22704127] (req 22700032->22704127) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.030813:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2d8b4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22700032->22704127] (req 22700032->22704127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.030821:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2d8b4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22700032->22704127] (req 22700032->22704127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760717.031014:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2746300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512568064:12345-192.168.203.56@tcp:4:dd.0 Request processed in 65659us (66433us total) trans 4294972861 rc 0/0 00000100:00100000:2.0:1789760717.031026:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5572 00000100:00100000:2.0:1789760717.031036:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 696470618928 time 696529636935 state 0x2 ref 0 00000100:00100000:2.0:1789760717.035824:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760717.035889:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512568192 00000100:00100000:1.0:1789760717.035899:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512568192 00000100:00100000:1.0:1789760717.035993:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 696470618928 time 696529636935 state 0x0 ref 1 00000100:00100000:1.0:1789760717.036011:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 696529897351 time 696529636935 state 0x1 ref 1 00000100:00100000:1.0:1789760717.036111:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 696529897351 time 696600409572 state 0x0 ref 1 00000100:00080000:2.0:1789760717.036211:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1789760717.036890:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5306300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512568192:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760717.036951:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22704128, mtime: 0x6aad94cc, atime: 0x0, ctime: 0x6aad94cc, blocks: 0xb007 00010000:00010000:0.0:1789760717.036960:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2d8bb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.036977:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=696s v=(0 0) 00010000:00010000:0.0:1789760717.036982:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760717.036988:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2d8bb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22704128->22708223] (req 22704128->22708223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.037002:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2d8bb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22704128->22708223] (req 22704128->22708223) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760717.071557:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972862 is committed 00002000:00010000:0.0:1789760717.071779:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22704128 -> 22708224 00010000:00010000:0.0:1789760717.071794:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2d8bb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22704128->22708223] (req 22704128->22708223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.071808:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2d8bb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22704128->22708223] (req 22704128->22708223) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.071819:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2d8bb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22704128->22708223] (req 22704128->22708223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.071826:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760717.071832:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2d8bb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22704128->22708223] (req 22704128->22708223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.071841:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2d8bb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22704128->22708223] (req 22704128->22708223) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.071860:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2d8bb lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22704128->22708223] (req 22704128->22708223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.071873:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2d8bb lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22704128->22708223] (req 22704128->22708223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760717.071972:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5306300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512568192:12345-192.168.203.56@tcp:4:dd.0 Request processed in 35846us (36151us total) trans 4294972862 rc 0/0 00000100:00100000:0.0:1789760717.071983:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5573 00000100:00100000:0.0:1789760717.071993:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 696529897351 time 696600409572 state 0x2 ref 0 00000100:00100000:1.0:1789760717.080807:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760717.080966:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512568320 00000100:00100000:0.0:1789760717.080977:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512568320 00000100:00100000:0.0:1789760717.081030:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 696529897351 time 696600409572 state 0x0 ref 1 00000100:00100000:0.0:1789760717.081046:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 696600669988 time 696600409572 state 0x1 ref 1 00000100:00100000:0.0:1789760717.081123:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 696600669988 time 696645422625 state 0x0 ref 1 00000100:00100000:0.0:1789760717.081136:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac28a1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512568320:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760717.081192:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22708224, mtime: 0x6aad94cc, atime: 0x0, ctime: 0x6aad94cc, blocks: 0xb007 00010000:00010000:0.0:1789760717.081201:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2d8c2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.081217:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=696s v=(0 0) 00010000:00010000:0.0:1789760717.081222:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760717.081228:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2d8c2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22708224->22712319] (req 22708224->22712319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.081249:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2d8c2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22708224->22712319] (req 22708224->22712319) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760717.124169:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22708224 -> 22712320 00010000:00010000:0.0:1789760717.124190:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2d8c2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22708224->22712319] (req 22708224->22712319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.124203:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2d8c2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22708224->22712319] (req 22708224->22712319) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.124213:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2d8c2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22708224->22712319] (req 22708224->22712319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.124221:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760717.124226:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2d8c2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22708224->22712319] (req 22708224->22712319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.124235:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2d8c2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22708224->22712319] (req 22708224->22712319) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.124255:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2d8c2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22708224->22712319] (req 22708224->22712319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.124263:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2d8c2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22708224->22712319] (req 22708224->22712319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760717.124389:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac28a1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2360:x1876699512568320:12345-192.168.203.56@tcp:4:dd.0 Request processed in 43249us (43586us total) trans 4294972863 rc 0/0 00000100:00100000:0.0:1789760717.124400:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5574 00000100:00100000:0.0:1789760717.124410:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 696600669988 time 696645422625 state 0x2 ref 0 00000001:00080000:3.0:1789760717.125005:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972863 is committed 00000100:00100000:3.0:1789760717.127688:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760717.127865:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512568448 00000100:00100000:2.0:1789760717.127877:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512568448 00000100:00100000:2.0:1789760717.127937:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 696600669988 time 696645422625 state 0x0 ref 1 00000100:00100000:2.0:1789760717.127954:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 696645683041 time 696645422625 state 0x1 ref 1 00000100:00100000:2.0:1789760717.127989:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 696645683041 time 696692288067 state 0x0 ref 1 00000100:00100000:2.0:1789760717.128001:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2745180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512568448:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760717.128053:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22712320, mtime: 0x6aad94cc, atime: 0x0, ctime: 0x6aad94cc, blocks: 0xb007 00010000:00010000:2.0:1789760717.128063:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2d8c9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.128082:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=696s v=(0 0) 00010000:00010000:2.0:1789760717.128087:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760717.128094:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2d8c9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22712320->22716415] (req 22712320->22716415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.128111:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2d8c9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22712320->22716415] (req 22712320->22716415) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760717.157499:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972864 is committed 00002000:00010000:1.0:1789760717.157650:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22712320 -> 22716416 00010000:00010000:1.0:1789760717.157663:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2d8c9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22712320->22716415] (req 22712320->22716415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760717.157677:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2d8c9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22712320->22716415] (req 22712320->22716415) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760717.157683:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2d8c9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22712320->22716415] (req 22712320->22716415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760717.157688:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760717.157691:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2d8c9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22712320->22716415] (req 22712320->22716415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760717.157755:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2d8c9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22712320->22716415] (req 22712320->22716415) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760717.157771:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2d8c9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22712320->22716415] (req 22712320->22716415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760717.157776:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2d8c9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22712320->22716415] (req 22712320->22716415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760717.157958:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2745180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512568448:12345-192.168.203.56@tcp:4:dd.0 Request processed in 29952us (30273us total) trans 4294972864 rc 0/0 00000100:00100000:0.0:1789760717.157968:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5575 00000100:00100000:0.0:1789760717.157978:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 696645683041 time 696692288067 state 0x2 ref 0 00000100:00100000:1.0:1789760717.162151:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760717.162618:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512568576 00000100:00100000:0.0:1789760717.162627:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512568576 00000100:00100000:0.0:1789760717.162685:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 696645683041 time 696692288067 state 0x0 ref 1 00000100:00100000:0.0:1789760717.162727:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 696692548483 time 696692288067 state 0x1 ref 1 00000100:00100000:0.0:1789760717.162757:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 696692548483 time 696727056334 state 0x0 ref 1 00000100:00100000:0.0:1789760717.162770:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac28a2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512568576:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760717.162817:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22716416, mtime: 0x6aad94cc, atime: 0x0, ctime: 0x6aad94cc, blocks: 0xb007 00010000:00010000:0.0:1789760717.162831:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2d8d0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.162847:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=696s v=(0 0) 00010000:00010000:0.0:1789760717.162852:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760717.162857:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2d8d0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22716416->22720511] (req 22716416->22720511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.162871:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2d8d0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22716416->22720511] (req 22716416->22720511) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.170886:0:5836:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:1.0:1789760717.205141:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972865 is committed 00002000:00010000:0.0:1789760717.205585:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22716416 -> 22720512 00010000:00010000:0.0:1789760717.205601:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2d8d0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22716416->22720511] (req 22716416->22720511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.205615:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2d8d0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22716416->22720511] (req 22716416->22720511) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.205624:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2d8d0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22716416->22720511] (req 22716416->22720511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.205631:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760717.205637:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2d8d0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22716416->22720511] (req 22716416->22720511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.205646:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2d8d0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22716416->22720511] (req 22716416->22720511) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.205666:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2d8d0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22716416->22720511] (req 22716416->22720511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.205680:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2d8d0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22716416->22720511] (req 22716416->22720511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760717.205793:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac28a2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512568576:12345-192.168.203.56@tcp:4:dd.0 Request processed in 43021us (43648us total) trans 4294972865 rc 0/0 00000100:00100000:0.0:1789760717.205803:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5576 00000100:00100000:0.0:1789760717.205814:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 696692548483 time 696727056334 state 0x2 ref 0 00000100:00100000:1.0:1789760717.215625:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760717.215788:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512568704 00000100:00100000:0.0:1789760717.215800:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512568704 00000100:00100000:0.0:1789760717.215861:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 696692548483 time 696727056334 state 0x0 ref 1 00000100:00100000:0.0:1789760717.215880:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 696727316750 time 696727056334 state 0x1 ref 1 00000100:00100000:0.0:1789760717.215925:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 696727316750 time 696780223976 state 0x0 ref 1 00000100:00100000:0.0:1789760717.215938:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac28a3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512568704:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760717.215998:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22720512, mtime: 0x6aad94cc, atime: 0x0, ctime: 0x6aad94cc, blocks: 0xb007 00010000:00010000:0.0:1789760717.216007:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2d8d7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.216024:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=696s v=(0 0) 00010000:00010000:0.0:1789760717.216029:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760717.216035:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2d8d7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22720512->22724607] (req 22720512->22724607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.216056:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2d8d7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22720512->22724607] (req 22720512->22724607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760717.279939:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972866 is committed 00002000:00010000:0.0:1789760717.280140:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22720512 -> 22724608 00010000:00010000:0.0:1789760717.280155:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2d8d7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22720512->22724607] (req 22720512->22724607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.280167:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2d8d7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22720512->22724607] (req 22720512->22724607) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.280177:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2d8d7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22720512->22724607] (req 22720512->22724607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.280184:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760717.280189:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2d8d7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22720512->22724607] (req 22720512->22724607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.280198:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2d8d7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22720512->22724607] (req 22720512->22724607) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.280218:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2d8d7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22720512->22724607] (req 22720512->22724607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.280225:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2d8d7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22720512->22724607] (req 22720512->22724607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760717.280323:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac28a3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512568704:12345-192.168.203.56@tcp:4:dd.0 Request processed in 64382us (64703us total) trans 4294972866 rc 0/0 00000100:00100000:0.0:1789760717.280334:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5577 00000100:00100000:0.0:1789760717.280344:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 696727316750 time 696780223976 state 0x2 ref 0 00000100:00100000:3.0:1789760717.287301:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760717.287449:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512568832 00000100:00100000:0.0:1789760717.287459:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512568832 00000100:00100000:0.0:1789760717.287529:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 696727316750 time 696780223976 state 0x0 ref 1 00000100:00100000:0.0:1789760717.287546:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 696780484392 time 696780223976 state 0x1 ref 1 00000100:00100000:0.0:1789760717.287580:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 696780484392 time 696851878946 state 0x0 ref 1 00000100:00100000:0.0:1789760717.287592:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2744a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512568832:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760717.287665:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22724608, mtime: 0x6aad94cc, atime: 0x0, ctime: 0x6aad94cc, blocks: 0xb007 00010000:00010000:0.0:1789760717.287675:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2d8de lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.287690:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=696s v=(0 0) 00010000:00010000:0.0:1789760717.287716:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760717.287723:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2d8de lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22724608->22728703] (req 22724608->22728703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.287739:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2d8de lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22724608->22728703] (req 22724608->22728703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760717.352209:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972867 is committed 00002000:00010000:0.0:1789760717.352364:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22724608 -> 22728704 00010000:00010000:0.0:1789760717.352382:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2d8de lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22724608->22728703] (req 22724608->22728703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.352395:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2d8de lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22724608->22728703] (req 22724608->22728703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.352405:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2d8de lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22724608->22728703] (req 22724608->22728703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.352412:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760717.352417:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2d8de lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22724608->22728703] (req 22724608->22728703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.352427:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2d8de lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22724608->22728703] (req 22724608->22728703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.352446:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2d8de lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22724608->22728703] (req 22724608->22728703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.352454:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2d8de lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22724608->22728703] (req 22724608->22728703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760717.352738:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2744a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512568832:12345-192.168.203.56@tcp:4:dd.0 Request processed in 65106us (65405us total) trans 4294972867 rc 0/0 00000100:00100000:0.0:1789760717.352752:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5578 00000100:00100000:0.0:1789760717.352762:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 696780484392 time 696851878946 state 0x2 ref 0 00000100:00100000:1.0:1789760717.361346:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760717.361447:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512568960 00000100:00100000:0.0:1789760717.361457:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512568960 00000100:00100000:0.0:1789760717.361530:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 696780484392 time 696851878946 state 0x0 ref 1 00000100:00100000:0.0:1789760717.361544:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 696852139362 time 696851878946 state 0x1 ref 1 00000100:00100000:0.0:1789760717.361572:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 696852139362 time 696925871982 state 0x0 ref 1 00000100:00100000:0.0:1789760717.361661:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac28a0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512568960:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760717.361747:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22728704, mtime: 0x6aad94cc, atime: 0x0, ctime: 0x6aad94cc, blocks: 0xb007 00010000:00010000:0.0:1789760717.361757:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2d8e5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.361773:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=696s v=(0 0) 00010000:00010000:0.0:1789760717.361778:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760717.361784:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2d8e5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22728704->22732799] (req 22728704->22732799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.361796:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2d8e5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22728704->22732799] (req 22728704->22732799) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760717.406325:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972868 is committed 00002000:00010000:2.0:1789760717.406505:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22728704 -> 22732800 00010000:00010000:2.0:1789760717.406530:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2d8e5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22728704->22732799] (req 22728704->22732799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.406543:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2d8e5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22728704->22732799] (req 22728704->22732799) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.406552:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2d8e5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22728704->22732799] (req 22728704->22732799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.406559:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760717.406564:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2d8e5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22728704->22732799] (req 22728704->22732799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.406573:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2d8e5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22728704->22732799] (req 22728704->22732799) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.406603:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2d8e5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22728704->22732799] (req 22728704->22732799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.406612:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2d8e5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22728704->22732799] (req 22728704->22732799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760717.406748:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac28a0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512568960:12345-192.168.203.56@tcp:4:dd.0 Request processed in 45086us (45407us total) trans 4294972868 rc 0/0 00000100:00100000:2.0:1789760717.406758:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5579 00000100:00100000:2.0:1789760717.406768:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 696852139362 time 696925871982 state 0x2 ref 0 00000100:00100000:1.0:1789760717.412101:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760717.412195:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512569088 00000100:00100000:0.0:1789760717.412207:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512569088 00000100:00100000:0.0:1789760717.412261:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 696852139362 time 696925871982 state 0x0 ref 1 00000100:00100000:0.0:1789760717.412278:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 696926132398 time 696925871982 state 0x1 ref 1 00000100:00100000:0.0:1789760717.412308:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 696926132398 time 696976607762 state 0x0 ref 1 00000100:00100000:0.0:1789760717.412321:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febc7b33480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512569088:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760717.412368:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22732800, mtime: 0x6aad94cc, atime: 0x0, ctime: 0x6aad94cc, blocks: 0xb007 00010000:00010000:0.0:1789760717.412377:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2d8ec lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.412392:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=696s v=(0 0) 00010000:00010000:0.0:1789760717.412396:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760717.412403:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2d8ec lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22732800->22736895] (req 22732800->22736895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.412416:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2d8ec lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22732800->22736895] (req 22732800->22736895) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760717.472356:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972869 is committed 00002000:00010000:2.0:1789760717.472473:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22732800 -> 22736896 00010000:00010000:2.0:1789760717.472488:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2d8ec lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22732800->22736895] (req 22732800->22736895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.472499:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2d8ec lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22732800->22736895] (req 22732800->22736895) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.472508:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2d8ec lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22732800->22736895] (req 22732800->22736895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.472527:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760717.472532:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2d8ec lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22732800->22736895] (req 22732800->22736895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.472542:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2d8ec lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22732800->22736895] (req 22732800->22736895) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.472564:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2d8ec lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22732800->22736895] (req 22732800->22736895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.472572:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2d8ec lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22732800->22736895] (req 22732800->22736895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760717.472667:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febc7b33480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512569088:12345-192.168.203.56@tcp:4:dd.0 Request processed in 60343us (60570us total) trans 4294972869 rc 0/0 00000100:00100000:2.0:1789760717.472678:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5580 00000100:00100000:2.0:1789760717.472687:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 696926132398 time 696976607762 state 0x2 ref 0 00000100:00100000:1.0:1789760717.476371:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760717.476895:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512569216 00000100:00100000:0.0:1789760717.476905:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512569216 00000100:00100000:0.0:1789760717.476960:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 696926132398 time 696976607762 state 0x0 ref 1 00000100:00100000:0.0:1789760717.476974:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 696976868178 time 696976607762 state 0x1 ref 1 00000100:00100000:0.0:1789760717.477006:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 696976868178 time 697041305869 state 0x0 ref 1 00000100:00100000:0.0:1789760717.477023:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febc7b32680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512569216:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760717.477073:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22736896, mtime: 0x6aad94cc, atime: 0x0, ctime: 0x6aad94cc, blocks: 0xb007 00010000:00010000:0.0:1789760717.477082:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2d8f3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.477098:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=697s v=(0 0) 00010000:00010000:0.0:1789760717.477103:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760717.477109:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2d8f3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22736896->22740991] (req 22736896->22740991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.477125:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2d8f3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22736896->22740991] (req 22736896->22740991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760717.576390:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972870 is committed 00002000:00010000:0.0:1789760717.576645:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22736896 -> 22740992 00010000:00010000:0.0:1789760717.576661:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2d8f3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22736896->22740991] (req 22736896->22740991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.576674:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2d8f3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22736896->22740991] (req 22736896->22740991) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.576683:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2d8f3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22736896->22740991] (req 22736896->22740991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.576691:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760717.576713:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2d8f3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22736896->22740991] (req 22736896->22740991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.576725:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2d8f3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22736896->22740991] (req 22736896->22740991) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.576743:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2d8f3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22736896->22740991] (req 22736896->22740991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.576752:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2d8f3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22736896->22740991] (req 22736896->22740991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760717.576839:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febc7b32680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512569216:12345-192.168.203.56@tcp:4:dd.0 Request processed in 99814us (100474us total) trans 4294972870 rc 0/0 00000100:00100000:0.0:1789760717.576850:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5581 00000100:00100000:0.0:1789760717.576860:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 696976868178 time 697041305869 state 0x2 ref 0 00000100:00100000:2.0:1789760717.585671:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760717.585802:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512569344 00000100:00100000:0.0:1789760717.585813:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512569344 00000100:00100000:0.0:1789760717.585864:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 696976868178 time 697041305869 state 0x0 ref 1 00000100:00100000:0.0:1789760717.585879:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 697041566285 time 697041305869 state 0x1 ref 1 00000100:00100000:0.0:1789760717.585914:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 697041566285 time 697150213988 state 0x0 ref 1 00000100:00100000:0.0:1789760717.585927:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5307800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512569344:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760717.585974:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22740992, mtime: 0x6aad94cc, atime: 0x0, ctime: 0x6aad94cc, blocks: 0xb007 00010000:00010000:0.0:1789760717.585984:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2d8fa lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.586001:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=697s v=(0 0) 00010000:00010000:0.0:1789760717.586006:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760717.586012:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2d8fa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22740992->22745087] (req 22740992->22745087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.586025:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2d8fa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22740992->22745087] (req 22740992->22745087) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760717.650907:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22740992 -> 22745088 00002000:00010000:0.0:1789760717.650916:0:13063:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760716 -> 1789760717 00002000:00010000:0.0:1789760717.650918:0:13063:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760716 -> 1789760717 00010000:00010000:0.0:1789760717.650931:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2d8fa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22740992->22745087] (req 22740992->22745087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.650943:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2d8fa lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22740992->22745087] (req 22740992->22745087) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.650952:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2d8fa lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22740992->22745087] (req 22740992->22745087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.650960:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760717.650965:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2d8fa lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22740992->22745087] (req 22740992->22745087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.650974:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2d8fa lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22740992->22745087] (req 22740992->22745087) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.650994:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2d8fa lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22740992->22745087] (req 22740992->22745087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.651001:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2d8fa lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22740992->22745087] (req 22740992->22745087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760717.651085:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5307800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2359:x1876699512569344:12345-192.168.203.56@tcp:4:dd.0 Request processed in 65155us (65418us total) trans 4294972871 rc 0/0 00000100:00100000:0.0:1789760717.651095:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5582 00000100:00100000:0.0:1789760717.651104:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 697041566285 time 697150213988 state 0x2 ref 0 00000001:00080000:2.0:1789760717.651871:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972871 is committed 00000100:00100000:1.0:1789760717.659365:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760717.659607:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512569472 00000100:00100000:2.0:1789760717.659617:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512569472 00000100:00100000:2.0:1789760717.659669:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 697041566285 time 697150213988 state 0x0 ref 1 00000100:00100000:2.0:1789760717.659684:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 697150474404 time 697150213988 state 0x1 ref 1 00000100:00100000:2.0:1789760717.659748:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 697150474404 time 697224047564 state 0x0 ref 1 00000100:00100000:2.0:1789760717.659761:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febfb894700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512569472:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760717.659807:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22745088, mtime: 0x6aad94cd, atime: 0x0, ctime: 0x6aad94cd, blocks: 0xb007 00010000:00010000:2.0:1789760717.659816:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2d901 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.659836:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=697s v=(0 0) 00010000:00010000:2.0:1789760717.659841:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760717.659847:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2d901 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22745088->22749183] (req 22745088->22749183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.659860:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2d901 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22745088->22749183] (req 22745088->22749183) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760717.727333:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972872 is committed 00002000:00010000:2.0:1789760717.727423:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22745088 -> 22749184 00010000:00010000:2.0:1789760717.727438:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2d901 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22745088->22749183] (req 22745088->22749183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.727450:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2d901 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22745088->22749183] (req 22745088->22749183) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.727458:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2d901 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22745088->22749183] (req 22745088->22749183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.727465:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760717.727470:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2d901 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22745088->22749183] (req 22745088->22749183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.727479:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2d901 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22745088->22749183] (req 22745088->22749183) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.727497:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2d901 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22745088->22749183] (req 22745088->22749183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.727504:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2d901 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22745088->22749183] (req 22745088->22749183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760717.727602:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febfb894700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512569472:12345-192.168.203.56@tcp:4:dd.0 Request processed in 67837us (68240us total) trans 4294972872 rc 0/0 00000100:00100000:2.0:1789760717.727612:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5583 00000100:00100000:2.0:1789760717.727621:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 697150474404 time 697224047564 state 0x2 ref 0 00000100:00100000:0.0:1789760717.737417:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760717.737536:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512569600 00000100:00100000:2.0:1789760717.737545:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512569600 00000100:00100000:2.0:1789760717.737600:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 697150474404 time 697224047564 state 0x0 ref 1 00000100:00100000:2.0:1789760717.737616:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 697224307980 time 697224047564 state 0x1 ref 1 00000100:00100000:2.0:1789760717.737650:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 697224307980 time 697301949028 state 0x0 ref 1 00000100:00100000:2.0:1789760717.737663:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febffb9c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512569600:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760717.737743:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22749184, mtime: 0x6aad94cd, atime: 0x0, ctime: 0x6aad94cd, blocks: 0xb007 00010000:00010000:2.0:1789760717.737753:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2d908 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.737768:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=697s v=(0 0) 00010000:00010000:2.0:1789760717.737773:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760717.737779:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2d908 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22749184->22753279] (req 22749184->22753279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.737793:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2d908 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22749184->22753279] (req 22749184->22753279) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760717.812567:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972873 is committed 00002000:00010000:0.0:1789760717.814683:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22749184 -> 22753280 00010000:00010000:0.0:1789760717.814720:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2d908 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22749184->22753279] (req 22749184->22753279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.814734:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2d908 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22749184->22753279] (req 22749184->22753279) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.814744:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2d908 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22749184->22753279] (req 22749184->22753279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.814751:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760717.814756:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2d908 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22749184->22753279] (req 22749184->22753279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.814772:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2d908 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22749184->22753279] (req 22749184->22753279) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.814792:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2d908 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22749184->22753279] (req 22749184->22753279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.814800:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2d908 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22749184->22753279] (req 22749184->22753279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760717.814968:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febffb9c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512569600:12345-192.168.203.56@tcp:4:dd.0 Request processed in 77303us (77555us total) trans 4294972873 rc 0/0 00000100:00100000:0.0:1789760717.814979:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5584 00000100:00100000:0.0:1789760717.814988:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 697224307980 time 697301949028 state 0x2 ref 0 00000100:00100000:2.0:1789760717.821268:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760717.821359:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512569728 00000100:00100000:0.0:1789760717.821380:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512569728 00000100:00100000:0.0:1789760717.821448:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 697224307980 time 697301949028 state 0x0 ref 1 00000100:00100000:0.0:1789760717.821463:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 697302209444 time 697301949028 state 0x1 ref 1 00000100:00100000:0.0:1789760717.821493:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 697302209444 time 697385792596 state 0x0 ref 1 00000100:00100000:0.0:1789760717.821507:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5304000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512569728:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760717.821566:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22753280, mtime: 0x6aad94cd, atime: 0x0, ctime: 0x6aad94cd, blocks: 0xb007 00010000:00010000:0.0:1789760717.821575:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2d90f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.821591:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=697s v=(0 0) 00010000:00010000:0.0:1789760717.821595:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760717.821601:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2d90f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22753280->22757375] (req 22753280->22757375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.821616:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2d90f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22753280->22757375] (req 22753280->22757375) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760717.861309:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972874 is committed 00002000:00010000:0.0:1789760717.861538:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22753280 -> 22757376 00010000:00010000:0.0:1789760717.861554:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2d90f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22753280->22757375] (req 22753280->22757375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.861568:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2d90f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22753280->22757375] (req 22753280->22757375) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.861578:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2d90f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22753280->22757375] (req 22753280->22757375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.861585:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760717.861590:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2d90f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22753280->22757375] (req 22753280->22757375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.861599:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2d90f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22753280->22757375] (req 22753280->22757375) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.861619:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2d90f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22753280->22757375] (req 22753280->22757375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.861628:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2d90f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22753280->22757375] (req 22753280->22757375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760717.861755:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5304000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512569728:12345-192.168.203.56@tcp:4:dd.0 Request processed in 40246us (40492us total) trans 4294972874 rc 0/0 00000100:00100000:0.0:1789760717.861764:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5585 00000100:00100000:0.0:1789760717.861773:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 697302209444 time 697385792596 state 0x2 ref 0 00000100:00100000:0.0:1789760717.865374:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760717.865628:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512569856 00000100:00100000:2.0:1789760717.865639:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512569856 00000100:00100000:2.0:1789760717.865888:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 697302209444 time 697385792596 state 0x0 ref 1 00000100:00100000:2.0:1789760717.865919:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 697386053012 time 697385792596 state 0x1 ref 1 00000100:00100000:2.0:1789760717.865978:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 697386053012 time 697430277502 state 0x0 ref 1 00000100:00100000:2.0:1789760717.865993:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febffb9c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512569856:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760717.866046:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22757376, mtime: 0x6aad94cd, atime: 0x0, ctime: 0x6aad94cd, blocks: 0xb007 00010000:00010000:2.0:1789760717.866054:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2d916 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.866068:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=697s v=(0 0) 00010000:00010000:2.0:1789760717.866073:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760717.866079:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2d916 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22757376->22761471] (req 22757376->22761471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.866092:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2d916 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22757376->22761471] (req 22757376->22761471) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760717.912942:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972875 is committed 00002000:00010000:2.0:1789760717.913156:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22757376 -> 22761472 00010000:00010000:2.0:1789760717.913174:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2d916 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22757376->22761471] (req 22757376->22761471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.913201:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2d916 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22757376->22761471] (req 22757376->22761471) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.913210:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2d916 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22757376->22761471] (req 22757376->22761471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.913218:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760717.913224:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2d916 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22757376->22761471] (req 22757376->22761471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.913233:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2d916 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22757376->22761471] (req 22757376->22761471) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.913253:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2d916 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22757376->22761471] (req 22757376->22761471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.913260:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2d916 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22757376->22761471] (req 22757376->22761471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760717.913358:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febffb9c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512569856:12345-192.168.203.56@tcp:4:dd.0 Request processed in 47363us (47989us total) trans 4294972875 rc 0/0 00000100:00100000:2.0:1789760717.913371:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5586 00000100:00100000:2.0:1789760717.913382:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 697386053012 time 697430277502 state 0x2 ref 0 00000100:00100000:0.0:1789760717.917595:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760717.917855:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512569984 00000100:00100000:2.0:1789760717.917866:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512569984 00000100:00100000:2.0:1789760717.917919:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 697386053012 time 697430277502 state 0x0 ref 1 00000100:00100000:2.0:1789760717.917933:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 697430537918 time 697430277502 state 0x1 ref 1 00000100:00100000:2.0:1789760717.918007:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 697430537918 time 697482305806 state 0x0 ref 1 00000100:00100000:2.0:1789760717.918021:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febffb9d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512569984:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760717.918066:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22761472, mtime: 0x6aad94cd, atime: 0x0, ctime: 0x6aad94cd, blocks: 0xb007 00010000:00010000:2.0:1789760717.918075:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2d91d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.918090:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=697s v=(0 0) 00010000:00010000:2.0:1789760717.918095:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760717.918101:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2d91d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22761472->22765567] (req 22761472->22765567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.918113:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2d91d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22761472->22765567] (req 22761472->22765567) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760717.984923:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972876 is committed 00002000:00010000:1.0:1789760717.986103:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22761472 -> 22765568 00010000:00010000:1.0:1789760717.986120:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2d91d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22761472->22765567] (req 22761472->22765567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760717.986129:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2d91d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22761472->22765567] (req 22761472->22765567) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760717.986135:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2d91d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22761472->22765567] (req 22761472->22765567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760717.986139:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760717.986143:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2d91d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22761472->22765567] (req 22761472->22765567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760717.986150:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2d91d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22761472->22765567] (req 22761472->22765567) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760717.986163:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2d91d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22761472->22765567] (req 22761472->22765567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760717.986167:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2d91d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22761472->22765567] (req 22761472->22765567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760717.986255:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febffb9d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512569984:12345-192.168.203.56@tcp:4:dd.0 Request processed in 68233us (68664us total) trans 4294972876 rc 0/0 00000100:00100000:1.0:1789760717.986266:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5587 00000100:00100000:1.0:1789760717.986276:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 697430537918 time 697482305806 state 0x2 ref 0 00000100:00100000:1.0:1789760717.988444:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760717.988543:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512570112 00000100:00100000:0.0:1789760717.988554:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512570112 00000100:00100000:0.0:1789760717.988625:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 697430537918 time 697482305806 state 0x0 ref 1 00000100:00100000:0.0:1789760717.988946:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 697482566222 time 697482305806 state 0x1 ref 1 00000100:00100000:1.0:1789760717.989154:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 697482566222 time 697553451485 state 0x0 ref 1 00000100:00100000:1.0:1789760717.989170:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febfb897100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512570112:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760717.989218:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22765568, mtime: 0x6aad94cd, atime: 0x0, ctime: 0x6aad94cd, blocks: 0xb007 00010000:00010000:1.0:1789760717.989235:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2d924 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760717.989252:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=697s v=(0 0) 00010000:00010000:1.0:1789760717.989256:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760717.989262:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2d924 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22765568->22769663] (req 22765568->22769663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760717.989276:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2d924 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22765568->22769663] (req 22765568->22769663) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760718.048128:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972877 is committed 00002000:00010000:1.0:1789760718.048572:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22765568 -> 22769664 00010000:00010000:1.0:1789760718.048587:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2d924 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22765568->22769663] (req 22765568->22769663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760718.048600:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2d924 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22765568->22769663] (req 22765568->22769663) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760718.048616:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2d924 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22765568->22769663] (req 22765568->22769663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760718.048623:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760718.048628:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2d924 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22765568->22769663] (req 22765568->22769663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760718.048810:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2d924 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22765568->22769663] (req 22765568->22769663) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760718.048832:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2d924 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22765568->22769663] (req 22765568->22769663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760718.048839:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2d924 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22765568->22769663] (req 22765568->22769663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760718.049063:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febfb897100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512570112:12345-192.168.203.56@tcp:4:dd.0 Request processed in 59891us (60621us total) trans 4294972877 rc 0/0 00000100:00100000:1.0:1789760718.049074:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5588 00000100:00100000:1.0:1789760718.049083:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 697482566222 time 697553451485 state 0x2 ref 0 00000100:00100000:0.0:1789760718.055878:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760718.056104:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512570240 00000100:00100000:2.0:1789760718.056115:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512570240 00000100:00100000:2.0:1789760718.056171:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 697482566222 time 697553451485 state 0x0 ref 1 00000100:00100000:2.0:1789760718.056186:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 697553711901 time 697553451485 state 0x1 ref 1 00000100:00100000:2.0:1789760718.056219:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 697553711901 time 697620518461 state 0x0 ref 1 00000100:00100000:2.0:1789760718.056231:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febffb9f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512570240:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760718.056279:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22769664, mtime: 0x6aad94cd, atime: 0x0, ctime: 0x6aad94cd, blocks: 0xb007 00010000:00010000:2.0:1789760718.056289:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2d92b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.056305:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=697s v=(0 0) 00010000:00010000:2.0:1789760718.056310:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760718.056316:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2d92b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22769664->22773759] (req 22769664->22773759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.056331:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2d92b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22769664->22773759] (req 22769664->22773759) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760718.120414:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972878 is committed 00002000:00010000:2.0:1789760718.120842:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22769664 -> 22773760 00010000:00010000:2.0:1789760718.120858:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2d92b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22769664->22773759] (req 22769664->22773759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.120871:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2d92b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22769664->22773759] (req 22769664->22773759) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.120880:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2d92b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22769664->22773759] (req 22769664->22773759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.120888:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760718.120893:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2d92b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22769664->22773759] (req 22769664->22773759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.121003:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2d92b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22769664->22773759] (req 22769664->22773759) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.121136:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2d92b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22769664->22773759] (req 22769664->22773759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.121144:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2d92b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22769664->22773759] (req 22769664->22773759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760718.121335:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febffb9f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512570240:12345-192.168.203.56@tcp:4:dd.0 Request processed in 65102us (65461us total) trans 4294972878 rc 0/0 00000100:00100000:2.0:1789760718.121346:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5589 00000100:00100000:2.0:1789760718.121355:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 697553711901 time 697620518461 state 0x2 ref 0 00000100:00100000:0.0:1789760718.129354:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760718.132088:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512570368 00000100:00100000:1.0:1789760718.132102:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512570368 00000100:00100000:0.0:1789760718.132253:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 697553711901 time 697620518461 state 0x0 ref 1 00000100:00100000:0.0:1789760718.132270:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 697620778877 time 697620518461 state 0x1 ref 1 00000100:00100000:0.0:1789760718.132356:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 697620778877 time 697696654908 state 0x0 ref 1 00000100:00100000:0.0:1789760718.132370:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febffb9f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512570368:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760718.132422:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22773760, mtime: 0x6aad94cd, atime: 0x0, ctime: 0x6aad94cd, blocks: 0xb007 00010000:00010000:0.0:1789760718.132431:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2d932 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.132447:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=697s v=(0 0) 00010000:00010000:0.0:1789760718.132451:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760718.132458:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2d932 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22773760->22777855] (req 22773760->22777855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.132472:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2d932 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22773760->22777855] (req 22773760->22777855) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760718.179146:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972879 is committed 00002000:00010000:2.0:1789760718.179363:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22773760 -> 22777856 00010000:00010000:2.0:1789760718.179378:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2d932 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22773760->22777855] (req 22773760->22777855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.179391:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2d932 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22773760->22777855] (req 22773760->22777855) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.179400:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2d932 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22773760->22777855] (req 22773760->22777855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.179407:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760718.179411:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2d932 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22773760->22777855] (req 22773760->22777855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.179420:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2d932 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22773760->22777855] (req 22773760->22777855) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.179440:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2d932 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22773760->22777855] (req 22773760->22777855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.179452:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2d932 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22773760->22777855] (req 22773760->22777855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760718.179560:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febffb9f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512570368:12345-192.168.203.56@tcp:4:dd.0 Request processed in 47187us (50209us total) trans 4294972879 rc 0/0 00000100:00100000:2.0:1789760718.179571:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5590 00000100:00100000:2.0:1789760718.179582:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 697620778877 time 697696654908 state 0x2 ref 0 00000100:00100000:1.0:1789760718.183198:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760718.183333:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512570496 00000100:00100000:0.0:1789760718.183344:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512570496 00000100:00100000:0.0:1789760718.183403:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 697620778877 time 697696654908 state 0x0 ref 1 00000100:00100000:0.0:1789760718.183418:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 697696915324 time 697696654908 state 0x1 ref 1 00000100:00100000:0.0:1789760718.183449:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 697696915324 time 697747749454 state 0x0 ref 1 00000100:00100000:0.0:1789760718.183462:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febfb895880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512570496:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760718.183530:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22777856, mtime: 0x6aad94cd, atime: 0x0, ctime: 0x6aad94cd, blocks: 0xb007 00010000:00010000:0.0:1789760718.183539:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2d939 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.183561:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=697s v=(0 0) 00010000:00010000:0.0:1789760718.183566:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760718.183572:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2d939 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22777856->22781951] (req 22777856->22781951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.183588:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2d939 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22777856->22781951] (req 22777856->22781951) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.196785:0:5835:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:1.0:1789760718.228449:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972880 is committed 00002000:00010000:0.0:1789760718.228528:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22777856 -> 22781952 00010000:00010000:0.0:1789760718.228544:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2d939 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22777856->22781951] (req 22777856->22781951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.228557:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2d939 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22777856->22781951] (req 22777856->22781951) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.228566:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2d939 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22777856->22781951] (req 22777856->22781951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.228573:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760718.228578:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2d939 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22777856->22781951] (req 22777856->22781951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.228588:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2d939 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22777856->22781951] (req 22777856->22781951) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.228612:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2d939 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22777856->22781951] (req 22777856->22781951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.228674:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2d939 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22777856->22781951] (req 22777856->22781951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760718.228886:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febfb895880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512570496:12345-192.168.203.56@tcp:4:dd.0 Request processed in 45421us (45705us total) trans 4294972880 rc 0/0 00000100:00100000:0.0:1789760718.228897:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5591 00000100:00100000:0.0:1789760718.228916:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 697696915324 time 697747749454 state 0x2 ref 0 00000100:00100000:0.0:1789760718.232085:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760718.232183:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512570624 00000100:00100000:2.0:1789760718.232193:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512570624 00000100:00100000:2.0:1789760718.232248:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 697696915324 time 697747749454 state 0x0 ref 1 00000100:00100000:2.0:1789760718.232263:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 697748009870 time 697747749454 state 0x1 ref 1 00000100:00100000:2.0:1789760718.232301:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 697748009870 time 697796594524 state 0x0 ref 1 00000100:00100000:2.0:1789760718.232315:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac3352300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512570624:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760718.232362:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22781952, mtime: 0x6aad94cd, atime: 0x0, ctime: 0x6aad94cd, blocks: 0xb007 00010000:00010000:2.0:1789760718.232370:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2d940 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.232385:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=697s v=(0 0) 00010000:00010000:2.0:1789760718.232390:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760718.232395:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2d940 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22781952->22786047] (req 22781952->22786047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.232413:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2d940 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22781952->22786047] (req 22781952->22786047) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760718.291004:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972881 is committed 00002000:00010000:2.0:1789760718.291028:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22781952 -> 22786048 00010000:00010000:2.0:1789760718.291058:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2d940 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22781952->22786047] (req 22781952->22786047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.291072:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2d940 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22781952->22786047] (req 22781952->22786047) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.291086:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2d940 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22781952->22786047] (req 22781952->22786047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.291094:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760718.291099:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2d940 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22781952->22786047] (req 22781952->22786047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.291109:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2d940 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22781952->22786047] (req 22781952->22786047) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.291129:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2d940 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22781952->22786047] (req 22781952->22786047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.291136:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2d940 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22781952->22786047] (req 22781952->22786047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760718.291232:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac3352300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512570624:12345-192.168.203.56@tcp:4:dd.0 Request processed in 58915us (59151us total) trans 4294972881 rc 0/0 00000100:00100000:2.0:1789760718.291242:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5592 00000100:00100000:2.0:1789760718.291251:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 697748009870 time 697796594524 state 0x2 ref 0 00000100:00100000:2.0:1789760718.296936:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760718.297096:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512570752 00000100:00100000:0.0:1789760718.297107:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512570752 00000100:00100000:0.0:1789760718.297160:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 697748009870 time 697796594524 state 0x0 ref 1 00000100:00100000:0.0:1789760718.297175:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 697796854940 time 697796594524 state 0x1 ref 1 00000100:00100000:0.0:1789760718.297210:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 697796854940 time 697861509214 state 0x0 ref 1 00000100:00100000:0.0:1789760718.297223:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5304380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512570752:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760718.297273:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22786048, mtime: 0x6aad94cd, atime: 0x0, ctime: 0x6aad94cd, blocks: 0xb007 00010000:00010000:0.0:1789760718.297282:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2d947 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.297302:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=697s v=(0 0) 00010000:00010000:0.0:1789760718.297307:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760718.297314:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2d947 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22786048->22790143] (req 22786048->22790143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.297839:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2d947 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22786048->22790143] (req 22786048->22790143) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760718.363638:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972882 is committed 00002000:00010000:2.0:1789760718.363779:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22786048 -> 22790144 00010000:00010000:2.0:1789760718.363796:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2d947 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22786048->22790143] (req 22786048->22790143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.363810:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2d947 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22786048->22790143] (req 22786048->22790143) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.363820:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2d947 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22786048->22790143] (req 22786048->22790143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.363827:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760718.363832:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2d947 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22786048->22790143] (req 22786048->22790143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.363842:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2d947 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22786048->22790143] (req 22786048->22790143) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.363862:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2d947 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22786048->22790143] (req 22786048->22790143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.363876:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2d947 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22786048->22790143] (req 22786048->22790143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760718.363975:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5304380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512570752:12345-192.168.203.56@tcp:4:dd.0 Request processed in 66750us (67043us total) trans 4294972882 rc 0/0 00000100:00100000:2.0:1789760718.363986:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5593 00000100:00100000:2.0:1789760718.363994:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 697796854940 time 697861509214 state 0x2 ref 0 00000100:00100000:3.0:1789760718.369096:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760718.369168:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512570880 00000100:00100000:0.0:1789760718.369183:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512570880 00000100:00100000:0.0:1789760718.369245:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 697796854940 time 697861509214 state 0x0 ref 1 00000100:00100000:0.0:1789760718.369261:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 697861769630 time 697861509214 state 0x1 ref 1 00000100:00100000:0.0:1789760718.369295:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 697861769630 time 697933594462 state 0x0 ref 1 00000100:00100000:0.0:1789760718.369308:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2746d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512570880:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760718.369364:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22790144, mtime: 0x6aad94cd, atime: 0x0, ctime: 0x6aad94cd, blocks: 0xb007 00010000:00010000:0.0:1789760718.369372:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2d94e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.369387:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=697s v=(0 0) 00010000:00010000:0.0:1789760718.369391:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760718.369397:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2d94e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22790144->22794239] (req 22790144->22794239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.369416:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2d94e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22790144->22794239] (req 22790144->22794239) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760718.412633:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22790144 -> 22794240 00010000:00010000:0.0:1789760718.412652:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2d94e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22790144->22794239] (req 22790144->22794239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.412665:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2d94e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22790144->22794239] (req 22790144->22794239) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.412674:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2d94e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22790144->22794239] (req 22790144->22794239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.412683:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760718.412688:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2d94e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22790144->22794239] (req 22790144->22794239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.412740:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2d94e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22790144->22794239] (req 22790144->22794239) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.412760:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2d94e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22790144->22794239] (req 22790144->22794239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.412769:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2d94e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22790144->22794239] (req 22790144->22794239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760718.412894:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2746d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2358:x1876699512570880:12345-192.168.203.56@tcp:4:dd.0 Request processed in 43582us (43801us total) trans 4294972883 rc 0/0 00000100:00100000:0.0:1789760718.412905:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5594 00000100:00100000:0.0:1789760718.412915:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 697861769630 time 697933594462 state 0x2 ref 0 00000001:00080000:3.0:1789760718.413686:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972883 is committed 00000100:00100000:3.0:1789760718.415351:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760718.415573:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512571008 00000100:00100000:2.0:1789760718.415583:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512571008 00000100:00100000:2.0:1789760718.415633:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 697861769630 time 697933594462 state 0x0 ref 1 00000100:00100000:2.0:1789760718.415647:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 697933854878 time 697933594462 state 0x1 ref 1 00000100:00100000:2.0:1789760718.415729:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 697933854878 time 697980027929 state 0x0 ref 1 00000100:00100000:2.0:1789760718.415742:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2744380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512571008:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760718.415788:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22794240, mtime: 0x6aad94cd, atime: 0x0, ctime: 0x6aad94cd, blocks: 0xb007 00010000:00010000:2.0:1789760718.415798:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2d955 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.415813:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=697s v=(0 0) 00010000:00010000:2.0:1789760718.415818:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760718.415824:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2d955 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22794240->22798335] (req 22794240->22798335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.415843:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2d955 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22794240->22798335] (req 22794240->22798335) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760718.446196:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972884 is committed 00002000:00010000:3.0:1789760718.446719:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22794240 -> 22798336 00010000:00010000:3.0:1789760718.446734:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2d955 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22794240->22798335] (req 22794240->22798335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.446745:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2d955 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22794240->22798335] (req 22794240->22798335) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.446753:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2d955 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22794240->22798335] (req 22794240->22798335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.446759:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760718.446765:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2d955 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22794240->22798335] (req 22794240->22798335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.446773:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2d955 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22794240->22798335] (req 22794240->22798335) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.446792:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2d955 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22794240->22798335] (req 22794240->22798335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.446799:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2d955 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22794240->22798335] (req 22794240->22798335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760718.446893:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2744380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512571008:12345-192.168.203.56@tcp:4:dd.0 Request processed in 31148us (31547us total) trans 4294972884 rc 0/0 00000100:00100000:3.0:1789760718.446903:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5595 00000100:00100000:3.0:1789760718.446913:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 697933854878 time 697980027929 state 0x2 ref 0 00000100:00100000:2.0:1789760718.453214:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760718.453461:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512571136 00000100:00100000:0.0:1789760718.453472:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512571136 00000100:00100000:0.0:1789760718.453539:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 697933854878 time 697980027929 state 0x0 ref 1 00000100:00100000:0.0:1789760718.453554:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 697980288345 time 697980027929 state 0x1 ref 1 00000100:00100000:0.0:1789760718.453592:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 697980288345 time 698017891999 state 0x0 ref 1 00000100:00100000:0.0:1789760718.453606:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5306680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512571136:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760718.453659:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22798336, mtime: 0x6aad94cd, atime: 0x0, ctime: 0x6aad94cd, blocks: 0xb007 00010000:00010000:0.0:1789760718.453667:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2d95c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.453682:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=698s v=(0 0) 00010000:00010000:0.0:1789760718.453686:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760718.453693:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2d95c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22798336->22802431] (req 22798336->22802431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.453764:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2d95c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22798336->22802431] (req 22798336->22802431) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760718.519881:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22798336 -> 22802432 00010000:00010000:1.0:1789760718.519902:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2d95c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22798336->22802431] (req 22798336->22802431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760718.519915:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2d95c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22798336->22802431] (req 22798336->22802431) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760718.519924:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2d95c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22798336->22802431] (req 22798336->22802431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760718.519932:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760718.519937:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2d95c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22798336->22802431] (req 22798336->22802431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760718.519946:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2d95c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22798336->22802431] (req 22798336->22802431) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760718.519966:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2d95c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22798336->22802431] (req 22798336->22802431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760718.519972:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972885 is committed 00010000:00010000:1.0:1789760718.519973:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2d95c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22798336->22802431] (req 22798336->22802431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760718.520599:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5306680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512571136:12345-192.168.203.56@tcp:4:dd.0 Request processed in 66988us (67386us total) trans 4294972885 rc 0/0 00000100:00100000:1.0:1789760718.520666:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5596 00000100:00100000:1.0:1789760718.520677:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 697980288345 time 698017891999 state 0x2 ref 0 00000100:00100000:2.0:1789760718.526008:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760718.526249:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512571264 00000100:00100000:0.0:1789760718.526259:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512571264 00000100:00100000:0.0:1789760718.526442:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 697980288345 time 698017891999 state 0x0 ref 1 00000100:00100000:0.0:1789760718.526457:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 698018152415 time 698017891999 state 0x1 ref 1 00000100:00100000:0.0:1789760718.526545:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 698018152415 time 698090844089 state 0x0 ref 1 00000100:00100000:0.0:1789760718.526558:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5307b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512571264:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760718.526608:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22802432, mtime: 0x6aad94cd, atime: 0x0, ctime: 0x6aad94cd, blocks: 0xb007 00010000:00010000:0.0:1789760718.526617:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2d963 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.526632:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=698s v=(0 0) 00010000:00010000:0.0:1789760718.526635:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760718.526641:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2d963 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22802432->22806527] (req 22802432->22806527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.526654:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2d963 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22802432->22806527] (req 22802432->22806527) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760718.565432:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22802432 -> 22806528 00002000:00010000:3.0:1789760718.565443:0:13063:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760717 -> 1789760718 00002000:00010000:3.0:1789760718.565446:0:13063:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760717 -> 1789760718 00010000:00010000:3.0:1789760718.565459:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2d963 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22802432->22806527] (req 22802432->22806527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.565472:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2d963 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22802432->22806527] (req 22802432->22806527) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.565487:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2d963 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22802432->22806527] (req 22802432->22806527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.565495:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760718.565500:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2d963 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22802432->22806527] (req 22802432->22806527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.565520:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2d963 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22802432->22806527] (req 22802432->22806527) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.565538:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2d963 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22802432->22806527] (req 22802432->22806527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.565546:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2d963 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22802432->22806527] (req 22802432->22806527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760718.565651:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5307b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2359:x1876699512571264:12345-192.168.203.56@tcp:4:dd.0 Request processed in 39089us (39644us total) trans 4294972886 rc 0/0 00000100:00100000:3.0:1789760718.565663:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5597 00000100:00100000:3.0:1789760718.565673:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 698018152415 time 698090844089 state 0x2 ref 0 00000001:00080000:3.0:1789760718.565756:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972886 is committed 00000100:00100000:3.0:1789760718.569492:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760718.569590:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512571392 00000100:00100000:2.0:1789760718.569601:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512571392 00000100:00100000:2.0:1789760718.569664:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 698018152415 time 698090844089 state 0x0 ref 1 00000100:00100000:2.0:1789760718.569680:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 698091104505 time 698090844089 state 0x1 ref 1 00000100:00100000:0.0:1789760718.569907:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 698091104505 time 698134193463 state 0x0 ref 1 00000100:00100000:0.0:1789760718.569926:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2746680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512571392:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760718.569983:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22806528, mtime: 0x6aad94ce, atime: 0x0, ctime: 0x6aad94ce, blocks: 0xb007 00010000:00010000:0.0:1789760718.569993:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2d96a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.570007:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=698s v=(0 0) 00010000:00010000:0.0:1789760718.570012:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760718.570018:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2d96a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22806528->22810623] (req 22806528->22810623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.570032:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2d96a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22806528->22810623] (req 22806528->22810623) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760718.612542:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972887 is committed 00002000:00010000:3.0:1789760718.612602:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22806528 -> 22810624 00010000:00010000:3.0:1789760718.612625:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2d96a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22806528->22810623] (req 22806528->22810623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.612637:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2d96a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22806528->22810623] (req 22806528->22810623) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.612646:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2d96a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22806528->22810623] (req 22806528->22810623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.612653:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760718.612658:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2d96a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22806528->22810623] (req 22806528->22810623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.612667:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2d96a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22806528->22810623] (req 22806528->22810623) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.612685:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2d96a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22806528->22810623] (req 22806528->22810623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.612693:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2d96a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22806528->22810623] (req 22806528->22810623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760718.612842:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2746680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512571392:12345-192.168.203.56@tcp:4:dd.0 Request processed in 42917us (43353us total) trans 4294972887 rc 0/0 00000100:00100000:3.0:1789760718.612852:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5598 00000100:00100000:3.0:1789760718.612861:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 698091104505 time 698134193463 state 0x2 ref 0 00000100:00100000:2.0:1789760718.618089:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760718.618188:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512571520 00000100:00100000:1.0:1789760718.618197:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512571520 00000100:00100000:1.0:1789760718.618244:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 698091104505 time 698134193463 state 0x0 ref 1 00000100:00100000:1.0:1789760718.618258:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 698134453879 time 698134193463 state 0x1 ref 1 00000100:00100000:1.0:1789760718.618284:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 698134453879 time 698182584525 state 0x0 ref 1 00000100:00100000:1.0:1789760718.618299:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5307100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512571520:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760718.618345:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22810624, mtime: 0x6aad94ce, atime: 0x0, ctime: 0x6aad94ce, blocks: 0xb007 00010000:00010000:1.0:1789760718.618353:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2d971 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760718.618367:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=698s v=(0 0) 00010000:00010000:1.0:1789760718.618372:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760718.618378:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2d971 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22810624->22814719] (req 22810624->22814719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760718.618392:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2d971 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22810624->22814719] (req 22810624->22814719) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760718.673158:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972888 is committed 00002000:00010000:3.0:1789760718.673669:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22810624 -> 22814720 00010000:00010000:3.0:1789760718.673685:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2d971 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22810624->22814719] (req 22810624->22814719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.673740:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2d971 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22810624->22814719] (req 22810624->22814719) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.673749:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2d971 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22810624->22814719] (req 22810624->22814719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.673756:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760718.673761:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2d971 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22810624->22814719] (req 22810624->22814719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.673771:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2d971 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22810624->22814719] (req 22810624->22814719) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.673798:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2d971 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22810624->22814719] (req 22810624->22814719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.673807:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2d971 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22810624->22814719] (req 22810624->22814719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760718.673895:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5307100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512571520:12345-192.168.203.56@tcp:4:dd.0 Request processed in 55591us (55810us total) trans 4294972888 rc 0/0 00000100:00100000:3.0:1789760718.673905:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5599 00000100:00100000:3.0:1789760718.673915:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 698134453879 time 698182584525 state 0x2 ref 0 00000100:00100000:2.0:1789760718.679343:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760718.679444:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512571648 00000100:00100000:3.0:1789760718.679455:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512571648 00000100:00100000:3.0:1789760718.679524:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 698134453879 time 698182584525 state 0x0 ref 1 00000100:00100000:3.0:1789760718.679540:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 698182844941 time 698182584525 state 0x1 ref 1 00000100:00100000:3.0:1789760718.679609:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 698182844941 time 698243907775 state 0x0 ref 1 00000100:00100000:3.0:1789760718.679622:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5305f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512571648:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760718.679672:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22814720, mtime: 0x6aad94ce, atime: 0x0, ctime: 0x6aad94ce, blocks: 0xb007 00010000:00010000:3.0:1789760718.679692:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2d978 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.679760:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=698s v=(0 0) 00010000:00010000:3.0:1789760718.679765:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760718.679772:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2d978 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22814720->22818815] (req 22814720->22818815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.679786:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2d978 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22814720->22818815] (req 22814720->22818815) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760718.749199:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972889 is committed 00002000:00010000:0.0:1789760718.749251:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22814720 -> 22818816 00010000:00010000:0.0:1789760718.749267:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2d978 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22814720->22818815] (req 22814720->22818815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.749280:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2d978 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22814720->22818815] (req 22814720->22818815) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.749290:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2d978 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22814720->22818815] (req 22814720->22818815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.749298:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760718.749303:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2d978 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22814720->22818815] (req 22814720->22818815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.749316:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2d978 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22814720->22818815] (req 22814720->22818815) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.749335:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2d978 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22814720->22818815] (req 22814720->22818815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.749342:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2d978 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22814720->22818815] (req 22814720->22818815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760718.749434:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5305f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512571648:12345-192.168.203.56@tcp:4:dd.0 Request processed in 69810us (70094us total) trans 4294972889 rc 0/0 00000100:00100000:0.0:1789760718.749443:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5600 00000100:00100000:0.0:1789760718.749451:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 698182844941 time 698243907775 state 0x2 ref 0 00000100:00100000:0.0:1789760718.757975:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760718.758055:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512571776 00000100:00100000:3.0:1789760718.758064:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512571776 00000100:00100000:3.0:1789760718.758119:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 698182844941 time 698243907775 state 0x0 ref 1 00000100:00100000:3.0:1789760718.758136:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 698244168191 time 698243907775 state 0x1 ref 1 00000100:00100000:3.0:1789760718.758173:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 698244168191 time 698322472568 state 0x0 ref 1 00000100:00100000:3.0:1789760718.758186:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac3353800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512571776:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760718.758234:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22818816, mtime: 0x6aad94ce, atime: 0x0, ctime: 0x6aad94ce, blocks: 0xb007 00010000:00010000:3.0:1789760718.758244:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2d97f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.758259:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=698s v=(0 0) 00010000:00010000:3.0:1789760718.758264:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760718.758271:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2d97f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22818816->22822911] (req 22818816->22822911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.758286:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2d97f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22818816->22822911] (req 22818816->22822911) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760718.789969:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972890 is committed 00002000:00010000:0.0:1789760718.790358:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22818816 -> 22822912 00010000:00010000:0.0:1789760718.790375:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2d97f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22818816->22822911] (req 22818816->22822911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.790389:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2d97f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22818816->22822911] (req 22818816->22822911) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.790399:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2d97f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22818816->22822911] (req 22818816->22822911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.790412:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760718.790418:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2d97f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22818816->22822911] (req 22818816->22822911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.790426:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2d97f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22818816->22822911] (req 22818816->22822911) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.790446:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2d97f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22818816->22822911] (req 22818816->22822911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.790581:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2d97f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22818816->22822911] (req 22818816->22822911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760718.790742:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac3353800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512571776:12345-192.168.203.56@tcp:4:dd.0 Request processed in 32554us (32771us total) trans 4294972890 rc 0/0 00000100:00100000:0.0:1789760718.790753:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5601 00000100:00100000:0.0:1789760718.790768:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 698244168191 time 698322472568 state 0x2 ref 0 00000100:00100000:3.0:1789760718.796262:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760718.797482:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512571904 00000100:00100000:1.0:1789760718.797494:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512571904 00000100:00100000:1.0:1789760718.797561:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 698244168191 time 698322472568 state 0x0 ref 1 00000100:00100000:1.0:1789760718.797578:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 698322732984 time 698322472568 state 0x1 ref 1 00000100:00100000:1.0:1789760718.797613:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 698322732984 time 698361912724 state 0x0 ref 1 00000100:00100000:1.0:1789760718.797626:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2747b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512571904:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760718.797674:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22822912, mtime: 0x6aad94ce, atime: 0x0, ctime: 0x6aad94ce, blocks: 0xb007 00010000:00010000:1.0:1789760718.797684:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2d986 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760718.797736:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=698s v=(0 0) 00010000:00010000:1.0:1789760718.797742:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760718.797748:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2d986 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22822912->22827007] (req 22822912->22827007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760718.797763:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2d986 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22822912->22827007] (req 22822912->22827007) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760718.850438:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22822912 -> 22827008 00010000:00010000:2.0:1789760718.850463:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2d986 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22822912->22827007] (req 22822912->22827007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.850476:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2d986 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22822912->22827007] (req 22822912->22827007) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.850486:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2d986 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22822912->22827007] (req 22822912->22827007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.850494:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760718.850499:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2d986 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22822912->22827007] (req 22822912->22827007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.850523:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2d986 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22822912->22827007] (req 22822912->22827007) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.850543:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2d986 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22822912->22827007] (req 22822912->22827007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.850551:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2d986 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22822912->22827007] (req 22822912->22827007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760718.850663:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2747b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2361:x1876699512571904:12345-192.168.203.56@tcp:4:dd.0 Request processed in 53035us (54405us total) trans 4294972891 rc 0/0 00000100:00100000:2.0:1789760718.850674:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5602 00000100:00100000:2.0:1789760718.850683:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 698322732984 time 698361912724 state 0x2 ref 0 00000001:00080000:0.0:1789760718.851458:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972891 is committed 00000100:00100000:2.0:1789760718.862959:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760718.863033:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512572032 00000100:00100000:0.0:1789760718.863044:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512572032 00000100:00100000:0.0:1789760718.863111:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 698322732984 time 698361912724 state 0x0 ref 1 00000100:00100000:0.0:1789760718.863127:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 698362173140 time 698361912724 state 0x1 ref 1 00000100:00100000:0.0:1789760718.863159:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 698362173140 time 698427458858 state 0x0 ref 1 00000100:00100000:0.0:1789760718.863173:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5307480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512572032:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760718.863231:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22827008, mtime: 0x6aad94ce, atime: 0x0, ctime: 0x6aad94ce, blocks: 0xb007 00010000:00010000:0.0:1789760718.863240:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2d98d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.863256:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=698s v=(0 0) 00010000:00010000:0.0:1789760718.863260:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760718.863267:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2d98d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22827008->22831103] (req 22827008->22831103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.863281:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2d98d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22827008->22831103] (req 22827008->22831103) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760718.941825:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972892 is committed 00002000:00010000:0.0:1789760718.941966:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22827008 -> 22831104 00010000:00010000:0.0:1789760718.941982:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2d98d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22827008->22831103] (req 22827008->22831103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.941994:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2d98d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22827008->22831103] (req 22827008->22831103) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.942003:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2d98d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22827008->22831103] (req 22827008->22831103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.942011:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760718.942016:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2d98d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22827008->22831103] (req 22827008->22831103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.942026:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2d98d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22827008->22831103] (req 22827008->22831103) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.942044:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2d98d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22827008->22831103] (req 22827008->22831103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.942053:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2d98d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22827008->22831103] (req 22827008->22831103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760718.942150:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5307480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512572032:12345-192.168.203.56@tcp:4:dd.0 Request processed in 78975us (79195us total) trans 4294972892 rc 0/0 00000100:00100000:0.0:1789760718.943756:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5603 00000100:00100000:0.0:1789760718.943768:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 698362173140 time 698427458858 state 0x2 ref 0 00000100:00100000:3.0:1789760718.946482:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760718.946573:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512572160 00000100:00100000:0.0:1789760718.946583:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512572160 00000100:00100000:0.0:1789760718.946646:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 698362173140 time 698427458858 state 0x0 ref 1 00000100:00100000:0.0:1789760718.946660:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 698427719274 time 698427458858 state 0x1 ref 1 00000100:00100000:0.0:1789760718.946693:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 698427719274 time 698510992310 state 0x0 ref 1 00000100:00100000:0.0:1789760718.946801:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2747800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512572160:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760718.946852:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22831104, mtime: 0x6aad94ce, atime: 0x0, ctime: 0x6aad94ce, blocks: 0xb007 00010000:00010000:0.0:1789760718.946861:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2d994 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.946876:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=698s v=(0 0) 00010000:00010000:0.0:1789760718.946881:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760718.946887:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2d994 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22831104->22835199] (req 22831104->22835199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.946901:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2d994 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22831104->22835199] (req 22831104->22835199) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760719.001610:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972893 is committed 00002000:00010000:3.0:1789760719.003730:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22831104 -> 22835200 00010000:00010000:3.0:1789760719.003751:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2d994 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22831104->22835199] (req 22831104->22835199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.003764:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2d994 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22831104->22835199] (req 22831104->22835199) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.003773:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2d994 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22831104->22835199] (req 22831104->22835199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.003786:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760719.003791:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2d994 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22831104->22835199] (req 22831104->22835199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.003801:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2d994 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22831104->22835199] (req 22831104->22835199) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.003821:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2d994 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22831104->22835199] (req 22831104->22835199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.003952:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2d994 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22831104->22835199] (req 22831104->22835199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760719.004071:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2747800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512572160:12345-192.168.203.56@tcp:4:dd.0 Request processed in 57329us (57591us total) trans 4294972893 rc 0/0 00000100:00100000:3.0:1789760719.004084:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5604 00000100:00100000:3.0:1789760719.004094:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 698427719274 time 698510992310 state 0x2 ref 0 00000100:00100000:3.0:1789760719.007322:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760719.007437:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512572288 00000100:00100000:2.0:1789760719.007449:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512572288 00000100:00100000:2.0:1789760719.007546:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 698427719274 time 698510992310 state 0x0 ref 1 00000100:00100000:2.0:1789760719.007563:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 698511252726 time 698510992310 state 0x1 ref 1 00000100:00100000:1.0:1789760719.007732:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 698511252726 time 698572029417 state 0x0 ref 1 00000100:00100000:1.0:1789760719.007747:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4217100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512572288:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760719.007793:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22835200, mtime: 0x6aad94ce, atime: 0x0, ctime: 0x6aad94ce, blocks: 0xb007 00010000:00010000:1.0:1789760719.007803:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2d99b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.007839:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=698s v=(0 0) 00010000:00010000:1.0:1789760719.007845:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760719.007852:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2d99b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22835200->22839295] (req 22835200->22839295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.007868:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2d99b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22835200->22839295] (req 22835200->22839295) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1789760719.008547:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:0.0:1789760719.063722:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972894 is committed 00002000:00010000:3.0:1789760719.063820:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22835200 -> 22839296 00010000:00010000:3.0:1789760719.063844:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2d99b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22835200->22839295] (req 22835200->22839295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.063856:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2d99b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22835200->22839295] (req 22835200->22839295) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.063870:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2d99b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22835200->22839295] (req 22835200->22839295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.063877:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760719.063882:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2d99b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22835200->22839295] (req 22835200->22839295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.063890:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2d99b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22835200->22839295] (req 22835200->22839295) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.063907:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2d99b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22835200->22839295] (req 22835200->22839295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.063914:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2d99b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22835200->22839295] (req 22835200->22839295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760719.064006:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4217100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512572288:12345-192.168.203.56@tcp:4:dd.0 Request processed in 56258us (56693us total) trans 4294972894 rc 0/0 00000100:00100000:3.0:1789760719.064015:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5605 00000100:00100000:3.0:1789760719.064024:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 698511252726 time 698572029417 state 0x2 ref 0 00000100:00100000:2.0:1789760719.068657:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760719.069269:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512572416 00000100:00100000:0.0:1789760719.069280:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512572416 00000100:00100000:0.0:1789760719.069455:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 698511252726 time 698572029417 state 0x0 ref 1 00000100:00100000:0.0:1789760719.069475:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 698572289833 time 698572029417 state 0x1 ref 1 00000100:00100000:0.0:1789760719.069548:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 698572289833 time 698633847285 state 0x0 ref 1 00000100:00100000:0.0:1789760719.069563:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5304700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512572416:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760719.069612:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22839296, mtime: 0x6aad94ce, atime: 0x0, ctime: 0x6aad94ce, blocks: 0xb007 00010000:00010000:0.0:1789760719.069622:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2d9a2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.069637:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=698s v=(0 0) 00010000:00010000:0.0:1789760719.069642:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760719.069652:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2d9a2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22839296->22843391] (req 22839296->22843391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.069666:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2d9a2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22839296->22843391] (req 22839296->22843391) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760719.108342:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760719.108387:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760719.108427:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760719.108459:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512572544 00000100:00100000:1.0:1789760719.108469:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512572544 00000100:00100000:1.0:1789760719.108525:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 388 00000100:00100000:1.0:1789760719.108536:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febe0500850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:e6cf7267-9b19-462a-a263-14115df15dab+13:2360:x1876699512572544:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:1.0:1789760719.108606:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febe0500850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:e6cf7267-9b19-462a-a263-14115df15dab+13:2360:x1876699512572544:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 72us (218us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760719.108614:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 388 00000100:00100000:1.0:1789760719.108646:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512572672 00000100:00100000:1.0:1789760719.108651:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512572672 00000100:00100000:1.0:1789760719.108684:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512572800 00000100:00100000:1.0:1789760719.108687:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512572800 00000100:00100000:1.0:1789760719.108726:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 449 00000100:00100000:1.0:1789760719.108734:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febd1362a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+8:2360:x1876699512572672:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:1.0:1789760719.108765:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febd1362a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+8:2360:x1876699512572672:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 32us (433us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760719.108772:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 449 00000100:00100000:1.0:1789760719.108785:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 450 00000100:00100000:1.0:1789760719.108790:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febd1362300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:d2c40150-e97b-40da-952b-cb9f18bf7e96+5:2360:x1876699512572800:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:1.0:1789760719.108814:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febd1362300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:d2c40150-e97b-40da-952b-cb9f18bf7e96+5:2360:x1876699512572800:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 24us (387us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760719.108820:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 450 00000100:00080000:1.0:1789760719.108911:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000001:00080000:1.0:1789760719.124408:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972895 is committed 00002000:00010000:2.0:1789760719.124680:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22839296 -> 22843392 00010000:00010000:2.0:1789760719.124729:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2d9a2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22839296->22843391] (req 22839296->22843391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760719.124748:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2d9a2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22839296->22843391] (req 22839296->22843391) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760719.124756:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2d9a2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22839296->22843391] (req 22839296->22843391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760719.124763:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760719.124769:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2d9a2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22839296->22843391] (req 22839296->22843391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760719.124778:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2d9a2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22839296->22843391] (req 22839296->22843391) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760719.124799:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2d9a2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22839296->22843391] (req 22839296->22843391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760719.124808:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2d9a2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22839296->22843391] (req 22839296->22843391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760719.124909:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5304700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512572416:12345-192.168.203.56@tcp:4:dd.0 Request processed in 55345us (56256us total) trans 4294972895 rc 0/0 00000100:00100000:2.0:1789760719.124918:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5606 00000100:00100000:2.0:1789760719.124927:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 698572289833 time 698633847285 state 0x2 ref 0 00000100:00100000:1.0:1789760719.133058:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760719.133163:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512572928 00000100:00100000:3.0:1789760719.133190:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512572928 00000100:00100000:3.0:1789760719.133269:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 698572289833 time 698633847285 state 0x0 ref 1 00000100:00100000:3.0:1789760719.133302:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 698634107701 time 698633847285 state 0x1 ref 1 00000100:00100000:3.0:1789760719.133337:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 698634107701 time 698697636590 state 0x0 ref 1 00000100:00100000:3.0:1789760719.133354:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febea771880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512572928:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760719.133421:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22843392, mtime: 0x6aad94ce, atime: 0x0, ctime: 0x6aad94ce, blocks: 0xb007 00010000:00010000:3.0:1789760719.133432:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2d9a9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.133451:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=698s v=(0 0) 00010000:00010000:3.0:1789760719.133490:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760719.133498:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2d9a9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22843392->22847487] (req 22843392->22847487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.133525:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2d9a9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22843392->22847487] (req 22843392->22847487) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760719.191385:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972896 is committed 00002000:00010000:3.0:1789760719.191591:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22843392 -> 22847488 00010000:00010000:3.0:1789760719.191742:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2d9a9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22843392->22847487] (req 22843392->22847487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.191757:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2d9a9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22843392->22847487] (req 22843392->22847487) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.191767:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2d9a9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22843392->22847487] (req 22843392->22847487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.191774:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760719.191781:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2d9a9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22843392->22847487] (req 22843392->22847487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.191791:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2d9a9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22843392->22847487] (req 22843392->22847487) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.191814:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2d9a9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22843392->22847487] (req 22843392->22847487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.191822:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2d9a9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22843392->22847487] (req 22843392->22847487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760719.192010:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febea771880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512572928:12345-192.168.203.56@tcp:4:dd.0 Request processed in 58654us (58953us total) trans 4294972896 rc 0/0 00000100:00100000:3.0:1789760719.192096:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5607 00000100:00100000:3.0:1789760719.192107:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 698634107701 time 698697636590 state 0x2 ref 0 00000100:00100000:3.0:1789760719.195409:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760719.196100:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512573056 00000100:00100000:1.0:1789760719.196114:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512573056 00000100:00100000:1.0:1789760719.196184:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 698634107701 time 698697636590 state 0x0 ref 1 00000100:00100000:1.0:1789760719.196203:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 698697897006 time 698697636590 state 0x1 ref 1 00000100:00100000:1.0:1789760719.196238:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 698697897006 time 698760537058 state 0x0 ref 1 00000100:00100000:1.0:1789760719.196253:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2fb8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512573056:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760719.196316:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22847488, mtime: 0x6aad94ce, atime: 0x0, ctime: 0x6aad94ce, blocks: 0xb007 00010000:00010000:1.0:1789760719.196327:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2d9b0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.196346:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=698s v=(0 0) 00010000:00010000:1.0:1789760719.196351:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760719.196358:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2d9b0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22847488->22851583] (req 22847488->22851583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.196375:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2d9b0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22847488->22851583] (req 22847488->22851583) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.227624:0:7647:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:2.0:1789760719.299824:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972897 is committed 00002000:00010000:3.0:1789760719.300036:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22847488 -> 22851584 00010000:00010000:3.0:1789760719.300059:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2d9b0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22847488->22851583] (req 22847488->22851583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.300073:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2d9b0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22847488->22851583] (req 22847488->22851583) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.300084:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2d9b0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22847488->22851583] (req 22847488->22851583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.300097:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760719.300102:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2d9b0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22847488->22851583] (req 22847488->22851583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.300112:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2d9b0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22847488->22851583] (req 22847488->22851583) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.300136:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2d9b0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22847488->22851583] (req 22847488->22851583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.300275:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2d9b0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22847488->22851583] (req 22847488->22851583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760719.300442:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2fb8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512573056:12345-192.168.203.56@tcp:4:dd.0 Request processed in 104185us (105045us total) trans 4294972897 rc 0/0 00000100:00100000:3.0:1789760719.300457:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5608 00000100:00100000:3.0:1789760719.300483:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 698697897006 time 698760537058 state 0x2 ref 0 00000100:00100000:1.0:1789760719.306025:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760719.306133:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512573184 00000100:00100000:2.0:1789760719.306147:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512573184 00000100:00100000:2.0:1789760719.306235:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 698697897006 time 698760537058 state 0x0 ref 1 00000100:00100000:2.0:1789760719.306255:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 698760797474 time 698760537058 state 0x1 ref 1 00000100:00100000:2.0:1789760719.306295:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 698760797474 time 698870594116 state 0x0 ref 1 00000100:00100000:2.0:1789760719.306310:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febc6bd6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512573184:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760719.306376:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22851584, mtime: 0x6aad94ce, atime: 0x0, ctime: 0x6aad94ce, blocks: 0xb007 00010000:00010000:2.0:1789760719.306387:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2d9b7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760719.306409:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=698s v=(0 0) 00010000:00010000:2.0:1789760719.306415:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760719.306423:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2d9b7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22851584->22855679] (req 22851584->22855679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760719.306441:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2d9b7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22851584->22855679] (req 22851584->22855679) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760719.357476:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972898 is committed 00002000:00010000:0.0:1789760719.357839:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22851584 -> 22855680 00010000:00010000:0.0:1789760719.357856:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2d9b7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22851584->22855679] (req 22851584->22855679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.357869:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2d9b7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22851584->22855679] (req 22851584->22855679) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.357878:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2d9b7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22851584->22855679] (req 22851584->22855679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.357886:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760719.357891:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2d9b7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22851584->22855679] (req 22851584->22855679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.357899:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2d9b7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22851584->22855679] (req 22851584->22855679) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.357919:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2d9b7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22851584->22855679] (req 22851584->22855679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.357928:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2d9b7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22851584->22855679] (req 22851584->22855679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760719.358015:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febc6bd6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512573184:12345-192.168.203.56@tcp:4:dd.0 Request processed in 51703us (52000us total) trans 4294972898 rc 0/0 00000100:00100000:0.0:1789760719.358025:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5609 00000100:00100000:0.0:1789760719.358034:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 698760797474 time 698870594116 state 0x2 ref 0 00000100:00100000:1.0:1789760719.371801:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760719.371972:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512573312 00000100:00100000:3.0:1789760719.371982:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512573312 00000100:00100000:3.0:1789760719.372039:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 698760797474 time 698870594116 state 0x0 ref 1 00000100:00100000:3.0:1789760719.372054:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 698870854532 time 698870594116 state 0x1 ref 1 00000100:00100000:3.0:1789760719.372799:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 698870854532 time 698937094869 state 0x0 ref 1 00000100:00100000:3.0:1789760719.372821:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febea650380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512573312:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760719.372868:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22855680, mtime: 0x6aad94ce, atime: 0x0, ctime: 0x6aad94ce, blocks: 0xb007 00010000:00010000:3.0:1789760719.372877:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2d9be lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.373168:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=698s v=(0 0) 00010000:00010000:3.0:1789760719.373351:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760719.373358:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2d9be lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22855680->22859775] (req 22855680->22859775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.373379:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2d9be lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22855680->22859775] (req 22855680->22859775) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760719.430905:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972899 is committed 00002000:00010000:0.0:1789760719.431355:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22855680 -> 22859776 00010000:00010000:0.0:1789760719.431370:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2d9be lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22855680->22859775] (req 22855680->22859775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.431382:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2d9be lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22855680->22859775] (req 22855680->22859775) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.431391:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2d9be lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22855680->22859775] (req 22855680->22859775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.431398:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760719.431403:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2d9be lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22855680->22859775] (req 22855680->22859775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.431412:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2d9be lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22855680->22859775] (req 22855680->22859775) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.431431:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2d9be lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22855680->22859775] (req 22855680->22859775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.431443:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2d9be lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22855680->22859775] (req 22855680->22859775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760719.431554:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febea650380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512573312:12345-192.168.203.56@tcp:4:dd.0 Request processed in 58735us (59757us total) trans 4294972899 rc 0/0 00000100:00100000:0.0:1789760719.431564:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5610 00000100:00100000:0.0:1789760719.431574:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 698870854532 time 698937094869 state 0x2 ref 0 00000100:00100000:2.0:1789760719.437227:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760719.437371:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512573440 00000100:00100000:3.0:1789760719.437381:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512573440 00000100:00100000:3.0:1789760719.437435:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 698870854532 time 698937094869 state 0x0 ref 1 00000100:00100000:3.0:1789760719.437449:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 698937355285 time 698937094869 state 0x1 ref 1 00000100:00100000:3.0:1789760719.437485:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 698937355285 time 699001782541 state 0x0 ref 1 00000100:00100000:3.0:1789760719.437497:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5306a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512573440:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760719.437557:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22859776, mtime: 0x6aad94ce, atime: 0x0, ctime: 0x6aad94ce, blocks: 0xb007 00010000:00010000:3.0:1789760719.437566:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2d9c5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.437586:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=699s v=(0 0) 00010000:00010000:3.0:1789760719.437590:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760719.437596:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2d9c5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22859776->22863871] (req 22859776->22863871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.437611:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2d9c5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22859776->22863871] (req 22859776->22863871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760719.494550:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972900 is committed 00002000:00010000:0.0:1789760719.495085:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22859776 -> 22863872 00010000:00010000:0.0:1789760719.495109:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2d9c5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22859776->22863871] (req 22859776->22863871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.495128:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2d9c5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22859776->22863871] (req 22859776->22863871) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.495138:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2d9c5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22859776->22863871] (req 22859776->22863871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.495147:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760719.495155:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2d9c5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22859776->22863871] (req 22859776->22863871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.495167:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2d9c5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22859776->22863871] (req 22859776->22863871) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.495201:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2d9c5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22859776->22863871] (req 22859776->22863871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.495209:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2d9c5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22859776->22863871] (req 22859776->22863871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760719.495395:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5306a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512573440:12345-192.168.203.56@tcp:4:dd.0 Request processed in 57892us (58169us total) trans 4294972900 rc 0/0 00000100:00100000:0.0:1789760719.495409:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5611 00000100:00100000:0.0:1789760719.495431:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 698937355285 time 699001782541 state 0x2 ref 0 00000100:00100000:1.0:1789760719.502799:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760719.502910:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512573568 00000100:00100000:0.0:1789760719.502924:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512573568 00000100:00100000:0.0:1789760719.502997:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 698937355285 time 699001782541 state 0x0 ref 1 00000100:00100000:0.0:1789760719.503017:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 699002042957 time 699001782541 state 0x1 ref 1 00000100:00100000:0.0:1789760719.503054:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 699002042957 time 699067353649 state 0x0 ref 1 00000100:00100000:0.0:1789760719.503068:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febea4bbc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512573568:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760719.503137:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22863872, mtime: 0x6aad94ce, atime: 0x0, ctime: 0x6aad94ce, blocks: 0xb007 00010000:00010000:0.0:1789760719.503148:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2d9cc lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.503492:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=699s v=(0 0) 00010000:00010000:0.0:1789760719.503510:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760719.503519:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2d9cc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22863872->22867967] (req 22863872->22867967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.503537:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2d9cc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22863872->22867967] (req 22863872->22867967) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760719.557079:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972901 is committed 00002000:00010000:0.0:1789760719.558655:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22863872 -> 22867968 00002000:00010000:0.0:1789760719.558663:0:13063:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760718 -> 1789760719 00002000:00010000:0.0:1789760719.558666:0:13063:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760718 -> 1789760719 00010000:00010000:0.0:1789760719.558679:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2d9cc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22863872->22867967] (req 22863872->22867967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.558692:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2d9cc lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22863872->22867967] (req 22863872->22867967) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.558716:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2d9cc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22863872->22867967] (req 22863872->22867967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.558724:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760719.558728:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2d9cc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22863872->22867967] (req 22863872->22867967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.558736:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2d9cc lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22863872->22867967] (req 22863872->22867967) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.558754:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2d9cc lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22863872->22867967] (req 22863872->22867967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.558762:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2d9cc lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22863872->22867967] (req 22863872->22867967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760719.558920:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febea4bbc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512573568:12345-192.168.203.56@tcp:4:dd.0 Request processed in 55850us (56130us total) trans 4294972901 rc 0/0 00000100:00100000:0.0:1789760719.558931:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5612 00000100:00100000:0.0:1789760719.558941:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 699002042957 time 699067353649 state 0x2 ref 0 00000100:00100000:1.0:1789760719.565236:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760719.565528:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512573696 00000100:00100000:0.0:1789760719.565539:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512573696 00000100:00100000:0.0:1789760719.565595:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 699002042957 time 699067353649 state 0x0 ref 1 00000100:00100000:0.0:1789760719.565611:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 699067614065 time 699067353649 state 0x1 ref 1 00000100:00100000:0.0:1789760719.565632:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 699067614065 time 699129932028 state 0x0 ref 1 00000100:00100000:0.0:1789760719.565645:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febea651500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512573696:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760719.565725:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22867968, mtime: 0x6aad94cf, atime: 0x0, ctime: 0x6aad94cf, blocks: 0xb007 00010000:00010000:0.0:1789760719.565735:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2d9d3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.565752:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=699s v=(0 0) 00010000:00010000:0.0:1789760719.565757:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760719.565763:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2d9d3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22867968->22872063] (req 22867968->22872063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.565783:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2d9d3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22867968->22872063] (req 22867968->22872063) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760719.639019:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972902 is committed 00002000:00010000:0.0:1789760719.639228:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22867968 -> 22872064 00010000:00010000:0.0:1789760719.639248:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2d9d3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22867968->22872063] (req 22867968->22872063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.639264:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2d9d3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22867968->22872063] (req 22867968->22872063) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.639276:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2d9d3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22867968->22872063] (req 22867968->22872063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.639285:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760719.639292:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2d9d3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22867968->22872063] (req 22867968->22872063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.639303:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2d9d3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22867968->22872063] (req 22867968->22872063) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.639327:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2d9d3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22867968->22872063] (req 22867968->22872063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.639336:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2d9d3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22867968->22872063] (req 22867968->22872063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760719.639581:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febea651500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512573696:12345-192.168.203.56@tcp:4:dd.0 Request processed in 73930us (74347us total) trans 4294972902 rc 0/0 00000100:00100000:0.0:1789760719.639594:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5613 00000100:00100000:0.0:1789760719.639605:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 699067614065 time 699129932028 state 0x2 ref 0 00000100:00100000:2.0:1789760719.655637:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760719.655778:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512573824 00000100:00100000:0.0:1789760719.655791:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512573824 00000100:00100000:0.0:1789760719.655875:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 699067614065 time 699129932028 state 0x0 ref 1 00000100:00100000:0.0:1789760719.655894:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 699130192444 time 699129932028 state 0x1 ref 1 00000100:00100000:0.0:1789760719.655935:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 699130192444 time 699220233529 state 0x0 ref 1 00000100:00100000:0.0:1789760719.655950:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5305880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512573824:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760719.656016:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22872064, mtime: 0x6aad94cf, atime: 0x0, ctime: 0x6aad94cf, blocks: 0xb007 00010000:00010000:0.0:1789760719.656027:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2d9da lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.656046:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=699s v=(0 0) 00010000:00010000:0.0:1789760719.656051:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760719.656059:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2d9da lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22872064->22876159] (req 22872064->22876159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.656075:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2d9da lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22872064->22876159] (req 22872064->22876159) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760719.734092:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22872064 -> 22876160 00010000:00010000:0.0:1789760719.734130:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2d9da lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22872064->22876159] (req 22872064->22876159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.734147:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2d9da lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22872064->22876159] (req 22872064->22876159) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.734158:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2d9da lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22872064->22876159] (req 22872064->22876159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.734176:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760719.734182:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2d9da lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22872064->22876159] (req 22872064->22876159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.734193:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2d9da lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22872064->22876159] (req 22872064->22876159) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.734218:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2d9da lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22872064->22876159] (req 22872064->22876159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.734226:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2d9da lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22872064->22876159] (req 22872064->22876159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760719.734518:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5305880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2361:x1876699512573824:12345-192.168.203.56@tcp:4:dd.0 Request processed in 78563us (78886us total) trans 4294972903 rc 0/0 00000100:00100000:0.0:1789760719.734535:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5614 00000100:00100000:0.0:1789760719.734547:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 699130192444 time 699220233529 state 0x2 ref 0 00000001:00080000:1.0:1789760719.735666:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972903 is committed 00000100:00100000:2.0:1789760719.738788:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760719.738920:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512573952 00000100:00100000:0.0:1789760719.738939:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512573952 00000100:00100000:0.0:1789760719.739012:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 699130192444 time 699220233529 state 0x0 ref 1 00000100:00100000:0.0:1789760719.739030:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 699220493945 time 699220233529 state 0x1 ref 1 00000100:00100000:0.0:1789760719.739057:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 699220493945 time 699303356537 state 0x0 ref 1 00000100:00100000:0.0:1789760719.739070:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5305180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512573952:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760719.739147:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22876160, mtime: 0x6aad94cf, atime: 0x0, ctime: 0x6aad94cf, blocks: 0xb007 00010000:00010000:0.0:1789760719.739159:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2d9e1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.739188:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=699s v=(0 0) 00010000:00010000:0.0:1789760719.739193:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760719.739201:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2d9e1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22876160->22880255] (req 22876160->22880255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.739218:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2d9e1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22876160->22880255] (req 22876160->22880255) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760719.848203:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972904 is committed 00002000:00010000:1.0:1789760719.848409:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22876160 -> 22880256 00010000:00010000:1.0:1789760719.848431:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2d9e1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22876160->22880255] (req 22876160->22880255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.848447:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2d9e1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22876160->22880255] (req 22876160->22880255) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.848457:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2d9e1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22876160->22880255] (req 22876160->22880255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.848478:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760719.848484:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2d9e1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22876160->22880255] (req 22876160->22880255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.848494:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2d9e1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22876160->22880255] (req 22876160->22880255) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.848531:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2d9e1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22876160->22880255] (req 22876160->22880255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.848540:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2d9e1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22876160->22880255] (req 22876160->22880255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760719.848660:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5305180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512573952:12345-192.168.203.56@tcp:4:dd.0 Request processed in 109585us (109882us total) trans 4294972904 rc 0/0 00000100:00100000:1.0:1789760719.848672:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5615 00000100:00100000:1.0:1789760719.848684:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 699220493945 time 699303356537 state 0x2 ref 0 00000100:00100000:2.0:1789760719.852275:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760719.852442:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512574080 00000100:00100000:0.0:1789760719.852455:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512574080 00000100:00100000:0.0:1789760719.852645:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 699220493945 time 699303356537 state 0x0 ref 1 00000100:00100000:0.0:1789760719.852665:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 699303616953 time 699303356537 state 0x1 ref 1 00000100:00100000:0.0:1789760719.852813:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 699303616953 time 699416995257 state 0x0 ref 1 00000100:00100000:0.0:1789760719.852830:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5305500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512574080:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760719.852997:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22880256, mtime: 0x6aad94cf, atime: 0x0, ctime: 0x6aad94cf, blocks: 0xb007 00010000:00010000:0.0:1789760719.853010:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2d9e8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.853030:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=699s v=(0 0) 00010000:00010000:0.0:1789760719.853035:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760719.853043:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2d9e8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22880256->22884351] (req 22880256->22884351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.853059:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2d9e8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22880256->22884351] (req 22880256->22884351) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760719.903691:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972905 is committed 00002000:00010000:0.0:1789760719.903847:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22880256 -> 22884352 00010000:00010000:0.0:1789760719.903866:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2d9e8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22880256->22884351] (req 22880256->22884351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.903882:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2d9e8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22880256->22884351] (req 22880256->22884351) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.903960:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2d9e8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22880256->22884351] (req 22880256->22884351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.903970:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760719.903976:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2d9e8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22880256->22884351] (req 22880256->22884351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.903988:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2d9e8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22880256->22884351] (req 22880256->22884351) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.904014:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2d9e8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22880256->22884351] (req 22880256->22884351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.904022:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2d9e8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22880256->22884351] (req 22880256->22884351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760719.904150:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5305500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512574080:12345-192.168.203.56@tcp:4:dd.0 Request processed in 51316us (51882us total) trans 4294972905 rc 0/0 00000100:00100000:0.0:1789760719.904162:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5616 00000100:00100000:0.0:1789760719.905065:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 699303616953 time 699416995257 state 0x2 ref 0 00000100:00100000:2.0:1789760719.907274:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760719.907506:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512574208 00000100:00100000:0.0:1789760719.907521:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512574208 00000100:00100000:0.0:1789760719.907596:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 699303616953 time 699416995257 state 0x0 ref 1 00000100:00100000:0.0:1789760719.907615:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 699417255673 time 699416995257 state 0x1 ref 1 00000100:00100000:0.0:1789760719.907650:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 699417255673 time 699471948844 state 0x0 ref 1 00000100:00100000:0.0:1789760719.907670:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5304a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512574208:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760719.907758:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22884352, mtime: 0x6aad94cf, atime: 0x0, ctime: 0x6aad94cf, blocks: 0xb007 00010000:00010000:0.0:1789760719.907771:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2d9ef lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.907793:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=699s v=(0 0) 00010000:00010000:0.0:1789760719.907799:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760719.907806:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2d9ef lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22884352->22888447] (req 22884352->22888447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.907823:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2d9ef lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22884352->22888447] (req 22884352->22888447) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760719.945593:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22884352 -> 22888448 00010000:00010000:1.0:1789760719.945620:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2d9ef lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22884352->22888447] (req 22884352->22888447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.945636:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2d9ef lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22884352->22888447] (req 22884352->22888447) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.945644:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2d9ef lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22884352->22888447] (req 22884352->22888447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.945649:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760719.945653:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2d9ef lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22884352->22888447] (req 22884352->22888447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.945665:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2d9ef lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22884352->22888447] (req 22884352->22888447) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.945684:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2d9ef lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22884352->22888447] (req 22884352->22888447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.945689:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2d9ef lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22884352->22888447] (req 22884352->22888447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760719.945780:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972906 is committed 00000100:00100000:1.0:1789760719.945816:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5304a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512574208:12345-192.168.203.56@tcp:4:dd.0 Request processed in 38141us (38552us total) trans 4294972906 rc 0/0 00000100:00100000:1.0:1789760719.945830:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5617 00000100:00100000:1.0:1789760719.945841:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 699417255673 time 699471948844 state 0x2 ref 0 00000100:00100000:2.0:1789760719.949033:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760719.949109:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512574336 00000100:00100000:1.0:1789760719.949129:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512574336 00000100:00100000:1.0:1789760719.949217:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 699417255673 time 699471948844 state 0x0 ref 1 00000100:00100000:1.0:1789760719.949237:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 699472209260 time 699471948844 state 0x1 ref 1 00000100:00100000:1.0:1789760719.949273:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 699472209260 time 699513572262 state 0x0 ref 1 00000100:00100000:1.0:1789760719.949289:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5306d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512574336:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760719.949354:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22888448, mtime: 0x6aad94cf, atime: 0x0, ctime: 0x6aad94cf, blocks: 0xb007 00010000:00010000:1.0:1789760719.949366:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2d9f6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.949388:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=699s v=(0 0) 00010000:00010000:1.0:1789760719.949393:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760719.949400:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2d9f6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22888448->22892543] (req 22888448->22892543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.949417:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2d9f6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22888448->22892543] (req 22888448->22892543) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760719.985270:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972907 is committed 00002000:00010000:1.0:1789760719.985351:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22888448 -> 22892544 00010000:00010000:1.0:1789760719.985369:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2d9f6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22888448->22892543] (req 22888448->22892543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.985380:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2d9f6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22888448->22892543] (req 22888448->22892543) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.985387:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2d9f6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22888448->22892543] (req 22888448->22892543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.985392:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760719.985396:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2d9f6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22888448->22892543] (req 22888448->22892543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.985404:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2d9f6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22888448->22892543] (req 22888448->22892543) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.985467:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2d9f6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22888448->22892543] (req 22888448->22892543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.985477:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2d9f6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22888448->22892543] (req 22888448->22892543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760719.985976:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5306d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512574336:12345-192.168.203.56@tcp:4:dd.0 Request processed in 36681us (36943us total) trans 4294972907 rc 0/0 00000100:00100000:1.0:1789760719.985988:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5618 00000100:00100000:1.0:1789760719.985999:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 699472209260 time 699513572262 state 0x2 ref 0 00000100:00100000:0.0:1789760719.989811:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760719.989934:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512574464 00000100:00100000:1.0:1789760719.989947:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512574464 00000100:00100000:1.0:1789760719.990009:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 699472209260 time 699513572262 state 0x0 ref 1 00000100:00100000:1.0:1789760719.990024:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 699513832678 time 699513572262 state 0x1 ref 1 00000100:00100000:1.0:1789760719.990051:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 699513832678 time 699554350482 state 0x0 ref 1 00000100:00100000:1.0:1789760719.990066:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac3352300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512574464:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760719.990124:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22892544, mtime: 0x6aad94cf, atime: 0x0, ctime: 0x6aad94cf, blocks: 0xb007 00010000:00010000:1.0:1789760719.990132:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2d9fd lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.990148:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=699s v=(0 0) 00010000:00010000:1.0:1789760719.990152:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760719.990158:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2d9fd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22892544->22896639] (req 22892544->22896639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.990222:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2d9fd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22892544->22896639] (req 22892544->22896639) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760720.019997:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972908 is committed 00002000:00010000:1.0:1789760720.020007:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22892544 -> 22896640 00010000:00010000:1.0:1789760720.020028:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2d9fd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22892544->22896639] (req 22892544->22896639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.020043:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2d9fd lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22892544->22896639] (req 22892544->22896639) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.020053:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2d9fd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22892544->22896639] (req 22892544->22896639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.020062:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760720.020068:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2d9fd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22892544->22896639] (req 22892544->22896639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.020078:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2d9fd lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22892544->22896639] (req 22892544->22896639) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.020101:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2d9fd lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22892544->22896639] (req 22892544->22896639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.020107:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2d9fd lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22892544->22896639] (req 22892544->22896639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760720.020239:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac3352300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2361:x1876699512574464:12345-192.168.203.56@tcp:4:dd.0 Request processed in 30168us (30434us total) trans 4294972908 rc 0/0 00000100:00100000:1.0:1789760720.020253:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5619 00000100:00100000:1.0:1789760720.020267:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 699513832678 time 699554350482 state 0x2 ref 0 00000100:00100000:3.0:1789760720.023905:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760720.023963:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512574592 00000100:00100000:1.0:1789760720.023976:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512574592 00000100:00100000:1.0:1789760720.024045:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 699513832678 time 699554350482 state 0x0 ref 1 00000100:00100000:1.0:1789760720.024064:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 699554610898 time 699554350482 state 0x1 ref 1 00000100:00100000:1.0:1789760720.024093:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 699554610898 time 699588393076 state 0x0 ref 1 00000100:00100000:1.0:1789760720.024105:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2fb9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512574592:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760720.024167:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22896640, mtime: 0x6aad94cf, atime: 0x0, ctime: 0x6aad94cf, blocks: 0xb007 00010000:00010000:1.0:1789760720.024176:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2da04 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.024195:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=699s v=(0 0) 00010000:00010000:1.0:1789760720.024200:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760720.024207:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2da04 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22896640->22900735] (req 22896640->22900735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.024224:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2da04 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22896640->22900735] (req 22896640->22900735) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760720.071320:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972909 is committed 00002000:00010000:1.0:1789760720.071575:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22896640 -> 22900736 00010000:00010000:1.0:1789760720.071599:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2da04 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22896640->22900735] (req 22896640->22900735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.071618:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2da04 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22896640->22900735] (req 22896640->22900735) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.071630:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2da04 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22896640->22900735] (req 22896640->22900735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.071638:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760720.071644:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2da04 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22896640->22900735] (req 22896640->22900735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.071656:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2da04 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22896640->22900735] (req 22896640->22900735) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.071680:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2da04 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22896640->22900735] (req 22896640->22900735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.071686:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2da04 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22896640->22900735] (req 22896640->22900735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760720.071800:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2fb9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512574592:12345-192.168.203.56@tcp:4:dd.0 Request processed in 47691us (47905us total) trans 4294972909 rc 0/0 00000100:00100000:1.0:1789760720.071812:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5620 00000100:00100000:1.0:1789760720.071822:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 699554610898 time 699588393076 state 0x2 ref 0 00000100:00100000:0.0:1789760720.075922:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760720.076137:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512574720 00000100:00100000:1.0:1789760720.076151:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512574720 00000100:00100000:1.0:1789760720.076237:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 699554610898 time 699588393076 state 0x0 ref 1 00000100:00100000:1.0:1789760720.076257:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 699588653492 time 699588393076 state 0x1 ref 1 00000100:00100000:1.0:1789760720.076294:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 699588653492 time 699640592902 state 0x0 ref 1 00000100:00100000:1.0:1789760720.076309:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac3353800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512574720:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760720.076384:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22900736, mtime: 0x6aad94cf, atime: 0x0, ctime: 0x6aad94cf, blocks: 0xb007 00010000:00010000:1.0:1789760720.076396:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2da0b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.076416:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=699s v=(0 0) 00010000:00010000:1.0:1789760720.076422:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760720.076429:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2da0b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22900736->22904831] (req 22900736->22904831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.076446:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2da0b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22900736->22904831] (req 22900736->22904831) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760720.147606:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972910 is committed 00002000:00010000:1.0:1789760720.147729:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22900736 -> 22904832 00010000:00010000:1.0:1789760720.147749:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2da0b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22900736->22904831] (req 22900736->22904831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.147774:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2da0b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22900736->22904831] (req 22900736->22904831) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.147785:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2da0b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22900736->22904831] (req 22900736->22904831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.147793:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760720.147799:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2da0b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22900736->22904831] (req 22900736->22904831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.148189:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2da0b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22900736->22904831] (req 22900736->22904831) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.148215:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2da0b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22900736->22904831] (req 22900736->22904831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.148223:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2da0b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22900736->22904831] (req 22900736->22904831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760720.148419:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac3353800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512574720:12345-192.168.203.56@tcp:4:dd.0 Request processed in 72106us (72502us total) trans 4294972910 rc 0/0 00000100:00100000:2.0:1789760720.148432:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5621 00000100:00100000:2.0:1789760720.148445:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 699588653492 time 699640592902 state 0x2 ref 0 00000100:00100000:0.0:1789760720.155863:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760720.156126:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512574848 00000100:00100000:1.0:1789760720.156140:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512574848 00000100:00100000:1.0:1789760720.156209:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 699588653492 time 699640592902 state 0x0 ref 1 00000100:00100000:1.0:1789760720.156228:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 699640853318 time 699640592902 state 0x1 ref 1 00000100:00100000:1.0:1789760720.156266:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 699640853318 time 699720565434 state 0x0 ref 1 00000100:00100000:1.0:1789760720.156281:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac3352a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512574848:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760720.156344:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22904832, mtime: 0x6aad94cf, atime: 0x0, ctime: 0x6aad94cf, blocks: 0xb007 00010000:00010000:1.0:1789760720.156356:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2da12 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.156375:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=699s v=(0 0) 00010000:00010000:1.0:1789760720.156381:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760720.156388:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2da12 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22904832->22908927] (req 22904832->22908927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.156405:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2da12 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22904832->22908927] (req 22904832->22908927) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760720.247452:0:5836:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:3.0:1789760720.263883:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972911 is committed 00002000:00010000:2.0:1789760720.266036:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22904832 -> 22908928 00010000:00010000:2.0:1789760720.266064:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2da12 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22904832->22908927] (req 22904832->22908927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760720.266079:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2da12 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22904832->22908927] (req 22904832->22908927) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760720.266089:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2da12 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22904832->22908927] (req 22904832->22908927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760720.266097:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760720.266103:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2da12 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22904832->22908927] (req 22904832->22908927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760720.266112:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2da12 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22904832->22908927] (req 22904832->22908927) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760720.266140:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2da12 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22904832->22908927] (req 22904832->22908927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760720.266149:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2da12 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22904832->22908927] (req 22904832->22908927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760720.266633:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac3352a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512574848:12345-192.168.203.56@tcp:4:dd.0 Request processed in 110346us (110778us total) trans 4294972911 rc 0/0 00000100:00100000:2.0:1789760720.266647:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5622 00000100:00100000:2.0:1789760720.266659:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 699640853318 time 699720565434 state 0x2 ref 0 00000100:00100000:2.0:1789760720.279681:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760720.280463:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512574976 00000100:00100000:1.0:1789760720.280480:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512574976 00000100:00100000:1.0:1789760720.280567:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 699640853318 time 699720565434 state 0x0 ref 1 00000100:00100000:1.0:1789760720.280586:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 699720825850 time 699720565434 state 0x1 ref 1 00000100:00100000:1.0:1789760720.280627:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 699720825850 time 699844926117 state 0x0 ref 1 00000100:00100000:1.0:1789760720.280645:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5306300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512574976:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760720.281398:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22908928, mtime: 0x6aad94cf, atime: 0x0, ctime: 0x6aad94cf, blocks: 0xb007 00010000:00010000:1.0:1789760720.281412:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2da19 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.281439:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=699s v=(0 0) 00010000:00010000:1.0:1789760720.281445:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760720.281454:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2da19 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22908928->22913023] (req 22908928->22913023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.281472:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2da19 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22908928->22913023] (req 22908928->22913023) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760720.337068:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972912 is committed 00002000:00010000:1.0:1789760720.337380:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22908928 -> 22913024 00010000:00010000:1.0:1789760720.337401:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2da19 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22908928->22913023] (req 22908928->22913023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.337417:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2da19 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22908928->22913023] (req 22908928->22913023) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.337428:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2da19 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22908928->22913023] (req 22908928->22913023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.337436:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760720.337442:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2da19 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22908928->22913023] (req 22908928->22913023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.337453:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2da19 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22908928->22913023] (req 22908928->22913023) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.337478:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2da19 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22908928->22913023] (req 22908928->22913023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.337487:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2da19 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22908928->22913023] (req 22908928->22913023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760720.337651:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5306300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512574976:12345-192.168.203.56@tcp:4:dd.0 Request processed in 57002us (57979us total) trans 4294972912 rc 0/0 00000100:00100000:1.0:1789760720.337663:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5623 00000100:00100000:1.0:1789760720.337675:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 699720825850 time 699844926117 state 0x2 ref 0 00000100:00100000:1.0:1789760720.342576:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760720.342666:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512575104 00000100:00100000:2.0:1789760720.342678:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512575104 00000100:00100000:2.0:1789760720.342782:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 699720825850 time 699844926117 state 0x0 ref 1 00000100:00100000:2.0:1789760720.342799:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 699845186533 time 699844926117 state 0x1 ref 1 00000100:00100000:2.0:1789760720.342830:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 699845186533 time 699907129809 state 0x0 ref 1 00000100:00100000:2.0:1789760720.342846:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febea650380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512575104:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760720.342908:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22913024, mtime: 0x6aad94cf, atime: 0x0, ctime: 0x6aad94cf, blocks: 0xb007 00010000:00010000:2.0:1789760720.342919:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2da20 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760720.342946:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=699s v=(0 0) 00010000:00010000:2.0:1789760720.342952:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760720.342969:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2da20 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22913024->22917119] (req 22913024->22917119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760720.342985:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2da20 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22913024->22917119] (req 22913024->22917119) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760720.398259:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22913024 -> 22917120 00010000:00010000:0.0:1789760720.398289:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2da20 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22913024->22917119] (req 22913024->22917119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760720.398306:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2da20 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22913024->22917119] (req 22913024->22917119) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760720.398316:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2da20 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22913024->22917119] (req 22913024->22917119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760720.398326:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760720.398332:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2da20 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22913024->22917119] (req 22913024->22917119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760720.398341:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2da20 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22913024->22917119] (req 22913024->22917119) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760720.398366:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2da20 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22913024->22917119] (req 22913024->22917119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760720.398375:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2da20 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22913024->22917119] (req 22913024->22917119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760720.398566:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febea650380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2359:x1876699512575104:12345-192.168.203.56@tcp:4:dd.0 Request processed in 55706us (55985us total) trans 4294972913 rc 0/0 00000100:00100000:0.0:1789760720.398580:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5624 00000100:00100000:0.0:1789760720.398592:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 699845186533 time 699907129809 state 0x2 ref 0 00000001:00080000:1.0:1789760720.402923:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972913 is committed 00000100:00100000:0.0:1789760720.409143:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760720.409432:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512575232 00000100:00100000:2.0:1789760720.409446:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512575232 00000100:00100000:2.0:1789760720.409546:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 699845186533 time 699907129809 state 0x0 ref 1 00000100:00100000:2.0:1789760720.409568:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 699907390225 time 699907129809 state 0x1 ref 1 00000100:00100000:2.0:1789760720.409607:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 699907390225 time 699973905719 state 0x0 ref 1 00000100:00100000:2.0:1789760720.409623:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac3353480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512575232:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760720.409745:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22917120, mtime: 0x6aad94cf, atime: 0x0, ctime: 0x6aad94cf, blocks: 0xb007 00010000:00010000:2.0:1789760720.409759:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2da27 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760720.409779:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=699s v=(0 0) 00010000:00010000:2.0:1789760720.409784:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760720.409793:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2da27 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22917120->22921215] (req 22917120->22921215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760720.409819:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2da27 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22917120->22921215] (req 22917120->22921215) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760720.496765:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972914 is committed 00002000:00010000:3.0:1789760720.496816:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22917120 -> 22921216 00010000:00010000:3.0:1789760720.496846:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2da27 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22917120->22921215] (req 22917120->22921215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.496857:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2da27 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22917120->22921215] (req 22917120->22921215) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.496864:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2da27 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22917120->22921215] (req 22917120->22921215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.496869:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760720.496872:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2da27 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22917120->22921215] (req 22917120->22921215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.496878:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2da27 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22917120->22921215] (req 22917120->22921215) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.496897:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2da27 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22917120->22921215] (req 22917120->22921215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.496902:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2da27 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22917120->22921215] (req 22917120->22921215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760720.496981:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac3353480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512575232:12345-192.168.203.56@tcp:4:dd.0 Request processed in 87357us (87851us total) trans 4294972914 rc 0/0 00000100:00100000:3.0:1789760720.496991:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5625 00000100:00100000:3.0:1789760720.497001:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 699907390225 time 699973905719 state 0x2 ref 0 00000100:00100000:3.0:1789760720.500160:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760720.500270:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512575360 00000100:00100000:1.0:1789760720.500280:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512575360 00000100:00100000:1.0:1789760720.500340:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 699907390225 time 699973905719 state 0x0 ref 1 00000100:00100000:1.0:1789760720.500355:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 699974166135 time 699973905719 state 0x1 ref 1 00000100:00100000:1.0:1789760720.500443:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 699974166135 time 700064741802 state 0x0 ref 1 00000100:00100000:1.0:1789760720.500456:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2fb8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512575360:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760720.500517:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22921216, mtime: 0x6aad94cf, atime: 0x0, ctime: 0x6aad94cf, blocks: 0xb007 00010000:00010000:1.0:1789760720.500526:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2da2e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.500543:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=700s v=(0 0) 00010000:00010000:1.0:1789760720.500548:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760720.500554:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2da2e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22921216->22925311] (req 22921216->22925311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.500570:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2da2e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22921216->22925311] (req 22921216->22925311) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760720.530225:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972915 is committed 00002000:00010000:3.0:1789760720.530406:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22921216 -> 22925312 00002000:00010000:3.0:1789760720.530410:0:6709:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760719 -> 1789760720 00002000:00010000:3.0:1789760720.530414:0:6709:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760719 -> 1789760720 00010000:00010000:3.0:1789760720.530427:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2da2e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22921216->22925311] (req 22921216->22925311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.530440:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2da2e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22921216->22925311] (req 22921216->22925311) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.530450:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2da2e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22921216->22925311] (req 22921216->22925311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.530458:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760720.530463:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2da2e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22921216->22925311] (req 22921216->22925311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.530474:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2da2e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22921216->22925311] (req 22921216->22925311) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.530506:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2da2e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22921216->22925311] (req 22921216->22925311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.530515:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2da2e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22921216->22925311] (req 22921216->22925311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760720.530900:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2fb8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512575360:12345-192.168.203.56@tcp:4:dd.0 Request processed in 30441us (30743us total) trans 4294972915 rc 0/0 00000100:00100000:3.0:1789760720.530911:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5626 00000100:00100000:3.0:1789760720.530920:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 699974166135 time 700064741802 state 0x2 ref 0 00000100:00100000:1.0:1789760720.536337:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760720.536479:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512575488 00000100:00100000:0.0:1789760720.536531:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512575488 00000100:00100000:0.0:1789760720.536585:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 699974166135 time 700064741802 state 0x0 ref 1 00000100:00100000:0.0:1789760720.536600:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 700065002218 time 700064741802 state 0x1 ref 1 00000100:00100000:0.0:1789760720.536629:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 700065002218 time 700100929116 state 0x0 ref 1 00000100:00100000:0.0:1789760720.536643:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4132300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512575488:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760720.536688:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22925312, mtime: 0x6aad94d0, atime: 0x0, ctime: 0x6aad94d0, blocks: 0xb007 00010000:00010000:0.0:1789760720.536728:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2da35 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760720.536744:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=700s v=(0 0) 00010000:00010000:0.0:1789760720.536748:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760720.536755:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2da35 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22925312->22929407] (req 22925312->22929407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760720.536768:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2da35 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22925312->22929407] (req 22925312->22929407) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760720.566761:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22925312 -> 22929408 00010000:00010000:2.0:1789760720.566782:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2da35 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22925312->22929407] (req 22925312->22929407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760720.566795:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2da35 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22925312->22929407] (req 22925312->22929407) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760720.566804:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2da35 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22925312->22929407] (req 22925312->22929407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760720.566812:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760720.566817:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2da35 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22925312->22929407] (req 22925312->22929407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760720.566827:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2da35 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22925312->22929407] (req 22925312->22929407) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760720.566846:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2da35 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22925312->22929407] (req 22925312->22929407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760720.566855:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2da35 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22925312->22929407] (req 22925312->22929407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760720.566941:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4132300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2360:x1876699512575488:12345-192.168.203.56@tcp:4:dd.0 Request processed in 30295us (30608us total) trans 4294972916 rc 0/0 00000100:00100000:2.0:1789760720.566952:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5627 00000100:00100000:2.0:1789760720.566962:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 700065002218 time 700100929116 state 0x2 ref 0 00000001:00080000:1.0:1789760720.567055:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972916 is committed 00000100:00100000:2.0:1789760720.570594:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760720.570681:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512575616 00000100:00100000:3.0:1789760720.570692:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512575616 00000100:00100000:3.0:1789760720.570810:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 700065002218 time 700100929116 state 0x0 ref 1 00000100:00100000:3.0:1789760720.570826:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 700101189532 time 700100929116 state 0x1 ref 1 00000100:00100000:3.0:1789760720.570903:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 700101189532 time 700135201694 state 0x0 ref 1 00000100:00100000:3.0:1789760720.570916:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5307800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512575616:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760720.570966:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22929408, mtime: 0x6aad94d0, atime: 0x0, ctime: 0x6aad94d0, blocks: 0xb007 00010000:00010000:3.0:1789760720.570975:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2da3c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.570989:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=700s v=(0 0) 00010000:00010000:3.0:1789760720.570993:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760720.570999:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2da3c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22929408->22933503] (req 22929408->22933503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.571024:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2da3c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22929408->22933503] (req 22929408->22933503) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760720.606857:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972917 is committed 00002000:00010000:0.0:1789760720.607238:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22929408 -> 22933504 00010000:00010000:0.0:1789760720.607254:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2da3c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22929408->22933503] (req 22929408->22933503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760720.607270:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2da3c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22929408->22933503] (req 22929408->22933503) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760720.607281:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2da3c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22929408->22933503] (req 22929408->22933503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760720.607288:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760720.607295:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2da3c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22929408->22933503] (req 22929408->22933503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760720.607306:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2da3c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22929408->22933503] (req 22929408->22933503) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760720.607328:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2da3c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22929408->22933503] (req 22929408->22933503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760720.607337:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2da3c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22929408->22933503] (req 22929408->22933503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760720.607445:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5307800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512575616:12345-192.168.203.56@tcp:4:dd.0 Request processed in 36524us (36851us total) trans 4294972917 rc 0/0 00000100:00100000:0.0:1789760720.607456:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5628 00000100:00100000:0.0:1789760720.607466:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 700101189532 time 700135201694 state 0x2 ref 0 00000100:00100000:0.0:1789760720.612722:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760720.612763:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512575744 00000100:00100000:3.0:1789760720.612778:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512575744 00000100:00100000:3.0:1789760720.612828:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 700101189532 time 700135201694 state 0x0 ref 1 00000100:00100000:3.0:1789760720.612840:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 700135462110 time 700135201694 state 0x1 ref 1 00000100:00100000:3.0:1789760720.612865:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 700135462110 time 700177165764 state 0x0 ref 1 00000100:00100000:3.0:1789760720.612877:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac38b6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512575744:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760720.612926:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22933504, mtime: 0x6aad94d0, atime: 0x0, ctime: 0x6aad94d0, blocks: 0xb007 00010000:00010000:3.0:1789760720.612935:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2da43 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.612951:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=700s v=(0 0) 00010000:00010000:3.0:1789760720.612958:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760720.612964:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2da43 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22933504->22937599] (req 22933504->22937599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.612991:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2da43 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22933504->22937599] (req 22933504->22937599) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760720.659424:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972918 is committed 00002000:00010000:0.0:1789760720.659519:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22933504 -> 22937600 00010000:00010000:0.0:1789760720.659534:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2da43 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22933504->22937599] (req 22933504->22937599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760720.659548:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2da43 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22933504->22937599] (req 22933504->22937599) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760720.659557:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2da43 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22933504->22937599] (req 22933504->22937599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760720.659570:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760720.659575:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2da43 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22933504->22937599] (req 22933504->22937599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760720.659586:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2da43 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22933504->22937599] (req 22933504->22937599) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760720.659604:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2da43 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22933504->22937599] (req 22933504->22937599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760720.659613:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2da43 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22933504->22937599] (req 22933504->22937599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760720.659780:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac38b6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512575744:12345-192.168.203.56@tcp:4:dd.0 Request processed in 46900us (47081us total) trans 4294972918 rc 0/0 00000100:00100000:0.0:1789760720.659791:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5629 00000100:00100000:0.0:1789760720.659800:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 700135462110 time 700177165764 state 0x2 ref 0 00000100:00100000:3.0:1789760720.663458:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760720.663529:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512575872 00000100:00100000:2.0:1789760720.663539:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512575872 00000100:00100000:2.0:1789760720.663592:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 700135462110 time 700177165764 state 0x0 ref 1 00000100:00100000:2.0:1789760720.663609:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 700177426180 time 700177165764 state 0x1 ref 1 00000100:00100000:2.0:1789760720.663641:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 700177426180 time 700227939777 state 0x0 ref 1 00000100:00100000:2.0:1789760720.663654:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2fba680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512575872:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760720.663727:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22937600, mtime: 0x6aad94d0, atime: 0x0, ctime: 0x6aad94d0, blocks: 0xb007 00010000:00010000:2.0:1789760720.663737:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2da4a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760720.663753:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=700s v=(0 0) 00010000:00010000:2.0:1789760720.663758:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760720.663765:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2da4a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22937600->22941695] (req 22937600->22941695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760720.663779:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2da4a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22937600->22941695] (req 22937600->22941695) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760720.693553:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22937600 -> 22941696 00010000:00010000:2.0:1789760720.693575:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2da4a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22937600->22941695] (req 22937600->22941695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760720.693588:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2da4a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22937600->22941695] (req 22937600->22941695) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760720.693597:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2da4a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22937600->22941695] (req 22937600->22941695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760720.693605:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760720.693611:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2da4a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22937600->22941695] (req 22937600->22941695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760720.693621:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2da4a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22937600->22941695] (req 22937600->22941695) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760720.693640:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2da4a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22937600->22941695] (req 22937600->22941695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760720.693649:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2da4a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22937600->22941695] (req 22937600->22941695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760720.693772:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2fba680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2358:x1876699512575872:12345-192.168.203.56@tcp:4:dd.0 Request processed in 30115us (30318us total) trans 4294972919 rc 0/0 00000100:00100000:2.0:1789760720.693786:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5630 00000100:00100000:2.0:1789760720.693796:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 700177426180 time 700227939777 state 0x2 ref 0 00000001:00080000:3.0:1789760720.694099:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972919 is committed 00000100:00100000:2.0:1789760720.697615:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760720.697686:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512576000 00000100:00100000:0.0:1789760720.697719:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512576000 00000100:00100000:0.0:1789760720.697770:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 700177426180 time 700227939777 state 0x0 ref 1 00000100:00100000:0.0:1789760720.697785:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 700228200193 time 700227939777 state 0x1 ref 1 00000100:00100000:0.0:1789760720.697814:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 700228200193 time 700262114201 state 0x0 ref 1 00000100:00100000:0.0:1789760720.697830:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5304000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512576000:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760720.697878:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22941696, mtime: 0x6aad94d0, atime: 0x0, ctime: 0x6aad94d0, blocks: 0xb007 00010000:00010000:0.0:1789760720.697897:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2da51 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760720.697914:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=700s v=(0 0) 00010000:00010000:0.0:1789760720.697919:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760720.697925:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2da51 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22941696->22945791] (req 22941696->22945791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760720.697939:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2da51 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22941696->22945791] (req 22941696->22945791) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760720.782668:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972920 is committed 00002000:00010000:3.0:1789760720.782872:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22941696 -> 22945792 00010000:00010000:3.0:1789760720.782889:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2da51 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22941696->22945791] (req 22941696->22945791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.782902:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2da51 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22941696->22945791] (req 22941696->22945791) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.782911:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2da51 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22941696->22945791] (req 22941696->22945791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.782919:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760720.782924:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2da51 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22941696->22945791] (req 22941696->22945791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.782933:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2da51 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22941696->22945791] (req 22941696->22945791) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.782952:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2da51 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22941696->22945791] (req 22941696->22945791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.782961:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2da51 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22941696->22945791] (req 22941696->22945791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760720.783178:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5304000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512576000:12345-192.168.203.56@tcp:4:dd.0 Request processed in 85343us (85565us total) trans 4294972920 rc 0/0 00000100:00100000:3.0:1789760720.783190:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5631 00000100:00100000:3.0:1789760720.783202:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 700228200193 time 700262114201 state 0x2 ref 0 00000100:00100000:2.0:1789760720.786886:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760720.787863:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512576128 00000100:00100000:0.0:1789760720.787876:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512576128 00000100:00100000:0.0:1789760720.787939:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 700228200193 time 700262114201 state 0x0 ref 1 00000100:00100000:0.0:1789760720.787968:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 700262374617 time 700262114201 state 0x1 ref 1 00000100:00100000:0.0:1789760720.788045:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 700262374617 time 700352344321 state 0x0 ref 1 00000100:00100000:0.0:1789760720.788059:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5304380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512576128:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760720.788109:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22945792, mtime: 0x6aad94d0, atime: 0x0, ctime: 0x6aad94d0, blocks: 0xb007 00010000:00010000:0.0:1789760720.788118:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2da58 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760720.788138:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=700s v=(0 0) 00010000:00010000:0.0:1789760720.788142:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760720.788149:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2da58 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22945792->22949887] (req 22945792->22949887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760720.788164:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2da58 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22945792->22949887] (req 22945792->22949887) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760720.819983:0:3312:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8feac2fb9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3312:1876699540219776:0@lo:400:kworker.0 00000100:00100000:1.0:1789760720.820382:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760720.820478:0:3312:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [2->3] 00000100:00100000:1.0:1789760720.820487:0:3312:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8feac2fb8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3312:1876699540219648:0@lo:400:kworker.0 00000100:00100000:1.0:1789760720.820524:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760720.820592:0:3310:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8feac2fbb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:6df98d54-eb52-4c5c-8fc9-ee9d74f90f98:3310:1876699540219520:0@lo:400:kworker.0 00000100:00100000:2.0:1789760720.820628:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760720.820657:0:3310:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8feac2fb8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3310:1876699540220032:0@lo:400:kworker.0 00000100:00100000:2.0:1789760720.820714:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540219520 00000100:00100000:2.0:1789760720.820721:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540219520 00000100:00100000:0.0:1789760720.820729:0:3309:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8feac2fbaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3309:1876699540219904:0@lo:400:kworker.0 00000100:00100000:0.0:1789760720.820769:0:3309:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760720.820769:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 389 00000100:00100000:0.0:1789760720.820805:0:10102:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540219904 00000100:00100000:0.0:1789760720.820810:0:10102:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540219904 00000100:00100000:0.0:1789760720.820851:0:10102:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 173 00000100:00100000:0.0:1789760720.820860:0:10102:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac38b6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3309:x1876699540219904:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760720.820909:0:10102:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac38b6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3309:x1876699540219904:12345-0@lo:400:kworker.0 Request processed in 50us (142us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760720.820916:0:10102:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 173 00000100:00080000:0.0:1789760720.820936:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1789760720.820965:0:3309:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8feac2fbaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3309:1876699540219904:0@lo:400:kworker.0 00000100:00100000:0.0:1789760720.821183:0:3311:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1789760720.821192:0:3311:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8feac38b4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3311:1876699540220288:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1789760720.821220:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760720.821372:0:6706:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540220288 00000100:00100000:0.0:1789760720.821376:0:6706:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540220288 00000100:00100000:0.0:1789760720.821401:0:6706:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 188 00000100:00100000:0.0:1789760720.821431:0:6706:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac38b5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3311:x1876699540220288:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1789760720.821448:0:6706:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 527892480 pending 1703936 free 3835691008 avail 3827302400 00000100:00100000:0.0:1789760720.821484:0:6706:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac38b5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3311:x1876699540220288:12345-0@lo:13:osp-pre-0-0.0 Request processed in 53us (262us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760720.821501:0:6706:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 188 00000100:00100000:0.0:1789760720.821536:0:3311:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8feac38b4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3311:1876699540220288:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789760720.821566:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febe0500c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:6df98d54-eb52-4c5c-8fc9-ee9d74f90f98+19:3310:x1876699540219520:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760720.821622:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febe0500c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:6df98d54-eb52-4c5c-8fc9-ee9d74f90f98+19:3310:x1876699540219520:12345-0@lo:400:kworker.0 Request processed in 60us (993us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760720.821630:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 389 00000100:00100000:1.0:1789760720.821784:0:3312:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8febe7907100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3312:1876699540220160:0@lo:400:kworker.0 00000100:00100000:1.0:1789760720.821812:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760720.821838:0:6705:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540220160 00000100:00100000:1.0:1789760720.821842:0:6705:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540220160 00000100:00100000:1.0:1789760720.821874:0:6705:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 174 00000100:00100000:1.0:1789760720.821880:0:6705:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4133100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3312:x1876699540220160:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760720.821916:0:6705:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4133100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3312:x1876699540220160:12345-0@lo:400:kworker.0 Request processed in 35us (102us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760720.821922:0:6705:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 174 00000100:00100000:1.0:1789760720.821956:0:3312:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8febe7907100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3312:1876699540220160:0@lo:400:kworker.0 00000100:00100000:3.0:1789760720.824195:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540219776 00000100:00100000:3.0:1789760720.824209:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540219776 00000100:00100000:0.0:1789760720.826687:0:5848:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540219648 00000100:00100000:0.0:1789760720.826722:0:5848:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540219648 00000100:00100000:0.0:1789760720.826771:0:5848:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 451 00000100:00100000:0.0:1789760720.826782:0:5848:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4131c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3312:x1876699540219776:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760720.826931:0:3312:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8feac2fb9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3312:1876699540219776:0@lo:400:kworker.0 00000100:00100000:0.0:1789760720.829014:0:5848:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4131c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3312:x1876699540219776:12345-0@lo:400:kworker.0 Request processed in 2214us (8612us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760720.829029:0:5848:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 451 00000100:00100000:0.0:1789760720.829069:0:5848:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 452 00000100:00100000:0.0:1789760720.829081:0:5848:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4130e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3312:x1876699540219648:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760720.829171:0:5848:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4130e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3312:x1876699540219648:12345-0@lo:400:kworker.0 Request processed in 94us (8644us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760720.829177:0:5848:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 452 00000100:00080000:0.0:1789760720.829199:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:0.0:1789760720.829204:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1789760720.829219:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760720.829277:0:3312:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8feac2fb8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3312:1876699540219648:0@lo:400:kworker.0 00000100:00100000:0.0:1789760720.829286:0:3310:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8feac2fbb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:6df98d54-eb52-4c5c-8fc9-ee9d74f90f98:3310:1876699540219520:0@lo:400:kworker.0 00000100:00100000:0.0:1789760720.829316:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540220032 00000100:00100000:0.0:1789760720.829322:0:5847:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540220032 00000100:00100000:0.0:1789760720.829371:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 453 00000100:00100000:0.0:1789760720.829378:0:5847:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac38b4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3310:x1876699540220032:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760720.829416:0:5847:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac38b4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3310:x1876699540220032:12345-0@lo:400:kworker.0 Request processed in 37us (196us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760720.829423:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 453 00000100:00080000:2.0:1789760720.829428:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789760720.829453:0:3310:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8feac2fb8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3310:1876699540220032:0@lo:400:kworker.0 00000001:00080000:0.0:1789760720.857345:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972921 is committed 00002000:00010000:0.0:1789760720.857417:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22945792 -> 22949888 00010000:00010000:0.0:1789760720.857431:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2da58 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22945792->22949887] (req 22945792->22949887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760720.857443:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2da58 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22945792->22949887] (req 22945792->22949887) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760720.857450:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2da58 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22945792->22949887] (req 22945792->22949887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760720.857457:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760720.857462:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2da58 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22945792->22949887] (req 22945792->22949887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760720.857470:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2da58 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22945792->22949887] (req 22945792->22949887) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760720.857499:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2da58 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22945792->22949887] (req 22945792->22949887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760720.857518:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2da58 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22945792->22949887] (req 22945792->22949887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760720.857990:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5304380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512576128:12345-192.168.203.56@tcp:4:dd.0 Request processed in 69926us (71105us total) trans 4294972921 rc 0/0 00000100:00100000:0.0:1789760720.858001:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5632 00000100:00100000:0.0:1789760720.858011:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 700262374617 time 700352344321 state 0x2 ref 0 00000100:00100000:0.0:1789760720.863466:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760720.863649:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512576256 00000100:00100000:2.0:1789760720.863659:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512576256 00000100:00100000:2.0:1789760720.863897:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 700262374617 time 700352344321 state 0x0 ref 1 00000100:00100000:2.0:1789760720.863912:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 700352604737 time 700352344321 state 0x1 ref 1 00000100:00100000:2.0:1789760720.863980:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 700352604737 time 700428278990 state 0x0 ref 1 00000100:00100000:2.0:1789760720.863993:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2fb8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512576256:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760720.864042:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22949888, mtime: 0x6aad94d0, atime: 0x0, ctime: 0x6aad94d0, blocks: 0xb007 00010000:00010000:2.0:1789760720.864052:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2da5f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760720.864070:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=700s v=(0 0) 00010000:00010000:2.0:1789760720.864074:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760720.864080:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2da5f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22949888->22953983] (req 22949888->22953983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760720.864093:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2da5f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22949888->22953983] (req 22949888->22953983) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760720.960139:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972922 is committed 00002000:00010000:2.0:1789760720.961054:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22949888 -> 22953984 00010000:00010000:2.0:1789760720.961071:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2da5f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22949888->22953983] (req 22949888->22953983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760720.961084:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2da5f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22949888->22953983] (req 22949888->22953983) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760720.961093:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2da5f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22949888->22953983] (req 22949888->22953983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760720.961099:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760720.961104:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2da5f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22949888->22953983] (req 22949888->22953983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760720.961113:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2da5f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22949888->22953983] (req 22949888->22953983) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760720.961133:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2da5f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22949888->22953983] (req 22949888->22953983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760720.961140:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2da5f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22949888->22953983] (req 22949888->22953983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760720.961356:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2fb8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512576256:12345-192.168.203.56@tcp:4:dd.0 Request processed in 97360us (97893us total) trans 4294972922 rc 0/0 00000100:00100000:2.0:1789760720.961366:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5633 00000100:00100000:2.0:1789760720.961377:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 700352604737 time 700428278990 state 0x2 ref 0 00000100:00100000:0.0:1789760720.968330:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760720.968541:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512576384 00000100:00100000:3.0:1789760720.968551:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512576384 00000100:00100000:3.0:1789760720.968610:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 700352604737 time 700428278990 state 0x0 ref 1 00000100:00100000:3.0:1789760720.968635:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 700428539406 time 700428278990 state 0x1 ref 1 00000100:00100000:3.0:1789760720.968668:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 700428539406 time 700532968452 state 0x0 ref 1 00000100:00100000:3.0:1789760720.968681:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2fbb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512576384:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760720.968758:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22953984, mtime: 0x6aad94d0, atime: 0x0, ctime: 0x6aad94d0, blocks: 0xb007 00010000:00010000:3.0:1789760720.968767:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2da66 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.968782:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=700s v=(0 0) 00010000:00010000:3.0:1789760720.968787:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760720.968793:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2da66 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22953984->22958079] (req 22953984->22958079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.968807:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2da66 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22953984->22958079] (req 22953984->22958079) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760721.043325:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972923 is committed 00002000:00010000:3.0:1789760721.043613:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22953984 -> 22958080 00010000:00010000:3.0:1789760721.043631:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2da66 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22953984->22958079] (req 22953984->22958079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.043643:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2da66 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22953984->22958079] (req 22953984->22958079) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.043652:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2da66 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22953984->22958079] (req 22953984->22958079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.043659:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760721.043663:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2da66 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22953984->22958079] (req 22953984->22958079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.043672:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2da66 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22953984->22958079] (req 22953984->22958079) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.043691:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2da66 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22953984->22958079] (req 22953984->22958079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.043716:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2da66 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22953984->22958079] (req 22953984->22958079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760721.043800:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2fbb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512576384:12345-192.168.203.56@tcp:4:dd.0 Request processed in 75118us (75476us total) trans 4294972923 rc 0/0 00000100:00100000:3.0:1789760721.043813:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5634 00000100:00100000:3.0:1789760721.043822:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 700428539406 time 700532968452 state 0x2 ref 0 00000100:00100000:1.0:1789760721.048194:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760721.048469:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512576512 00000100:00100000:2.0:1789760721.048479:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512576512 00000100:00100000:2.0:1789760721.048564:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 700428539406 time 700532968452 state 0x0 ref 1 00000100:00100000:2.0:1789760721.048579:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 700533228868 time 700532968452 state 0x1 ref 1 00000100:00100000:2.0:1789760721.048607:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 700533228868 time 700612907467 state 0x0 ref 1 00000100:00100000:2.0:1789760721.048620:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4131f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512576512:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760721.048667:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22958080, mtime: 0x6aad94d0, atime: 0x0, ctime: 0x6aad94d0, blocks: 0xb007 00010000:00010000:2.0:1789760721.048676:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2da6d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1789760721.048873:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:2.0:1789760721.048964:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=700s v=(0 0) 00010000:00010000:2.0:1789760721.048968:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760721.048974:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2da6d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22958080->22962175] (req 22958080->22962175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760721.049000:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2da6d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22958080->22962175] (req 22958080->22962175) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760721.132391:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972924 is committed 00002000:00010000:2.0:1789760721.132506:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22958080 -> 22962176 00010000:00010000:2.0:1789760721.132521:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2da6d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22958080->22962175] (req 22958080->22962175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760721.132534:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2da6d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22958080->22962175] (req 22958080->22962175) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760721.132544:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2da6d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22958080->22962175] (req 22958080->22962175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760721.132551:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760721.132556:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2da6d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22958080->22962175] (req 22958080->22962175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760721.132566:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2da6d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22958080->22962175] (req 22958080->22962175) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760721.132648:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2da6d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22958080->22962175] (req 22958080->22962175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760721.132657:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2da6d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22958080->22962175] (req 22958080->22962175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760721.132821:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4131f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512576512:12345-192.168.203.56@tcp:4:dd.0 Request processed in 84199us (84631us total) trans 4294972924 rc 0/0 00000100:00100000:2.0:1789760721.132831:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5635 00000100:00100000:2.0:1789760721.132841:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 700533228868 time 700612907467 state 0x2 ref 0 00000100:00100000:0.0:1789760721.147815:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760721.147919:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512576640 00000100:00100000:2.0:1789760721.147930:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512576640 00000100:00100000:2.0:1789760721.147983:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 700533228868 time 700612907467 state 0x0 ref 1 00000100:00100000:2.0:1789760721.148010:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 700613167883 time 700612907467 state 0x1 ref 1 00000100:00100000:2.0:1789760721.148039:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 700613167883 time 700712339244 state 0x0 ref 1 00000100:00100000:2.0:1789760721.148051:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2fb8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512576640:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760721.148097:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22962176, mtime: 0x6aad94d0, atime: 0x0, ctime: 0x6aad94d0, blocks: 0xb007 00010000:00010000:2.0:1789760721.148106:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2da74 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760721.148122:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=700s v=(0 0) 00010000:00010000:2.0:1789760721.148127:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760721.148134:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2da74 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22962176->22966271] (req 22962176->22966271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760721.148153:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2da74 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22962176->22966271] (req 22962176->22966271) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760721.212928:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22962176 -> 22966272 00010000:00010000:0.0:1789760721.212952:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2da74 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22962176->22966271] (req 22962176->22966271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760721.212967:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2da74 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22962176->22966271] (req 22962176->22966271) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760721.212976:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2da74 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22962176->22966271] (req 22962176->22966271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760721.212983:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760721.212988:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2da74 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22962176->22966271] (req 22962176->22966271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760721.212997:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2da74 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22962176->22966271] (req 22962176->22966271) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760721.213020:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2da74 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22962176->22966271] (req 22962176->22966271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760721.213029:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2da74 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22962176->22966271] (req 22962176->22966271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760721.213124:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2fb8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2359:x1876699512576640:12345-192.168.203.56@tcp:4:dd.0 Request processed in 65069us (65313us total) trans 4294972925 rc 0/0 00000100:00100000:0.0:1789760721.213134:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5636 00000100:00100000:0.0:1789760721.213143:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 700613167883 time 700712339244 state 0x2 ref 0 00000001:00080000:1.0:1789760721.213592:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972925 is committed 00000100:00100000:0.0:1789760721.220986:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760721.221075:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512576768 00000100:00100000:3.0:1789760721.221085:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512576768 00000100:00100000:3.0:1789760721.221139:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 700613167883 time 700712339244 state 0x0 ref 1 00000100:00100000:3.0:1789760721.221155:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 700712599660 time 700712339244 state 0x1 ref 1 00000100:00100000:3.0:1789760721.221361:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 700712599660 time 700785659327 state 0x0 ref 1 00000100:00100000:3.0:1789760721.221375:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2fb9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512576768:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760721.221426:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22966272, mtime: 0x6aad94d0, atime: 0x0, ctime: 0x6aad94d0, blocks: 0xb007 00010000:00010000:3.0:1789760721.221435:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2da7b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.221450:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=700s v=(0 0) 00010000:00010000:3.0:1789760721.221454:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760721.221459:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2da7b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22966272->22970367] (req 22966272->22970367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.221471:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2da7b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22966272->22970367] (req 22966272->22970367) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760721.267790:0:5835:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:1.0:1789760721.315382:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972926 is committed 00002000:00010000:0.0:1789760721.315494:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22966272 -> 22970368 00010000:00010000:0.0:1789760721.315511:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2da7b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22966272->22970367] (req 22966272->22970367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760721.315524:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2da7b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22966272->22970367] (req 22966272->22970367) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760721.315534:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2da7b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22966272->22970367] (req 22966272->22970367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760721.315542:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760721.315556:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2da7b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22966272->22970367] (req 22966272->22970367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760721.315568:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2da7b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22966272->22970367] (req 22966272->22970367) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760721.315587:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2da7b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22966272->22970367] (req 22966272->22970367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760721.315595:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2da7b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22966272->22970367] (req 22966272->22970367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760721.316029:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2fb9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512576768:12345-192.168.203.56@tcp:4:dd.0 Request processed in 94650us (95045us total) trans 4294972926 rc 0/0 00000100:00100000:0.0:1789760721.316043:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5637 00000100:00100000:0.0:1789760721.316054:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 700712599660 time 700785659327 state 0x2 ref 0 00000100:00100000:2.0:1789760721.319536:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760721.319783:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512576896 00000100:00100000:0.0:1789760721.319794:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512576896 00000100:00100000:0.0:1789760721.319851:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 700712599660 time 700785659327 state 0x0 ref 1 00000100:00100000:0.0:1789760721.319865:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 700785919743 time 700785659327 state 0x1 ref 1 00000100:00100000:0.0:1789760721.319988:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 700785919743 time 700884286896 state 0x0 ref 1 00000100:00100000:0.0:1789760721.320002:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5306680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512576896:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760721.320050:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22970368, mtime: 0x6aad94d0, atime: 0x0, ctime: 0x6aad94d0, blocks: 0xb007 00010000:00010000:0.0:1789760721.320060:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2da82 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760721.320076:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=700s v=(0 0) 00010000:00010000:0.0:1789760721.320081:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760721.320087:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2da82 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22970368->22974463] (req 22970368->22974463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760721.320102:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2da82 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22970368->22974463] (req 22970368->22974463) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760721.331087:0:3309:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8feac2fbaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3309:1876699540220416:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1789760721.331170:0:3309:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760721.331254:0:6706:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540220416 00000100:00100000:0.0:1789760721.331262:0:6706:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540220416 00000100:00100000:0.0:1789760721.331303:0:6706:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 189 00000100:00100000:0.0:1789760721.331312:0:6706:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4362d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3309:x1876699540220416:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1789760721.331350:0:6706:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3859808256 avail 3857711104 00000100:00100000:0.0:1789760721.331394:0:6706:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4362d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3309:x1876699540220416:12345-0@lo:13:osp-pre-1-0.0 Request processed in 82us (223us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760721.331401:0:6706:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 189 00000100:00100000:0.0:1789760721.331443:0:3309:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8feac2fbaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3309:1876699540220416:0@lo:13:osp-pre-1-0.0 00000001:00080000:2.0:1789760721.388168:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972927 is committed 00002000:00010000:0.0:1789760721.388402:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22970368 -> 22974464 00010000:00010000:0.0:1789760721.388419:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2da82 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22970368->22974463] (req 22970368->22974463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760721.388431:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2da82 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22970368->22974463] (req 22970368->22974463) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760721.388445:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2da82 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22970368->22974463] (req 22970368->22974463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760721.388453:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760721.388457:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2da82 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22970368->22974463] (req 22970368->22974463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760721.388466:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2da82 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22970368->22974463] (req 22970368->22974463) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760721.388495:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2da82 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22970368->22974463] (req 22970368->22974463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760721.388512:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2da82 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22970368->22974463] (req 22970368->22974463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760721.388616:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5306680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512576896:12345-192.168.203.56@tcp:4:dd.0 Request processed in 68611us (69084us total) trans 4294972927 rc 0/0 00000100:00100000:0.0:1789760721.388627:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5638 00000100:00100000:0.0:1789760721.388640:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 700785919743 time 700884286896 state 0x2 ref 0 00000100:00100000:1.0:1789760721.398939:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760721.405434:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512577024 00000100:00100000:0.0:1789760721.405456:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512577024 00000100:00100000:0.0:1789760721.405558:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 700785919743 time 700884286896 state 0x0 ref 1 00000100:00100000:0.0:1789760721.408609:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 700884547312 time 700884286896 state 0x1 ref 1 00000100:00100000:0.0:1789760721.408664:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 700884547312 time 700972964253 state 0x0 ref 1 00000100:00100000:0.0:1789760721.408721:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4130380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512577024:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760721.408829:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22974464, mtime: 0x6aad94d0, atime: 0x0, ctime: 0x6aad94d0, blocks: 0xb007 00010000:00010000:0.0:1789760721.408840:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2da89 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760721.408861:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=700s v=(0 0) 00010000:00010000:0.0:1789760721.408866:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760721.408873:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2da89 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22974464->22978559] (req 22974464->22978559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760721.408889:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2da89 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22974464->22978559] (req 22974464->22978559) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760721.480750:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972928 is committed 00002000:00010000:0.0:1789760721.480883:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22974464 -> 22978560 00010000:00010000:0.0:1789760721.480899:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2da89 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22974464->22978559] (req 22974464->22978559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760721.480911:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2da89 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22974464->22978559] (req 22974464->22978559) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760721.480920:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2da89 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22974464->22978559] (req 22974464->22978559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760721.480928:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760721.480933:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2da89 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22974464->22978559] (req 22974464->22978559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760721.480944:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2da89 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22974464->22978559] (req 22974464->22978559) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760721.480966:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2da89 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22974464->22978559] (req 22974464->22978559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760721.480974:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2da89 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22974464->22978559] (req 22974464->22978559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760721.481105:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4130380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512577024:12345-192.168.203.56@tcp:4:dd.0 Request processed in 72422us (82177us total) trans 4294972928 rc 0/0 00000100:00100000:0.0:1789760721.481116:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5639 00000100:00100000:0.0:1789760721.481127:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 700884547312 time 700972964253 state 0x2 ref 0 00000100:00100000:2.0:1789760721.486345:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760721.486399:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512577152 00000100:00100000:0.0:1789760721.486410:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512577152 00000100:00100000:0.0:1789760721.486496:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 700884547312 time 700972964253 state 0x0 ref 1 00000100:00100000:0.0:1789760721.486514:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 700973224669 time 700972964253 state 0x1 ref 1 00000100:00100000:0.0:1789760721.486544:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 700973224669 time 701050843469 state 0x0 ref 1 00000100:00100000:0.0:1789760721.486556:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5307b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512577152:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760721.486603:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22978560, mtime: 0x6aad94d0, atime: 0x0, ctime: 0x6aad94d0, blocks: 0xb007 00010000:00010000:0.0:1789760721.486612:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2da90 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760721.486628:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=701s v=(0 0) 00010000:00010000:0.0:1789760721.486633:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760721.486639:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2da90 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22978560->22982655] (req 22978560->22982655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760721.486663:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2da90 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22978560->22982655] (req 22978560->22982655) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760721.522907:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22978560 -> 22982656 00010000:00010000:1.0:1789760721.522928:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2da90 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22978560->22982655] (req 22978560->22982655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.522942:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2da90 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22978560->22982655] (req 22978560->22982655) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.522951:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2da90 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22978560->22982655] (req 22978560->22982655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.522959:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760721.522964:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2da90 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22978560->22982655] (req 22978560->22982655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.522974:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2da90 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22978560->22982655] (req 22978560->22982655) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.522994:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2da90 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22978560->22982655] (req 22978560->22982655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.523002:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2da90 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22978560->22982655] (req 22978560->22982655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760721.523068:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972929 is committed 00000100:00100000:1.0:1789760721.523462:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5307b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512577152:12345-192.168.203.56@tcp:4:dd.0 Request processed in 36903us (37120us total) trans 4294972929 rc 0/0 00000100:00100000:1.0:1789760721.523472:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5640 00000100:00100000:1.0:1789760721.523482:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 700973224669 time 701050843469 state 0x2 ref 0 00000100:00100000:2.0:1789760721.526267:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760721.526321:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512577280 00000100:00100000:0.0:1789760721.526330:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512577280 00000100:00100000:0.0:1789760721.526377:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 700973224669 time 701050843469 state 0x0 ref 1 00000100:00100000:0.0:1789760721.526388:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 701051103885 time 701050843469 state 0x1 ref 1 00000100:00100000:0.0:1789760721.526410:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 701051103885 time 701090711426 state 0x0 ref 1 00000100:00100000:0.0:1789760721.526419:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5307100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512577280:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760721.526457:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22982656, mtime: 0x6aad94d0, atime: 0x0, ctime: 0x6aad94d0, blocks: 0xb007 00010000:00010000:0.0:1789760721.526464:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2da97 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760721.526476:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=701s v=(0 0) 00010000:00010000:0.0:1789760721.526479:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760721.526494:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2da97 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22982656->22986751] (req 22982656->22986751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760721.526509:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2da97 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22982656->22986751] (req 22982656->22986751) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760721.561593:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972930 is committed 00002000:00010000:2.0:1789760721.561751:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22982656 -> 22986752 00002000:00010000:2.0:1789760721.561756:0:6709:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760720 -> 1789760721 00002000:00010000:2.0:1789760721.561758:0:6709:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760720 -> 1789760721 00010000:00010000:2.0:1789760721.561772:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2da97 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22982656->22986751] (req 22982656->22986751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760721.561786:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2da97 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22982656->22986751] (req 22982656->22986751) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760721.561796:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2da97 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22982656->22986751] (req 22982656->22986751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760721.561803:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760721.561808:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2da97 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22982656->22986751] (req 22982656->22986751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760721.561820:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2da97 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22982656->22986751] (req 22982656->22986751) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760721.561842:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2da97 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22982656->22986751] (req 22982656->22986751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760721.561850:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2da97 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22982656->22986751] (req 22982656->22986751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760721.561987:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5307100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512577280:12345-192.168.203.56@tcp:4:dd.0 Request processed in 35563us (35725us total) trans 4294972930 rc 0/0 00000100:00100000:2.0:1789760721.561998:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5641 00000100:00100000:2.0:1789760721.562008:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 701051103885 time 701090711426 state 0x2 ref 0 00000100:00100000:0.0:1789760721.564864:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760721.564957:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512577408 00000100:00100000:2.0:1789760721.564965:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512577408 00000100:00100000:2.0:1789760721.565007:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 701051103885 time 701090711426 state 0x0 ref 1 00000100:00100000:2.0:1789760721.565018:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 701090971842 time 701090711426 state 0x1 ref 1 00000100:00100000:2.0:1789760721.565073:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 701090971842 time 701129373438 state 0x0 ref 1 00000100:00100000:2.0:1789760721.565082:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2fbaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512577408:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760721.565130:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22986752, mtime: 0x6aad94d1, atime: 0x0, ctime: 0x6aad94d1, blocks: 0xb007 00010000:00010000:2.0:1789760721.565137:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2da9e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760721.565150:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=701s v=(0 0) 00010000:00010000:2.0:1789760721.565154:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760721.565159:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2da9e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22986752->22990847] (req 22986752->22990847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760721.565168:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2da9e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22986752->22990847] (req 22986752->22990847) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760721.608854:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972931 is committed 00002000:00010000:2.0:1789760721.609060:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22986752 -> 22990848 00010000:00010000:2.0:1789760721.609076:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2da9e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22986752->22990847] (req 22986752->22990847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760721.609089:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2da9e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22986752->22990847] (req 22986752->22990847) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760721.609097:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2da9e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22986752->22990847] (req 22986752->22990847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760721.609105:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760721.609111:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2da9e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22986752->22990847] (req 22986752->22990847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760721.609121:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2da9e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22986752->22990847] (req 22986752->22990847) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760721.609151:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2da9e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22986752->22990847] (req 22986752->22990847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760721.609274:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2da9e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22986752->22990847] (req 22986752->22990847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760721.609369:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2fbaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512577408:12345-192.168.203.56@tcp:4:dd.0 Request processed in 44282us (44508us total) trans 4294972931 rc 0/0 00000100:00100000:2.0:1789760721.609379:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5642 00000100:00100000:2.0:1789760721.609388:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 701090971842 time 701129373438 state 0x2 ref 0 00000100:00100000:0.0:1789760721.614791:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760721.614920:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512577536 00000100:00100000:1.0:1789760721.614931:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512577536 00000100:00100000:1.0:1789760721.614985:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 701090971842 time 701129373438 state 0x0 ref 1 00000100:00100000:1.0:1789760721.615000:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 701129633854 time 701129373438 state 0x1 ref 1 00000100:00100000:1.0:1789760721.615034:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 701129633854 time 701179333673 state 0x0 ref 1 00000100:00100000:1.0:1789760721.615047:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac28a1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512577536:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760721.615097:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22990848, mtime: 0x6aad94d1, atime: 0x0, ctime: 0x6aad94d1, blocks: 0xb007 00010000:00010000:1.0:1789760721.615107:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2daa5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.615133:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=701s v=(0 0) 00010000:00010000:1.0:1789760721.615139:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760721.615145:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2daa5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22990848->22994943] (req 22990848->22994943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.615318:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2daa5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22990848->22994943] (req 22990848->22994943) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760721.656543:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972932 is committed 00002000:00010000:0.0:1789760721.656645:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22990848 -> 22994944 00010000:00010000:0.0:1789760721.656662:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2daa5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22990848->22994943] (req 22990848->22994943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760721.656675:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2daa5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22990848->22994943] (req 22990848->22994943) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760721.656684:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2daa5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22990848->22994943] (req 22990848->22994943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760721.656692:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760721.656723:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2daa5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22990848->22994943] (req 22990848->22994943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760721.656733:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2daa5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22990848->22994943] (req 22990848->22994943) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760721.656753:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2daa5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22990848->22994943] (req 22990848->22994943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760721.656766:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2daa5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22990848->22994943] (req 22990848->22994943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760721.656852:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac28a1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512577536:12345-192.168.203.56@tcp:4:dd.0 Request processed in 41801us (42067us total) trans 4294972932 rc 0/0 00000100:00100000:0.0:1789760721.656862:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5643 00000100:00100000:0.0:1789760721.656871:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 701129633854 time 701179333673 state 0x2 ref 0 00000100:00100000:1.0:1789760721.661481:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760721.661574:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512577664 00000100:00100000:2.0:1789760721.661584:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512577664 00000100:00100000:2.0:1789760721.661641:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 701129633854 time 701179333673 state 0x0 ref 1 00000100:00100000:2.0:1789760721.661657:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 701179594089 time 701179333673 state 0x1 ref 1 00000100:00100000:2.0:1789760721.662282:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 701179594089 time 701225993255 state 0x0 ref 1 00000100:00100000:2.0:1789760721.662305:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4133b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512577664:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760721.662361:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22994944, mtime: 0x6aad94d1, atime: 0x0, ctime: 0x6aad94d1, blocks: 0xb007 00010000:00010000:2.0:1789760721.662369:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2daac lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760721.662385:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=701s v=(0 0) 00010000:00010000:2.0:1789760721.662389:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760721.662395:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2daac lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22994944->22999039] (req 22994944->22999039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760721.663283:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2daac lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22994944->22999039] (req 22994944->22999039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760721.713502:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972933 is committed 00002000:00010000:0.0:1789760721.713595:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22994944 -> 22999040 00010000:00010000:0.0:1789760721.713611:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2daac lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22994944->22999039] (req 22994944->22999039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760721.713623:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2daac lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22994944->22999039] (req 22994944->22999039) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760721.713632:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2daac lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22994944->22999039] (req 22994944->22999039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760721.713639:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760721.713644:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2daac lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22994944->22999039] (req 22994944->22999039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760721.713652:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2daac lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22994944->22999039] (req 22994944->22999039) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760721.713674:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2daac lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22994944->22999039] (req 22994944->22999039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760721.713682:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2daac lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22994944->22999039] (req 22994944->22999039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760721.713843:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4133b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512577664:12345-192.168.203.56@tcp:4:dd.0 Request processed in 51540us (52365us total) trans 4294972933 rc 0/0 00000100:00100000:0.0:1789760721.713853:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5644 00000100:00100000:0.0:1789760721.713863:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 701179594089 time 701225993255 state 0x2 ref 0 00000100:00100000:2.0:1789760721.723311:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760721.726274:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512577792 00000100:00100000:0.0:1789760721.726291:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512577792 00000100:00100000:1.0:1789760721.726672:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 701179594089 time 701225993255 state 0x0 ref 1 00000100:00100000:1.0:1789760721.726690:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 701226253671 time 701225993255 state 0x1 ref 1 00000100:00100000:1.0:1789760721.727025:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 701226253671 time 701291322679 state 0x0 ref 1 00000100:00100000:1.0:1789760721.727040:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5305f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512577792:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760721.727093:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22999040, mtime: 0x6aad94d1, atime: 0x0, ctime: 0x6aad94d1, blocks: 0xb007 00010000:00010000:1.0:1789760721.727102:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2dab3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.727118:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=701s v=(0 0) 00010000:00010000:1.0:1789760721.727123:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760721.727129:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2dab3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22999040->23003135] (req 22999040->23003135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.727143:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2dab3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22999040->23003135] (req 22999040->23003135) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760721.766275:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972934 is committed 00002000:00010000:2.0:1789760721.766342:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22999040 -> 23003136 00010000:00010000:2.0:1789760721.766359:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2dab3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22999040->23003135] (req 22999040->23003135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760721.766372:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2dab3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22999040->23003135] (req 22999040->23003135) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760721.766382:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2dab3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22999040->23003135] (req 22999040->23003135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760721.766389:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760721.766394:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2dab3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22999040->23003135] (req 22999040->23003135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760721.766402:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2dab3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22999040->23003135] (req 22999040->23003135) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760721.766420:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2dab3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22999040->23003135] (req 22999040->23003135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760721.766427:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2dab3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22999040->23003135] (req 22999040->23003135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760721.766528:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5305f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512577792:12345-192.168.203.56@tcp:4:dd.0 Request processed in 39487us (43222us total) trans 4294972934 rc 0/0 00000100:00100000:2.0:1789760721.766537:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5645 00000100:00100000:2.0:1789760721.766547:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 701226253671 time 701291322679 state 0x2 ref 0 00000100:00100000:2.0:1789760721.768968:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760721.769079:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512577920 00000100:00100000:0.0:1789760721.769089:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512577920 00000100:00100000:0.0:1789760721.769141:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 701226253671 time 701291322679 state 0x0 ref 1 00000100:00100000:0.0:1789760721.769166:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 701291583095 time 701291322679 state 0x1 ref 1 00000100:00100000:0.0:1789760721.769193:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 701291583095 time 701333492938 state 0x0 ref 1 00000100:00100000:0.0:1789760721.769204:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5307480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512577920:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760721.769256:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23003136, mtime: 0x6aad94d1, atime: 0x0, ctime: 0x6aad94d1, blocks: 0xb007 00010000:00010000:0.0:1789760721.769264:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2daba lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760721.769284:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=701s v=(0 0) 00010000:00010000:0.0:1789760721.769287:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760721.769294:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2daba lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23003136->23007231] (req 23003136->23007231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760721.769307:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2daba lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23003136->23007231] (req 23003136->23007231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760721.800647:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972935 is committed 00002000:00010000:0.0:1789760721.804844:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23003136 -> 23007232 00010000:00010000:0.0:1789760721.804864:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2daba lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23003136->23007231] (req 23003136->23007231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760721.804879:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2daba lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23003136->23007231] (req 23003136->23007231) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760721.804888:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2daba lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23003136->23007231] (req 23003136->23007231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760721.804896:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760721.804901:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2daba lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23003136->23007231] (req 23003136->23007231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760721.804910:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2daba lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23003136->23007231] (req 23003136->23007231) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760721.804930:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2daba lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23003136->23007231] (req 23003136->23007231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760721.804939:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2daba lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23003136->23007231] (req 23003136->23007231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760721.805142:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5307480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512577920:12345-192.168.203.56@tcp:4:dd.0 Request processed in 35935us (36178us total) trans 4294972935 rc 0/0 00000100:00100000:0.0:1789760721.805161:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5646 00000100:00100000:0.0:1789760721.805170:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 701291583095 time 701333492938 state 0x2 ref 0 00000100:00100000:1.0:1789760721.820866:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760721.820988:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512578048 00000100:00100000:0.0:1789760721.820998:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512578048 00000100:00100000:0.0:1789760721.821060:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 701291583095 time 701333492938 state 0x0 ref 1 00000100:00100000:0.0:1789760721.821076:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 701333753354 time 701333492938 state 0x1 ref 1 00000100:00100000:0.0:1789760721.821108:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 701333753354 time 701385407792 state 0x0 ref 1 00000100:00100000:0.0:1789760721.821121:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4133800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512578048:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760721.821199:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23007232, mtime: 0x6aad94d1, atime: 0x0, ctime: 0x6aad94d1, blocks: 0xb007 00010000:00010000:0.0:1789760721.821212:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2dac1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760721.821229:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=701s v=(0 0) 00010000:00010000:0.0:1789760721.821234:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760721.821240:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2dac1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23007232->23011327] (req 23007232->23011327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760721.821259:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2dac1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23007232->23011327] (req 23007232->23011327) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760721.871269:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972936 is committed 00002000:00010000:2.0:1789760721.871489:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23007232 -> 23011328 00010000:00010000:2.0:1789760721.871504:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2dac1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23007232->23011327] (req 23007232->23011327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760721.871516:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2dac1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23007232->23011327] (req 23007232->23011327) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760721.871524:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2dac1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23007232->23011327] (req 23007232->23011327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760721.871530:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760721.871535:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2dac1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23007232->23011327] (req 23007232->23011327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760721.871543:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2dac1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23007232->23011327] (req 23007232->23011327) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760721.871560:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2dac1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23007232->23011327] (req 23007232->23011327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760721.871571:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2dac1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23007232->23011327] (req 23007232->23011327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760721.871684:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4133800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512578048:12345-192.168.203.56@tcp:4:dd.0 Request processed in 50560us (50829us total) trans 4294972936 rc 0/0 00000100:00100000:2.0:1789760721.871727:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5647 00000100:00100000:2.0:1789760721.871737:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 701333753354 time 701385407792 state 0x2 ref 0 00000100:00100000:2.0:1789760721.876601:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760721.876829:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512578176 00000100:00100000:1.0:1789760721.876839:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512578176 00000100:00100000:1.0:1789760721.876895:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 701333753354 time 701385407792 state 0x0 ref 1 00000100:00100000:1.0:1789760721.876917:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 701385668208 time 701385407792 state 0x1 ref 1 00000100:00100000:1.0:1789760721.876947:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 701385668208 time 701441246935 state 0x0 ref 1 00000100:00100000:1.0:1789760721.876960:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5304700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512578176:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760721.877019:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23011328, mtime: 0x6aad94d1, atime: 0x0, ctime: 0x6aad94d1, blocks: 0xb007 00010000:00010000:1.0:1789760721.877029:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2dac8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.877045:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=701s v=(0 0) 00010000:00010000:1.0:1789760721.877049:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760721.877064:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2dac8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23011328->23015423] (req 23011328->23015423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.877079:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2dac8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23011328->23015423] (req 23011328->23015423) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760721.946126:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972937 is committed 00002000:00010000:3.0:1789760721.946343:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23011328 -> 23015424 00010000:00010000:3.0:1789760721.946361:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2dac8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23011328->23015423] (req 23011328->23015423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.946374:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2dac8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23011328->23015423] (req 23011328->23015423) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.946382:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2dac8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23011328->23015423] (req 23011328->23015423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.946390:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760721.946396:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2dac8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23011328->23015423] (req 23011328->23015423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.946405:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2dac8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23011328->23015423] (req 23011328->23015423) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.946423:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2dac8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23011328->23015423] (req 23011328->23015423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.946431:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2dac8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23011328->23015423] (req 23011328->23015423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760721.946561:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5304700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512578176:12345-192.168.203.56@tcp:4:dd.0 Request processed in 69600us (69965us total) trans 4294972937 rc 0/0 00000100:00100000:3.0:1789760721.946573:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5648 00000100:00100000:3.0:1789760721.946583:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 701385668208 time 701441246935 state 0x2 ref 0 00000100:00100000:2.0:1789760721.953571:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760721.953654:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512578304 00000100:00100000:0.0:1789760721.953665:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512578304 00000100:00100000:0.0:1789760721.953746:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 701385668208 time 701441246935 state 0x0 ref 1 00000100:00100000:0.0:1789760721.953761:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 701441507351 time 701441246935 state 0x1 ref 1 00000100:00100000:0.0:1789760721.953788:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 701441507351 time 701518088174 state 0x0 ref 1 00000100:00100000:0.0:1789760721.953801:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5306a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512578304:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760721.953852:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23015424, mtime: 0x6aad94d1, atime: 0x0, ctime: 0x6aad94d1, blocks: 0xb007 00010000:00010000:0.0:1789760721.953862:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2dacf lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760721.953878:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=701s v=(0 0) 00010000:00010000:0.0:1789760721.953883:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760721.953890:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2dacf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23015424->23019519] (req 23015424->23019519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760721.953905:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2dacf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23015424->23019519] (req 23015424->23019519) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760721.987865:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972938 is committed 00002000:00010000:1.0:1789760721.988371:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23015424 -> 23019520 00010000:00010000:1.0:1789760721.988387:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2dacf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23015424->23019519] (req 23015424->23019519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.988398:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2dacf lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23015424->23019519] (req 23015424->23019519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.988407:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2dacf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23015424->23019519] (req 23015424->23019519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.988413:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760721.988418:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2dacf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23015424->23019519] (req 23015424->23019519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.988426:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2dacf lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23015424->23019519] (req 23015424->23019519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.988443:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2dacf lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23015424->23019519] (req 23015424->23019519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.988451:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2dacf lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23015424->23019519] (req 23015424->23019519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760721.988585:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5306a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512578304:12345-192.168.203.56@tcp:4:dd.0 Request processed in 34782us (35019us total) trans 4294972938 rc 0/0 00000100:00100000:1.0:1789760721.988595:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5649 00000100:00100000:1.0:1789760721.988605:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 701441507351 time 701518088174 state 0x2 ref 0 00000100:00100000:0.0:1789760722.001169:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760722.003132:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512578432 00000100:00100000:2.0:1789760722.003146:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512578432 00000100:00100000:2.0:1789760722.003207:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 701441507351 time 701518088174 state 0x0 ref 1 00000100:00100000:2.0:1789760722.003222:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 701518348590 time 701518088174 state 0x1 ref 1 00000100:00100000:2.0:1789760722.003309:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 701518348590 time 701567608140 state 0x0 ref 1 00000100:00100000:2.0:1789760722.003324:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac28a2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512578432:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760722.003373:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23019520, mtime: 0x6aad94d1, atime: 0x0, ctime: 0x6aad94d1, blocks: 0xb007 00010000:00010000:2.0:1789760722.003382:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2dad6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.003396:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=701s v=(0 0) 00010000:00010000:2.0:1789760722.003400:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760722.003407:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2dad6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23019520->23023615] (req 23019520->23023615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.003421:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2dad6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23019520->23023615] (req 23019520->23023615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760722.044841:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972939 is committed 00002000:00010000:0.0:1789760722.045021:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23019520 -> 23023616 00010000:00010000:0.0:1789760722.045036:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2dad6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23019520->23023615] (req 23019520->23023615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.045047:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2dad6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23019520->23023615] (req 23019520->23023615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.045128:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2dad6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23019520->23023615] (req 23019520->23023615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.045136:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760722.045141:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2dad6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23019520->23023615] (req 23019520->23023615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.045204:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2dad6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23019520->23023615] (req 23019520->23023615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.045225:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2dad6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23019520->23023615] (req 23019520->23023615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.045233:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2dad6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23019520->23023615] (req 23019520->23023615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760722.045362:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac28a2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512578432:12345-192.168.203.56@tcp:4:dd.0 Request processed in 42036us (44198us total) trans 4294972939 rc 0/0 00000100:00100000:0.0:1789760722.045372:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5650 00000100:00100000:0.0:1789760722.045382:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 701518348590 time 701567608140 state 0x2 ref 0 00000100:00100000:2.0:1789760722.048147:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760722.048404:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512578560 00000100:00100000:3.0:1789760722.048412:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512578560 00000100:00100000:3.0:1789760722.048466:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 701518348590 time 701567608140 state 0x0 ref 1 00000100:00100000:3.0:1789760722.048492:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 701567868556 time 701567608140 state 0x1 ref 1 00000100:00100000:3.0:1789760722.048577:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 701567868556 time 701612877068 state 0x0 ref 1 00000100:00100000:3.0:1789760722.048590:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febfbeae300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512578560:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760722.048636:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23023616, mtime: 0x6aad94d1, atime: 0x0, ctime: 0x6aad94d1, blocks: 0xb007 00010000:00010000:3.0:1789760722.048644:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2dadd lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.048659:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=701s v=(0 0) 00010000:00010000:3.0:1789760722.048663:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760722.048669:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2dadd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23023616->23027711] (req 23023616->23027711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.048682:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2dadd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23023616->23027711] (req 23023616->23027711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760722.081388:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972940 is committed 00002000:00010000:0.0:1789760722.081471:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23023616 -> 23027712 00010000:00010000:0.0:1789760722.081496:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2dadd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23023616->23027711] (req 23023616->23027711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.081509:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2dadd lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23023616->23027711] (req 23023616->23027711) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.081518:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2dadd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23023616->23027711] (req 23023616->23027711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.081524:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760722.081529:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2dadd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23023616->23027711] (req 23023616->23027711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.081537:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2dadd lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23023616->23027711] (req 23023616->23027711) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.081554:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2dadd lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23023616->23027711] (req 23023616->23027711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.081560:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2dadd lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23023616->23027711] (req 23023616->23027711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760722.081891:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febfbeae300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512578560:12345-192.168.203.56@tcp:4:dd.0 Request processed in 33298us (33749us total) trans 4294972940 rc 0/0 00000100:00100000:0.0:1789760722.081905:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5651 00000100:00100000:0.0:1789760722.081914:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 701567868556 time 701612877068 state 0x2 ref 0 00000100:00100000:2.0:1789760722.084673:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760722.084872:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512578688 00000100:00100000:1.0:1789760722.084882:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512578688 00000100:00100000:1.0:1789760722.084938:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 701567868556 time 701612877068 state 0x0 ref 1 00000100:00100000:1.0:1789760722.084953:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 701613137484 time 701612877068 state 0x1 ref 1 00000100:00100000:1.0:1789760722.085013:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 701613137484 time 701649312402 state 0x0 ref 1 00000100:00100000:1.0:1789760722.085027:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2875c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512578688:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760722.085077:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23027712, mtime: 0x6aad94d1, atime: 0x0, ctime: 0x6aad94d1, blocks: 0xb007 00010000:00010000:1.0:1789760722.085085:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2dae4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.085102:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=701s v=(0 0) 00010000:00010000:1.0:1789760722.085106:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760722.085113:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2dae4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23027712->23031807] (req 23027712->23031807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.085127:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2dae4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23027712->23031807] (req 23027712->23031807) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760722.113220:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23027712 -> 23031808 00010000:00010000:1.0:1789760722.113242:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2dae4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23027712->23031807] (req 23027712->23031807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.113255:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2dae4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23027712->23031807] (req 23027712->23031807) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.113265:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2dae4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23027712->23031807] (req 23027712->23031807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.113273:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760722.113278:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2dae4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23027712->23031807] (req 23027712->23031807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.113288:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2dae4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23027712->23031807] (req 23027712->23031807) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.113307:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2dae4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23027712->23031807] (req 23027712->23031807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.113315:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2dae4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23027712->23031807] (req 23027712->23031807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760722.113404:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2875c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2359:x1876699512578688:12345-192.168.203.56@tcp:4:dd.0 Request processed in 28375us (28736us total) trans 4294972941 rc 0/0 00000100:00100000:1.0:1789760722.113416:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5652 00000100:00100000:1.0:1789760722.113428:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 701613137484 time 701649312402 state 0x2 ref 0 00000001:00080000:2.0:1789760722.113636:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972941 is committed 00000100:00100000:2.0:1789760722.118954:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760722.119044:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512578816 00000100:00100000:0.0:1789760722.119053:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512578816 00000100:00100000:0.0:1789760722.119105:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 701613137484 time 701649312402 state 0x0 ref 1 00000100:00100000:0.0:1789760722.119120:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 701649572818 time 701649312402 state 0x1 ref 1 00000100:00100000:0.0:1789760722.119180:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 701649572818 time 701683480109 state 0x0 ref 1 00000100:00100000:0.0:1789760722.119194:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2877100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512578816:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760722.119239:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23031808, mtime: 0x6aad94d1, atime: 0x0, ctime: 0x6aad94d1, blocks: 0xb007 00010000:00010000:0.0:1789760722.119247:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2daeb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.119262:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=701s v=(0 0) 00010000:00010000:0.0:1789760722.119266:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760722.119272:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2daeb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23031808->23035903] (req 23031808->23035903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.119284:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2daeb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23031808->23035903] (req 23031808->23035903) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760722.163465:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972942 is committed 00002000:00010000:2.0:1789760722.163639:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23031808 -> 23035904 00010000:00010000:2.0:1789760722.163715:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2daeb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23031808->23035903] (req 23031808->23035903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.163725:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2daeb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23031808->23035903] (req 23031808->23035903) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.163731:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2daeb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23031808->23035903] (req 23031808->23035903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.163735:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760722.163739:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2daeb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23031808->23035903] (req 23031808->23035903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.163745:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2daeb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23031808->23035903] (req 23031808->23035903) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.163762:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2daeb lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23031808->23035903] (req 23031808->23035903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.163767:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2daeb lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23031808->23035903] (req 23031808->23035903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760722.163898:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2877100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512578816:12345-192.168.203.56@tcp:4:dd.0 Request processed in 44702us (44947us total) trans 4294972942 rc 0/0 00000100:00100000:2.0:1789760722.163907:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5653 00000100:00100000:2.0:1789760722.163917:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 701649572818 time 701683480109 state 0x2 ref 0 00000100:00100000:2.0:1789760722.170391:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760722.170459:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512578944 00000100:00100000:0.0:1789760722.170469:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512578944 00000100:00100000:0.0:1789760722.170789:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 701649572818 time 701683480109 state 0x0 ref 1 00000100:00100000:0.0:1789760722.170805:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 701683740525 time 701683480109 state 0x1 ref 1 00000100:00100000:0.0:1789760722.170832:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 701683740525 time 701735132050 state 0x0 ref 1 00000100:00100000:0.0:1789760722.170844:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2877b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512578944:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760722.170889:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23035904, mtime: 0x6aad94d1, atime: 0x0, ctime: 0x6aad94d1, blocks: 0xb007 00010000:00010000:0.0:1789760722.170898:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2daf2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.170913:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=701s v=(0 0) 00010000:00010000:0.0:1789760722.170917:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760722.170923:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2daf2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23035904->23039999] (req 23035904->23039999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.170935:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2daf2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23035904->23039999] (req 23035904->23039999) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760722.203176:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972943 is committed 00002000:00010000:1.0:1789760722.203288:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23035904 -> 23040000 00010000:00010000:1.0:1789760722.203304:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2daf2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23035904->23039999] (req 23035904->23039999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.203318:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2daf2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23035904->23039999] (req 23035904->23039999) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.203327:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2daf2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23035904->23039999] (req 23035904->23039999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.203339:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760722.203343:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2daf2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23035904->23039999] (req 23035904->23039999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.203352:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2daf2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23035904->23039999] (req 23035904->23039999) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.203369:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2daf2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23035904->23039999] (req 23035904->23039999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.203386:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2daf2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23035904->23039999] (req 23035904->23039999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760722.203477:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2877b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512578944:12345-192.168.203.56@tcp:4:dd.0 Request processed in 32630us (33090us total) trans 4294972943 rc 0/0 00000100:00100000:1.0:1789760722.203497:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5654 00000100:00100000:1.0:1789760722.203507:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 701683740525 time 701735132050 state 0x2 ref 0 00000100:00100000:3.0:1789760722.206561:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760722.206753:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512579072 00000100:00100000:0.0:1789760722.206762:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512579072 00000100:00100000:0.0:1789760722.206818:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 701683740525 time 701735132050 state 0x0 ref 1 00000100:00100000:0.0:1789760722.206832:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 701735392466 time 701735132050 state 0x1 ref 1 00000100:00100000:0.0:1789760722.206855:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 701735392466 time 701771156570 state 0x0 ref 1 00000100:00100000:0.0:1789760722.206867:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febe7907100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512579072:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760722.206920:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23040000, mtime: 0x6aad94d1, atime: 0x0, ctime: 0x6aad94d1, blocks: 0xb007 00010000:00010000:0.0:1789760722.206927:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2daf9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.207240:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=701s v=(0 0) 00010000:00010000:0.0:1789760722.207243:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760722.207249:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2daf9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23040000->23044095] (req 23040000->23044095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.207260:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2daf9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23040000->23044095] (req 23040000->23044095) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760722.229232:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972944 is committed 00002000:00010000:0.0:1789760722.229299:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23040000 -> 23044096 00010000:00010000:0.0:1789760722.229314:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2daf9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23040000->23044095] (req 23040000->23044095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.229327:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2daf9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23040000->23044095] (req 23040000->23044095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.229336:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2daf9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23040000->23044095] (req 23040000->23044095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.229343:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760722.229348:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2daf9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23040000->23044095] (req 23040000->23044095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.229358:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2daf9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23040000->23044095] (req 23040000->23044095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.229377:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2daf9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23040000->23044095] (req 23040000->23044095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.229384:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2daf9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23040000->23044095] (req 23040000->23044095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760722.229472:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febe7907100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512579072:12345-192.168.203.56@tcp:4:dd.0 Request processed in 22600us (22918us total) trans 4294972944 rc 0/0 00000100:00100000:0.0:1789760722.229495:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5655 00000100:00100000:0.0:1789760722.229506:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 701735392466 time 701771156570 state 0x2 ref 0 00000100:00100000:0.0:1789760722.240826:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760722.240935:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512579200 00000100:00100000:2.0:1789760722.240945:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512579200 00000100:00100000:2.0:1789760722.240991:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 701735392466 time 701771156570 state 0x0 ref 1 00000100:00100000:2.0:1789760722.241001:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 701771416986 time 701771156570 state 0x1 ref 1 00000100:00100000:2.0:1789760722.241063:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 701771416986 time 701805363006 state 0x0 ref 1 00000100:00100000:2.0:1789760722.241073:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac28a3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512579200:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760722.241114:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23044096, mtime: 0x6aad94d1, atime: 0x0, ctime: 0x6aad94d1, blocks: 0xb007 00010000:00010000:2.0:1789760722.241120:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2db00 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.241136:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=701s v=(0 0) 00010000:00010000:2.0:1789760722.241139:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760722.241144:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2db00 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23044096->23048191] (req 23044096->23048191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.241154:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2db00 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23044096->23048191] (req 23044096->23048191) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760722.269254:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23044096 -> 23048192 00000001:00080000:3.0:1789760722.269271:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972945 is committed 00010000:00010000:0.0:1789760722.269274:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2db00 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23044096->23048191] (req 23044096->23048191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.269286:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2db00 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23044096->23048191] (req 23044096->23048191) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.269295:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2db00 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23044096->23048191] (req 23044096->23048191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.269309:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760722.269313:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2db00 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23044096->23048191] (req 23044096->23048191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.269322:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2db00 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23044096->23048191] (req 23044096->23048191) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.269341:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2db00 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23044096->23048191] (req 23044096->23048191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.269349:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2db00 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23044096->23048191] (req 23044096->23048191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760722.269436:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac28a3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512579200:12345-192.168.203.56@tcp:4:dd.0 Request processed in 28358us (28614us total) trans 4294972945 rc 0/0 00000100:00100000:0.0:1789760722.269446:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5656 00000100:00100000:0.0:1789760722.269455:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 701771416986 time 701805363006 state 0x2 ref 0 00000100:00100000:3.0:1789760722.274071:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760722.274517:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512579328 00000100:00100000:1.0:1789760722.274527:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512579328 00000100:00100000:1.0:1789760722.274576:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 701771416986 time 701805363006 state 0x0 ref 1 00000100:00100000:1.0:1789760722.274588:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 701805623422 time 701805363006 state 0x1 ref 1 00000100:00100000:1.0:1789760722.274613:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 701805623422 time 701838912630 state 0x0 ref 1 00000100:00100000:1.0:1789760722.274628:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2fb6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512579328:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760722.274675:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23048192, mtime: 0x6aad94d1, atime: 0x0, ctime: 0x6aad94d1, blocks: 0xb007 00010000:00010000:1.0:1789760722.274682:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2db07 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.274721:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=701s v=(0 0) 00010000:00010000:1.0:1789760722.274726:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760722.274733:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2db07 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23048192->23052287] (req 23048192->23052287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.274758:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2db07 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23048192->23052287] (req 23048192->23052287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.290789:0:7647:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:3.0:1789760722.295099:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972946 is committed 00002000:00010000:0.0:1789760722.295200:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23048192 -> 23052288 00010000:00010000:0.0:1789760722.295215:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2db07 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23048192->23052287] (req 23048192->23052287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.295228:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2db07 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23048192->23052287] (req 23048192->23052287) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.295359:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2db07 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23048192->23052287] (req 23048192->23052287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.295367:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760722.295372:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2db07 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23048192->23052287] (req 23048192->23052287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.295467:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2db07 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23048192->23052287] (req 23048192->23052287) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.295493:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2db07 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23048192->23052287] (req 23048192->23052287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.295502:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2db07 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23048192->23052287] (req 23048192->23052287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760722.295631:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2fb6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512579328:12345-192.168.203.56@tcp:4:dd.0 Request processed in 21000us (21565us total) trans 4294972946 rc 0/0 00000100:00100000:0.0:1789760722.295641:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5657 00000100:00100000:0.0:1789760722.295650:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 701805623422 time 701838912630 state 0x2 ref 0 00000100:00100000:3.0:1789760722.298514:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760722.298585:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512579456 00000100:00100000:0.0:1789760722.298595:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512579456 00000100:00100000:0.0:1789760722.298643:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 701805623422 time 701838912630 state 0x0 ref 1 00000100:00100000:0.0:1789760722.298656:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 701839173046 time 701838912630 state 0x1 ref 1 00000100:00100000:0.0:1789760722.298685:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 701839173046 time 701862984956 state 0x0 ref 1 00000100:00100000:0.0:1789760722.298730:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febfc0a3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512579456:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760722.298778:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23052288, mtime: 0x6aad94d1, atime: 0x0, ctime: 0x6aad94d1, blocks: 0xb007 00010000:00010000:0.0:1789760722.298787:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2db0e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.298800:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=701s v=(0 0) 00010000:00010000:0.0:1789760722.298805:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760722.298811:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2db0e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23052288->23056383] (req 23052288->23056383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.298824:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2db0e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23052288->23056383] (req 23052288->23056383) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760722.335475:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972947 is committed 00002000:00010000:0.0:1789760722.336106:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23052288 -> 23056384 00010000:00010000:0.0:1789760722.336122:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2db0e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23052288->23056383] (req 23052288->23056383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.336134:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2db0e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23052288->23056383] (req 23052288->23056383) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.336142:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2db0e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23052288->23056383] (req 23052288->23056383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.336265:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760722.336273:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2db0e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23052288->23056383] (req 23052288->23056383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.336283:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2db0e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23052288->23056383] (req 23052288->23056383) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.336302:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2db0e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23052288->23056383] (req 23052288->23056383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.336310:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2db0e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23052288->23056383] (req 23052288->23056383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760722.336440:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febfc0a3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512579456:12345-192.168.203.56@tcp:4:dd.0 Request processed in 37740us (37931us total) trans 4294972947 rc 0/0 00000100:00100000:0.0:1789760722.336451:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5658 00000100:00100000:0.0:1789760722.336460:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 701839173046 time 701862984956 state 0x2 ref 0 00000100:00100000:2.0:1789760722.341942:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760722.342111:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512579584 00000100:00100000:0.0:1789760722.342121:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512579584 00000100:00100000:0.0:1789760722.342187:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 701839173046 time 701862984956 state 0x0 ref 1 00000100:00100000:0.0:1789760722.342202:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 701863245372 time 701862984956 state 0x1 ref 1 00000100:00100000:0.0:1789760722.342231:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 701863245372 time 701906530539 state 0x0 ref 1 00000100:00100000:0.0:1789760722.342243:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2874700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512579584:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760722.342289:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23056384, mtime: 0x6aad94d1, atime: 0x0, ctime: 0x6aad94d1, blocks: 0xb007 00010000:00010000:0.0:1789760722.342299:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2db15 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.342315:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=701s v=(0 0) 00010000:00010000:0.0:1789760722.342320:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760722.342326:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2db15 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23056384->23060479] (req 23056384->23060479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.342340:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2db15 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23056384->23060479] (req 23056384->23060479) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760722.379261:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972948 is committed 00002000:00010000:0.0:1789760722.379418:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23056384 -> 23060480 00010000:00010000:0.0:1789760722.379435:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2db15 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23056384->23060479] (req 23056384->23060479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.379448:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2db15 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23056384->23060479] (req 23056384->23060479) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.379456:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2db15 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23056384->23060479] (req 23056384->23060479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.379463:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760722.379468:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2db15 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23056384->23060479] (req 23056384->23060479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.379489:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2db15 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23056384->23060479] (req 23056384->23060479) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.379516:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2db15 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23056384->23060479] (req 23056384->23060479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.379526:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2db15 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23056384->23060479] (req 23056384->23060479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760722.379637:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2874700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512579584:12345-192.168.203.56@tcp:4:dd.0 Request processed in 37388us (37697us total) trans 4294972948 rc 0/0 00000100:00100000:0.0:1789760722.379648:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5659 00000100:00100000:0.0:1789760722.379661:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 701863245372 time 701906530539 state 0x2 ref 0 00000100:00100000:2.0:1789760722.383379:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760722.383469:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512579712 00000100:00100000:0.0:1789760722.383488:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512579712 00000100:00100000:0.0:1789760722.383540:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 701863245372 time 701906530539 state 0x0 ref 1 00000100:00100000:0.0:1789760722.383557:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 701906790955 time 701906530539 state 0x1 ref 1 00000100:00100000:0.0:1789760722.383621:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 701906790955 time 701947919839 state 0x0 ref 1 00000100:00100000:0.0:1789760722.383633:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2876a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512579712:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760722.383680:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23060480, mtime: 0x6aad94d1, atime: 0x0, ctime: 0x6aad94d1, blocks: 0xb007 00010000:00010000:0.0:1789760722.383689:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2db1c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.383744:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=701s v=(0 0) 00010000:00010000:0.0:1789760722.383750:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760722.383756:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2db1c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23060480->23064575] (req 23060480->23064575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.383771:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2db1c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23060480->23064575] (req 23060480->23064575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760722.415611:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972949 is committed 00002000:00010000:1.0:1789760722.415747:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23060480 -> 23064576 00010000:00010000:1.0:1789760722.415764:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2db1c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23060480->23064575] (req 23060480->23064575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.415775:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2db1c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23060480->23064575] (req 23060480->23064575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.415783:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2db1c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23060480->23064575] (req 23060480->23064575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.415789:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760722.415794:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2db1c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23060480->23064575] (req 23060480->23064575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.415802:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2db1c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23060480->23064575] (req 23060480->23064575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.415820:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2db1c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23060480->23064575] (req 23060480->23064575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.415828:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2db1c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23060480->23064575] (req 23060480->23064575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760722.416033:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2876a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512579712:12345-192.168.203.56@tcp:4:dd.0 Request processed in 32396us (32657us total) trans 4294972949 rc 0/0 00000100:00100000:1.0:1789760722.416043:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5660 00000100:00100000:1.0:1789760722.416052:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 701906790955 time 701947919839 state 0x2 ref 0 00000100:00100000:2.0:1789760722.420400:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760722.424728:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512579840 00000100:00100000:0.0:1789760722.424742:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512579840 00000100:00100000:1.0:1789760722.424888:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 701906790955 time 701947919839 state 0x0 ref 1 00000100:00100000:1.0:1789760722.424905:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 701948180255 time 701947919839 state 0x1 ref 1 00000100:00100000:1.0:1789760722.424929:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 701948180255 time 701989229271 state 0x0 ref 1 00000100:00100000:1.0:1789760722.424942:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2875880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512579840:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760722.424987:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23064576, mtime: 0x6aad94d1, atime: 0x0, ctime: 0x6aad94d1, blocks: 0xb007 00010000:00010000:1.0:1789760722.424996:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2db23 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.425014:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=701s v=(0 0) 00010000:00010000:1.0:1789760722.425018:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760722.425025:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2db23 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23064576->23068671] (req 23064576->23068671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.425037:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2db23 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23064576->23068671] (req 23064576->23068671) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760722.455543:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972950 is committed 00002000:00010000:0.0:1789760722.455731:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23064576 -> 23068672 00010000:00010000:0.0:1789760722.455747:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2db23 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23064576->23068671] (req 23064576->23068671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.455760:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2db23 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23064576->23068671] (req 23064576->23068671) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.455769:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2db23 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23064576->23068671] (req 23064576->23068671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.455777:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760722.455782:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2db23 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23064576->23068671] (req 23064576->23068671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.455791:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2db23 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23064576->23068671] (req 23064576->23068671) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.455812:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2db23 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23064576->23068671] (req 23064576->23068671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.455824:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2db23 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23064576->23068671] (req 23064576->23068671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760722.455958:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2875880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512579840:12345-192.168.203.56@tcp:4:dd.0 Request processed in 31013us (35563us total) trans 4294972950 rc 0/0 00000100:00100000:0.0:1789760722.455968:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5661 00000100:00100000:0.0:1789760722.455978:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 701948180255 time 701989229271 state 0x2 ref 0 00000100:00100000:2.0:1789760722.458609:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760722.458683:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512579968 00000100:00100000:0.0:1789760722.458693:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512579968 00000100:00100000:0.0:1789760722.458780:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 701948180255 time 701989229271 state 0x0 ref 1 00000100:00100000:0.0:1789760722.458800:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 701989489687 time 701989229271 state 0x1 ref 1 00000100:00100000:0.0:1789760722.458832:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 701989489687 time 702023131478 state 0x0 ref 1 00000100:00100000:0.0:1789760722.458845:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febfbddc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512579968:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760722.458887:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23068672, mtime: 0x6aad94d1, atime: 0x0, ctime: 0x6aad94d1, blocks: 0xb007 00010000:00010000:0.0:1789760722.458895:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2db2a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.458909:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=702s v=(0 0) 00010000:00010000:0.0:1789760722.458912:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760722.458916:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2db2a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23068672->23072767] (req 23068672->23072767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.458929:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2db2a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23068672->23072767] (req 23068672->23072767) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760722.498410:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972951 is committed 00002000:00010000:1.0:1789760722.498524:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23068672 -> 23072768 00002000:00010000:1.0:1789760722.498530:0:6709:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb blocks from disk: 45063 -> 47111 00010000:00010000:1.0:1789760722.498552:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2db2a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23068672->23072767] (req 23068672->23072767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.498565:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2db2a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23068672->23072767] (req 23068672->23072767) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.498574:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2db2a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23068672->23072767] (req 23068672->23072767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.498581:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760722.498585:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2db2a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23068672->23072767] (req 23068672->23072767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.498596:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2db2a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23068672->23072767] (req 23068672->23072767) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.498619:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2db2a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23068672->23072767] (req 23068672->23072767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.498625:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2db2a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23068672->23072767] (req 23068672->23072767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760722.498744:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febfbddc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512579968:12345-192.168.203.56@tcp:4:dd.0 Request processed in 39898us (40139us total) trans 4294972951 rc 0/0 00000100:00100000:1.0:1789760722.498753:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5662 00000100:00100000:1.0:1789760722.498761:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 701989489687 time 702023131478 state 0x2 ref 0 00000100:00100000:3.0:1789760722.502402:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760722.502497:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512580096 00000100:00100000:1.0:1789760722.502506:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512580096 00000100:00100000:1.0:1789760722.502630:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 701989489687 time 702023131478 state 0x0 ref 1 00000100:00100000:1.0:1789760722.502645:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 702023391894 time 702023131478 state 0x1 ref 1 00000100:00100000:1.0:1789760722.502675:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 702023391894 time 702066974813 state 0x0 ref 1 00000100:00100000:1.0:1789760722.502687:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febfc0a3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512580096:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760722.502776:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23072768, mtime: 0x6aad94d1, atime: 0x0, ctime: 0x6aad94d1, blocks: 0xb807 00010000:00010000:1.0:1789760722.502785:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2db31 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.502800:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=702s v=(0 0) 00010000:00010000:1.0:1789760722.502804:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760722.502810:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2db31 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23072768->23076863] (req 23072768->23076863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.502824:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2db31 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23072768->23076863] (req 23072768->23076863) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760722.541664:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23072768 -> 23076864 00002000:00010000:1.0:1789760722.541672:0:13063:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760721 -> 1789760722 00002000:00010000:1.0:1789760722.541675:0:13063:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760721 -> 1789760722 00010000:00010000:1.0:1789760722.541688:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2db31 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23072768->23076863] (req 23072768->23076863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.541722:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2db31 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23072768->23076863] (req 23072768->23076863) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.541731:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2db31 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23072768->23076863] (req 23072768->23076863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.541738:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760722.541742:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2db31 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23072768->23076863] (req 23072768->23076863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.541750:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2db31 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23072768->23076863] (req 23072768->23076863) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.541768:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2db31 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23072768->23076863] (req 23072768->23076863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.541775:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2db31 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23072768->23076863] (req 23072768->23076863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760722.541875:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febfc0a3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2358:x1876699512580096:12345-192.168.203.56@tcp:4:dd.0 Request processed in 39179us (39470us total) trans 4294972952 rc 0/0 00000100:00100000:1.0:1789760722.541885:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5663 00000100:00100000:1.0:1789760722.541896:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 702023391894 time 702066974813 state 0x2 ref 0 00000001:00080000:2.0:1789760722.542013:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972952 is committed 00000100:00100000:0.0:1789760722.545475:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760722.545572:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512580224 00000100:00100000:1.0:1789760722.545582:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512580224 00000100:00100000:1.0:1789760722.545632:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 702023391894 time 702066974813 state 0x0 ref 1 00000100:00100000:1.0:1789760722.545648:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 702067235229 time 702066974813 state 0x1 ref 1 00000100:00100000:1.0:1789760722.545678:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 702067235229 time 702109977660 state 0x0 ref 1 00000100:00100000:1.0:1789760722.545690:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac28a0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512580224:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760722.545763:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23076864, mtime: 0x6aad94d2, atime: 0x0, ctime: 0x6aad94d2, blocks: 0xb807 00010000:00010000:1.0:1789760722.545772:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2db38 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.545788:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=702s v=(0 0) 00010000:00010000:1.0:1789760722.545792:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760722.545798:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2db38 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23076864->23080959] (req 23076864->23080959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.545817:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2db38 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23076864->23080959] (req 23076864->23080959) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760722.585433:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23076864 -> 23080960 00010000:00010000:0.0:1789760722.585455:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2db38 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23076864->23080959] (req 23076864->23080959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.585487:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2db38 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23076864->23080959] (req 23076864->23080959) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.585497:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2db38 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23076864->23080959] (req 23076864->23080959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.585505:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760722.585510:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2db38 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23076864->23080959] (req 23076864->23080959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.585519:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2db38 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23076864->23080959] (req 23076864->23080959) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.585666:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2db38 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23076864->23080959] (req 23076864->23080959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.585676:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2db38 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23076864->23080959] (req 23076864->23080959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760722.586639:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac28a0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2359:x1876699512580224:12345-192.168.203.56@tcp:4:dd.0 Request processed in 40168us (40390us total) trans 4294972953 rc 0/0 00000100:00100000:0.0:1789760722.586653:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5664 00000100:00100000:0.0:1789760722.586663:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 702067235229 time 702109977660 state 0x2 ref 0 00000001:00080000:2.0:1789760722.589219:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972953 is committed 00000100:00100000:0.0:1789760722.591788:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760722.592279:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512580352 00000100:00100000:2.0:1789760722.592290:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512580352 00000100:00100000:2.0:1789760722.592353:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 702067235229 time 702109977660 state 0x0 ref 1 00000100:00100000:2.0:1789760722.592371:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 702110238076 time 702109977660 state 0x1 ref 1 00000100:00100000:2.0:1789760722.592457:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 702110238076 time 702156756083 state 0x0 ref 1 00000100:00100000:2.0:1789760722.592489:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febc7b33480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512580352:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760722.592548:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23080960, mtime: 0x6aad94d2, atime: 0x0, ctime: 0x6aad94d2, blocks: 0xb807 00010000:00010000:2.0:1789760722.592559:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2db3f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.592574:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=702s v=(0 0) 00010000:00010000:2.0:1789760722.592579:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760722.592585:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2db3f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23080960->23085055] (req 23080960->23085055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.592599:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2db3f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23080960->23085055] (req 23080960->23085055) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760722.626726:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972954 is committed 00002000:00010000:2.0:1789760722.626860:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23080960 -> 23085056 00010000:00010000:2.0:1789760722.626975:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2db3f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23080960->23085055] (req 23080960->23085055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.626989:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2db3f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23080960->23085055] (req 23080960->23085055) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.626999:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2db3f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23080960->23085055] (req 23080960->23085055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.627007:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760722.627012:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2db3f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23080960->23085055] (req 23080960->23085055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.627021:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2db3f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23080960->23085055] (req 23080960->23085055) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.627046:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2db3f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23080960->23085055] (req 23080960->23085055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.627054:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2db3f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23080960->23085055] (req 23080960->23085055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760722.627276:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febc7b33480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512580352:12345-192.168.203.56@tcp:4:dd.0 Request processed in 34803us (35492us total) trans 4294972954 rc 0/0 00000100:00100000:2.0:1789760722.627286:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5665 00000100:00100000:2.0:1789760722.627296:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 702110238076 time 702156756083 state 0x2 ref 0 00000100:00100000:3.0:1789760722.631218:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760722.631318:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512580480 00000100:00100000:1.0:1789760722.631328:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512580480 00000100:00100000:1.0:1789760722.631383:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 702110238076 time 702156756083 state 0x0 ref 1 00000100:00100000:1.0:1789760722.631400:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 702157016499 time 702156756083 state 0x1 ref 1 00000100:00100000:1.0:1789760722.631432:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 702157016499 time 702195731817 state 0x0 ref 1 00000100:00100000:1.0:1789760722.631442:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac28f0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512580480:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760722.631502:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23085056, mtime: 0x6aad94d2, atime: 0x0, ctime: 0x6aad94d2, blocks: 0xb807 00010000:00010000:1.0:1789760722.631510:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2db46 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.631524:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=702s v=(0 0) 00010000:00010000:1.0:1789760722.631529:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760722.631536:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2db46 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23085056->23089151] (req 23085056->23089151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.631621:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2db46 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23085056->23089151] (req 23085056->23089151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760722.664680:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972955 is committed 00002000:00010000:2.0:1789760722.664802:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23085056 -> 23089152 00010000:00010000:2.0:1789760722.664816:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2db46 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23085056->23089151] (req 23085056->23089151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.664829:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2db46 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23085056->23089151] (req 23085056->23089151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.664838:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2db46 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23085056->23089151] (req 23085056->23089151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.664845:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760722.664849:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2db46 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23085056->23089151] (req 23085056->23089151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.664863:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2db46 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23085056->23089151] (req 23085056->23089151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.664891:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2db46 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23085056->23089151] (req 23085056->23089151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.664899:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2db46 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23085056->23089151] (req 23085056->23089151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760722.664985:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac28f0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512580480:12345-192.168.203.56@tcp:4:dd.0 Request processed in 33539us (33772us total) trans 4294972955 rc 0/0 00000100:00100000:2.0:1789760722.664996:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5666 00000100:00100000:2.0:1789760722.665005:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 702157016499 time 702195731817 state 0x2 ref 0 00000100:00100000:0.0:1789760722.667126:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760722.667188:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512580608 00000100:00100000:1.0:1789760722.667198:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512580608 00000100:00100000:1.0:1789760722.668585:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 702157016499 time 702195731817 state 0x0 ref 1 00000100:00100000:1.0:1789760722.668603:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 702195992233 time 702195731817 state 0x1 ref 1 00000100:00100000:1.0:1789760722.668631:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 702195992233 time 702232931447 state 0x0 ref 1 00000100:00100000:1.0:1789760722.668652:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febc7b32680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512580608:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760722.668717:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23089152, mtime: 0x6aad94d2, atime: 0x0, ctime: 0x6aad94d2, blocks: 0xb807 00010000:00010000:1.0:1789760722.668728:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2db4d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.668745:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=702s v=(0 0) 00010000:00010000:1.0:1789760722.668749:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760722.668755:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2db4d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23089152->23093247] (req 23089152->23093247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.668771:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2db4d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23089152->23093247] (req 23089152->23093247) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760722.701078:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972956 is committed 00002000:00010000:2.0:1789760722.701239:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23089152 -> 23093248 00010000:00010000:2.0:1789760722.701254:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2db4d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23089152->23093247] (req 23089152->23093247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.701267:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2db4d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23089152->23093247] (req 23089152->23093247) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.701276:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2db4d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23089152->23093247] (req 23089152->23093247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.701283:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760722.701288:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2db4d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23089152->23093247] (req 23089152->23093247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.701297:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2db4d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23089152->23093247] (req 23089152->23093247) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.701316:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2db4d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23089152->23093247] (req 23089152->23093247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.701324:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2db4d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23089152->23093247] (req 23089152->23093247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760722.701414:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febc7b32680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512580608:12345-192.168.203.56@tcp:4:dd.0 Request processed in 32767us (34292us total) trans 4294972956 rc 0/0 00000100:00100000:2.0:1789760722.701424:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5667 00000100:00100000:2.0:1789760722.701434:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 702195992233 time 702232931447 state 0x2 ref 0 00000100:00100000:0.0:1789760722.705369:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760722.705452:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512580736 00000100:00100000:2.0:1789760722.705461:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512580736 00000100:00100000:2.0:1789760722.705529:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 702195992233 time 702232931447 state 0x0 ref 1 00000100:00100000:2.0:1789760722.705545:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 702233191863 time 702232931447 state 0x1 ref 1 00000100:00100000:2.0:1789760722.705581:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 702233191863 time 702269880318 state 0x0 ref 1 00000100:00100000:2.0:1789760722.705594:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febffb9f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512580736:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760722.705644:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23093248, mtime: 0x6aad94d2, atime: 0x0, ctime: 0x6aad94d2, blocks: 0xb807 00010000:00010000:2.0:1789760722.705654:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2db54 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.705670:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=702s v=(0 0) 00010000:00010000:2.0:1789760722.705674:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760722.705685:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2db54 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23093248->23097343] (req 23093248->23097343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.705740:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2db54 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23093248->23097343] (req 23093248->23097343) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760722.743605:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972957 is committed 00002000:00010000:2.0:1789760722.743693:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23093248 -> 23097344 00010000:00010000:2.0:1789760722.743729:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2db54 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23093248->23097343] (req 23093248->23097343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.743742:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2db54 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23093248->23097343] (req 23093248->23097343) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.743752:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2db54 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23093248->23097343] (req 23093248->23097343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.743760:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760722.743765:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2db54 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23093248->23097343] (req 23093248->23097343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.743775:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2db54 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23093248->23097343] (req 23093248->23097343) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.743800:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2db54 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23093248->23097343] (req 23093248->23097343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.743810:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2db54 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23093248->23097343] (req 23093248->23097343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760722.743916:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febffb9f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512580736:12345-192.168.203.56@tcp:4:dd.0 Request processed in 38319us (38680us total) trans 4294972957 rc 0/0 00000100:00100000:2.0:1789760722.743928:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5668 00000100:00100000:2.0:1789760722.743938:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 702233191863 time 702269880318 state 0x2 ref 0 00000100:00100000:3.0:1789760722.746750:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760722.746858:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512580864 00000100:00100000:2.0:1789760722.746867:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512580864 00000100:00100000:2.0:1789760722.746923:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 702233191863 time 702269880318 state 0x0 ref 1 00000100:00100000:2.0:1789760722.746938:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 702270140734 time 702269880318 state 0x1 ref 1 00000100:00100000:2.0:1789760722.747005:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 702270140734 time 702311304418 state 0x0 ref 1 00000100:00100000:2.0:1789760722.747018:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febea4ba050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512580864:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760722.747070:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23097344, mtime: 0x6aad94d2, atime: 0x0, ctime: 0x6aad94d2, blocks: 0xb807 00010000:00010000:2.0:1789760722.747080:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2db5b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.747097:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=702s v=(0 0) 00010000:00010000:2.0:1789760722.747102:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760722.747109:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2db5b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23097344->23101439] (req 23097344->23101439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.747124:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2db5b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23097344->23101439] (req 23097344->23101439) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760722.773393:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972958 is committed 00002000:00010000:3.0:1789760722.773558:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23097344 -> 23101440 00010000:00010000:3.0:1789760722.773573:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2db5b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23097344->23101439] (req 23097344->23101439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.773586:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2db5b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23097344->23101439] (req 23097344->23101439) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.773595:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2db5b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23097344->23101439] (req 23097344->23101439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.773602:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760722.773607:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2db5b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23097344->23101439] (req 23097344->23101439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.773616:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2db5b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23097344->23101439] (req 23097344->23101439) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.773641:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2db5b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23097344->23101439] (req 23097344->23101439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.773648:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2db5b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23097344->23101439] (req 23097344->23101439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760722.773971:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febea4ba050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512580864:12345-192.168.203.56@tcp:4:dd.0 Request processed in 26948us (27271us total) trans 4294972958 rc 0/0 00000100:00100000:0.0:1789760722.773983:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5669 00000100:00100000:0.0:1789760722.773993:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 702270140734 time 702311304418 state 0x2 ref 0 00000100:00100000:0.0:1789760722.777493:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760722.777612:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512580992 00000100:00100000:2.0:1789760722.777621:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512580992 00000100:00100000:2.0:1789760722.777672:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 702270140734 time 702311304418 state 0x0 ref 1 00000100:00100000:2.0:1789760722.777687:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 702311564834 time 702311304418 state 0x1 ref 1 00000100:00100000:2.0:1789760722.777827:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 702311564834 time 702342126590 state 0x0 ref 1 00000100:00100000:2.0:1789760722.777839:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febffb9f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512580992:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760722.777884:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23101440, mtime: 0x6aad94d2, atime: 0x0, ctime: 0x6aad94d2, blocks: 0xb807 00010000:00010000:2.0:1789760722.777894:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2db62 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.777910:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=702s v=(0 0) 00010000:00010000:2.0:1789760722.777914:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760722.777921:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2db62 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23101440->23105535] (req 23101440->23105535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.777935:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2db62 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23101440->23105535] (req 23101440->23105535) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760722.817031:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23101440 -> 23105536 00010000:00010000:2.0:1789760722.817059:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2db62 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23101440->23105535] (req 23101440->23105535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.817073:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2db62 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23101440->23105535] (req 23101440->23105535) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.817082:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2db62 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23101440->23105535] (req 23101440->23105535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.817090:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760722.817095:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2db62 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23101440->23105535] (req 23101440->23105535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.817105:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2db62 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23101440->23105535] (req 23101440->23105535) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.817124:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2db62 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23101440->23105535] (req 23101440->23105535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.817132:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2db62 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23101440->23105535] (req 23101440->23105535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760722.817310:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febffb9f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2358:x1876699512580992:12345-192.168.203.56@tcp:4:dd.0 Request processed in 39467us (39829us total) trans 4294972959 rc 0/0 00000100:00100000:2.0:1789760722.817324:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5670 00000100:00100000:2.0:1789760722.817334:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 702311564834 time 702342126590 state 0x2 ref 0 00000001:00080000:2.0:1789760722.817378:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972959 is committed 00000100:00100000:1.0:1789760722.821482:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760722.821553:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512581120 00000100:00100000:2.0:1789760722.821563:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512581120 00000100:00100000:2.0:1789760722.821615:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 702311564834 time 702342126590 state 0x0 ref 1 00000100:00100000:2.0:1789760722.821630:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 702342387006 time 702342126590 state 0x1 ref 1 00000100:00100000:2.0:1789760722.821665:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 702342387006 time 702385964120 state 0x0 ref 1 00000100:00100000:2.0:1789760722.821678:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4130000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512581120:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760722.821773:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23105536, mtime: 0x6aad94d2, atime: 0x0, ctime: 0x6aad94d2, blocks: 0xb807 00010000:00010000:2.0:1789760722.821783:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2db69 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.821804:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=702s v=(0 0) 00010000:00010000:2.0:1789760722.821809:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760722.821816:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2db69 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23105536->23109631] (req 23105536->23109631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.821830:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2db69 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23105536->23109631] (req 23105536->23109631) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760722.858026:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23105536 -> 23109632 00010000:00010000:2.0:1789760722.858048:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2db69 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23105536->23109631] (req 23105536->23109631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.858061:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2db69 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23105536->23109631] (req 23105536->23109631) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.858071:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2db69 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23105536->23109631] (req 23105536->23109631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.858078:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760722.858083:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2db69 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23105536->23109631] (req 23105536->23109631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.858092:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2db69 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23105536->23109631] (req 23105536->23109631) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.858110:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2db69 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23105536->23109631] (req 23105536->23109631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.858118:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2db69 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23105536->23109631] (req 23105536->23109631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760722.858220:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4130000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2359:x1876699512581120:12345-192.168.203.56@tcp:4:dd.0 Request processed in 36540us (36752us total) trans 4294972960 rc 0/0 00000100:00100000:2.0:1789760722.858230:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5671 00000100:00100000:2.0:1789760722.858240:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 702342387006 time 702385964120 state 0x2 ref 0 00000001:00080000:1.0:1789760722.858834:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972960 is committed 00000100:00100000:1.0:1789760722.864665:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760722.865430:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512581248 00000100:00100000:0.0:1789760722.865516:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512581248 00000100:00100000:0.0:1789760722.865575:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 702342387006 time 702385964120 state 0x0 ref 1 00000100:00100000:0.0:1789760722.865591:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 702386224536 time 702385964120 state 0x1 ref 1 00000100:00100000:0.0:1789760722.865979:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 702386224536 time 702430276365 state 0x0 ref 1 00000100:00100000:0.0:1789760722.865995:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4133480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512581248:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760722.866049:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23109632, mtime: 0x6aad94d2, atime: 0x0, ctime: 0x6aad94d2, blocks: 0xb807 00010000:00010000:0.0:1789760722.866058:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2db70 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.866074:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=702s v=(0 0) 00010000:00010000:0.0:1789760722.866079:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760722.866085:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2db70 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23109632->23113727] (req 23109632->23113727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.866098:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2db70 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23109632->23113727] (req 23109632->23113727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760722.913401:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972961 is committed 00002000:00010000:2.0:1789760722.913558:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23109632 -> 23113728 00010000:00010000:2.0:1789760722.913573:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2db70 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23109632->23113727] (req 23109632->23113727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.913588:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2db70 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23109632->23113727] (req 23109632->23113727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.913598:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2db70 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23109632->23113727] (req 23109632->23113727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.913606:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760722.913611:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2db70 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23109632->23113727] (req 23109632->23113727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.913621:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2db70 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23109632->23113727] (req 23109632->23113727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.913641:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2db70 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23109632->23113727] (req 23109632->23113727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.913649:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2db70 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23109632->23113727] (req 23109632->23113727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760722.913787:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4133480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512581248:12345-192.168.203.56@tcp:4:dd.0 Request processed in 47791us (49126us total) trans 4294972961 rc 0/0 00000100:00100000:2.0:1789760722.913797:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5672 00000100:00100000:2.0:1789760722.913808:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 702386224536 time 702430276365 state 0x2 ref 0 00000100:00100000:0.0:1789760722.916668:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760722.917046:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512581376 00000100:00100000:2.0:1789760722.917057:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512581376 00000100:00100000:2.0:1789760722.917114:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 702386224536 time 702430276365 state 0x0 ref 1 00000100:00100000:2.0:1789760722.917130:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 702430536781 time 702430276365 state 0x1 ref 1 00000100:00100000:2.0:1789760722.917169:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 702430536781 time 702481468898 state 0x0 ref 1 00000100:00100000:2.0:1789760722.917183:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febffb9ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512581376:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760722.917236:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23113728, mtime: 0x6aad94d2, atime: 0x0, ctime: 0x6aad94d2, blocks: 0xb807 00010000:00010000:2.0:1789760722.917246:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2db77 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.917264:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=702s v=(0 0) 00010000:00010000:2.0:1789760722.917268:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760722.917274:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2db77 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23113728->23117823] (req 23113728->23117823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.917288:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2db77 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23113728->23117823] (req 23113728->23117823) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760723.008451:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972962 is committed 00002000:00010000:2.0:1789760723.008584:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23113728 -> 23117824 00010000:00010000:2.0:1789760723.008600:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2db77 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23113728->23117823] (req 23113728->23117823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.008617:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2db77 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23113728->23117823] (req 23113728->23117823) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.008625:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2db77 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23113728->23117823] (req 23113728->23117823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.008631:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760723.008635:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2db77 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23113728->23117823] (req 23113728->23117823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.008643:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2db77 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23113728->23117823] (req 23113728->23117823) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.008660:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2db77 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23113728->23117823] (req 23113728->23117823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.008667:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2db77 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23113728->23117823] (req 23113728->23117823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760723.009004:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febffb9ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512581376:12345-192.168.203.56@tcp:4:dd.0 Request processed in 91817us (92339us total) trans 4294972962 rc 0/0 00000100:00100000:2.0:1789760723.009015:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5673 00000100:00100000:2.0:1789760723.009025:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 702430536781 time 702481468898 state 0x2 ref 0 00000100:00100000:1.0:1789760723.011598:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760723.011688:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512581504 00000100:00100000:2.0:1789760723.011766:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512581504 00000100:00100000:1.0:1789760723.011992:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 702430536781 time 702481468898 state 0x0 ref 1 00000100:00100000:1.0:1789760723.012009:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 702481729314 time 702481468898 state 0x1 ref 1 00000100:00100000:1.0:1789760723.012044:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 702481729314 time 702576343536 state 0x0 ref 1 00000100:00100000:1.0:1789760723.012056:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4130700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512581504:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760723.012099:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23117824, mtime: 0x6aad94d2, atime: 0x0, ctime: 0x6aad94d2, blocks: 0xb807 00010000:00010000:1.0:1789760723.012107:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2db7e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.012123:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=702s v=(0 0) 00010000:00010000:1.0:1789760723.012127:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760723.012134:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2db7e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23117824->23121919] (req 23117824->23121919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.012259:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2db7e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23117824->23121919] (req 23117824->23121919) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1789760723.012651:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:2.0:1789760723.048431:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972963 is committed 00002000:00010000:1.0:1789760723.048521:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23117824 -> 23121920 00010000:00010000:1.0:1789760723.048538:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2db7e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23117824->23121919] (req 23117824->23121919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.048552:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2db7e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23117824->23121919] (req 23117824->23121919) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.048567:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2db7e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23117824->23121919] (req 23117824->23121919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.048575:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760723.048580:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2db7e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23117824->23121919] (req 23117824->23121919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.048590:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2db7e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23117824->23121919] (req 23117824->23121919) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.048609:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2db7e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23117824->23121919] (req 23117824->23121919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.048617:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2db7e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23117824->23121919] (req 23117824->23121919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760723.048771:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4130700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512581504:12345-192.168.203.56@tcp:4:dd.0 Request processed in 36711us (37175us total) trans 4294972963 rc 0/0 00000100:00100000:1.0:1789760723.048781:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5674 00000100:00100000:1.0:1789760723.048790:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 702481729314 time 702576343536 state 0x2 ref 0 00000100:00100000:1.0:1789760723.052420:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760723.052522:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512581632 00000100:00100000:2.0:1789760723.052531:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512581632 00000100:00100000:2.0:1789760723.052584:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 702481729314 time 702576343536 state 0x0 ref 1 00000100:00100000:2.0:1789760723.052600:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 702576603952 time 702576343536 state 0x1 ref 1 00000100:00100000:2.0:1789760723.052665:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 702576603952 time 702616963635 state 0x0 ref 1 00000100:00100000:2.0:1789760723.052678:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4131180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512581632:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760723.052765:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23121920, mtime: 0x6aad94d2, atime: 0x0, ctime: 0x6aad94d2, blocks: 0xb807 00010000:00010000:2.0:1789760723.052774:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2db85 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.052790:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=702s v=(0 0) 00010000:00010000:2.0:1789760723.052794:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760723.052801:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2db85 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23121920->23126015] (req 23121920->23126015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.052816:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2db85 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23121920->23126015] (req 23121920->23126015) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760723.094602:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972964 is committed 00002000:00010000:0.0:1789760723.094770:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23121920 -> 23126016 00010000:00010000:0.0:1789760723.094787:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2db85 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23121920->23126015] (req 23121920->23126015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.094800:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2db85 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23121920->23126015] (req 23121920->23126015) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.094809:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2db85 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23121920->23126015] (req 23121920->23126015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.094817:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760723.094822:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2db85 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23121920->23126015] (req 23121920->23126015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.094838:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2db85 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23121920->23126015] (req 23121920->23126015) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.094858:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2db85 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23121920->23126015] (req 23121920->23126015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.094997:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2db85 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23121920->23126015] (req 23121920->23126015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760723.095123:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4131180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512581632:12345-192.168.203.56@tcp:4:dd.0 Request processed in 42443us (42707us total) trans 4294972964 rc 0/0 00000100:00100000:0.0:1789760723.095134:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5675 00000100:00100000:0.0:1789760723.095206:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 702576603952 time 702616963635 state 0x2 ref 0 00000100:00100000:0.0:1789760723.097811:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760723.098084:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512581760 00000100:00100000:2.0:1789760723.098093:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512581760 00000100:00100000:2.0:1789760723.098197:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 702576603952 time 702616963635 state 0x0 ref 1 00000100:00100000:2.0:1789760723.098215:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 702617224051 time 702616963635 state 0x1 ref 1 00000100:00100000:2.0:1789760723.098269:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 702617224051 time 702662568473 state 0x0 ref 1 00000100:00100000:2.0:1789760723.098283:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febffb9c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512581760:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760723.098329:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23126016, mtime: 0x6aad94d2, atime: 0x0, ctime: 0x6aad94d2, blocks: 0xb807 00010000:00010000:2.0:1789760723.098338:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2db8c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.098354:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=702s v=(0 0) 00010000:00010000:2.0:1789760723.098358:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760723.098365:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2db8c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23126016->23130111] (req 23126016->23130111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.098379:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2db8c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23126016->23130111] (req 23126016->23130111) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760723.132619:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972965 is committed 00002000:00010000:0.0:1789760723.132879:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23126016 -> 23130112 00010000:00010000:0.0:1789760723.132897:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2db8c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23126016->23130111] (req 23126016->23130111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.132909:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2db8c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23126016->23130111] (req 23126016->23130111) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.132917:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2db8c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23126016->23130111] (req 23126016->23130111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.132928:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760723.132933:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2db8c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23126016->23130111] (req 23126016->23130111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.132941:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2db8c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23126016->23130111] (req 23126016->23130111) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.132959:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2db8c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23126016->23130111] (req 23126016->23130111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.132967:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2db8c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23126016->23130111] (req 23126016->23130111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760723.133094:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febffb9c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512581760:12345-192.168.203.56@tcp:4:dd.0 Request processed in 34809us (35286us total) trans 4294972965 rc 0/0 00000100:00100000:0.0:1789760723.133104:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5676 00000100:00100000:0.0:1789760723.133114:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 702617224051 time 702662568473 state 0x2 ref 0 00000100:00100000:1.0:1789760723.136643:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760723.136814:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512581888 00000100:00100000:2.0:1789760723.136824:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512581888 00000100:00100000:2.0:1789760723.136949:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 702617224051 time 702662568473 state 0x0 ref 1 00000100:00100000:2.0:1789760723.136963:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 702662828889 time 702662568473 state 0x1 ref 1 00000100:00100000:2.0:1789760723.137028:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 702662828889 time 702701326970 state 0x0 ref 1 00000100:00100000:2.0:1789760723.137041:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4131500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512581888:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760723.137091:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23130112, mtime: 0x6aad94d2, atime: 0x0, ctime: 0x6aad94d2, blocks: 0xb807 00010000:00010000:2.0:1789760723.137101:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2db93 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.137115:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=702s v=(0 0) 00010000:00010000:2.0:1789760723.137120:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760723.137126:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2db93 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23130112->23134207] (req 23130112->23134207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.137139:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2db93 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23130112->23134207] (req 23130112->23134207) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760723.172531:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23130112 -> 23134208 00010000:00010000:3.0:1789760723.172550:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2db93 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23130112->23134207] (req 23130112->23134207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760723.172564:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2db93 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23130112->23134207] (req 23130112->23134207) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760723.172574:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2db93 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23130112->23134207] (req 23130112->23134207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760723.172580:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760723.172585:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2db93 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23130112->23134207] (req 23130112->23134207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760723.172595:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2db93 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23130112->23134207] (req 23130112->23134207) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760723.172614:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2db93 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23130112->23134207] (req 23130112->23134207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760723.172628:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2db93 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23130112->23134207] (req 23130112->23134207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760723.172745:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4131500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2361:x1876699512581888:12345-192.168.203.56@tcp:4:dd.0 Request processed in 35701us (36106us total) trans 4294972966 rc 0/0 00000100:00100000:3.0:1789760723.172755:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5677 00000100:00100000:3.0:1789760723.172765:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 702662828889 time 702701326970 state 0x2 ref 0 00000001:00080000:0.0:1789760723.173169:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972966 is committed 00000100:00100000:3.0:1789760723.175616:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760723.175739:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512582016 00000100:00100000:0.0:1789760723.175756:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512582016 00000100:00100000:0.0:1789760723.175805:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 702662828889 time 702701326970 state 0x0 ref 1 00000100:00100000:0.0:1789760723.175819:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 702701587386 time 702701326970 state 0x1 ref 1 00000100:00100000:0.0:1789760723.175843:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 702701587386 time 702740143269 state 0x0 ref 1 00000100:00100000:0.0:1789760723.175854:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac28f0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512582016:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760723.175900:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23134208, mtime: 0x6aad94d2, atime: 0x0, ctime: 0x6aad94d2, blocks: 0xb807 00010000:00010000:0.0:1789760723.175908:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2db9a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.175926:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=702s v=(0 0) 00010000:00010000:0.0:1789760723.175931:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760723.175938:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2db9a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23134208->23138303] (req 23134208->23138303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.175951:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2db9a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23134208->23138303] (req 23134208->23138303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760723.218859:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972967 is committed 00002000:00010000:1.0:1789760723.219114:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23134208 -> 23138304 00010000:00010000:1.0:1789760723.219129:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2db9a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23134208->23138303] (req 23134208->23138303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.219142:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2db9a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23134208->23138303] (req 23134208->23138303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.219152:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2db9a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23134208->23138303] (req 23134208->23138303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.219159:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760723.219164:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2db9a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23134208->23138303] (req 23134208->23138303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.219173:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2db9a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23134208->23138303] (req 23134208->23138303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.219192:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2db9a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23134208->23138303] (req 23134208->23138303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.219200:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2db9a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23134208->23138303] (req 23134208->23138303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760723.219287:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac28f0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512582016:12345-192.168.203.56@tcp:4:dd.0 Request processed in 43429us (43674us total) trans 4294972967 rc 0/0 00000100:00100000:1.0:1789760723.219297:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5678 00000100:00100000:1.0:1789760723.219305:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 702701587386 time 702740143269 state 0x2 ref 0 00000100:00100000:3.0:1789760723.222853:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760723.223012:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512582144 00000100:00100000:0.0:1789760723.223023:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512582144 00000100:00100000:0.0:1789760723.223076:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 702701587386 time 702740143269 state 0x0 ref 1 00000100:00100000:0.0:1789760723.223093:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 702740403685 time 702740143269 state 0x1 ref 1 00000100:00100000:0.0:1789760723.223163:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 702740403685 time 702787461703 state 0x0 ref 1 00000100:00100000:0.0:1789760723.223177:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac28f2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512582144:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760723.223228:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23138304, mtime: 0x6aad94d2, atime: 0x0, ctime: 0x6aad94d2, blocks: 0xb807 00010000:00010000:0.0:1789760723.223238:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2dba1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.223255:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=702s v=(0 0) 00010000:00010000:0.0:1789760723.223259:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760723.223265:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2dba1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23138304->23142399] (req 23138304->23142399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.223279:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2dba1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23138304->23142399] (req 23138304->23142399) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760723.264173:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972968 is committed 00002000:00010000:1.0:1789760723.264298:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23138304 -> 23142400 00010000:00010000:1.0:1789760723.264313:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2dba1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23138304->23142399] (req 23138304->23142399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.264325:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2dba1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23138304->23142399] (req 23138304->23142399) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.264335:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2dba1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23138304->23142399] (req 23138304->23142399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.264342:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760723.264346:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2dba1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23138304->23142399] (req 23138304->23142399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.264356:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2dba1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23138304->23142399] (req 23138304->23142399) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.264374:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2dba1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23138304->23142399] (req 23138304->23142399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.264381:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2dba1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23138304->23142399] (req 23138304->23142399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760723.264480:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac28f2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512582144:12345-192.168.203.56@tcp:4:dd.0 Request processed in 41295us (41637us total) trans 4294972968 rc 0/0 00000100:00100000:1.0:1789760723.264490:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5679 00000100:00100000:1.0:1789760723.264500:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 702740403685 time 702787461703 state 0x2 ref 0 00000100:00100000:0.0:1789760723.273335:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760723.273484:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512582272 00000100:00100000:3.0:1789760723.273493:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512582272 00000100:00100000:3.0:1789760723.273541:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 702740403685 time 702787461703 state 0x0 ref 1 00000100:00100000:3.0:1789760723.273557:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 702787722119 time 702787461703 state 0x1 ref 1 00000100:00100000:3.0:1789760723.273620:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 702787722119 time 702837919226 state 0x0 ref 1 00000100:00100000:3.0:1789760723.273633:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febffb9c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512582272:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760723.273677:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23142400, mtime: 0x6aad94d2, atime: 0x0, ctime: 0x6aad94d2, blocks: 0xb807 00010000:00010000:3.0:1789760723.273686:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2dba8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760723.273718:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=702s v=(0 0) 00010000:00010000:3.0:1789760723.273723:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760723.273730:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2dba8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23142400->23146495] (req 23142400->23146495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760723.273744:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2dba8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23142400->23146495] (req 23142400->23146495) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.314832:0:5836:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:1.0:1789760723.322170:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972969 is committed 00002000:00010000:3.0:1789760723.322296:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23142400 -> 23146496 00010000:00010000:3.0:1789760723.322311:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2dba8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23142400->23146495] (req 23142400->23146495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760723.322324:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2dba8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23142400->23146495] (req 23142400->23146495) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760723.322332:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2dba8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23142400->23146495] (req 23142400->23146495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760723.322339:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760723.322343:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2dba8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23142400->23146495] (req 23142400->23146495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760723.322351:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2dba8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23142400->23146495] (req 23142400->23146495) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760723.322370:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2dba8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23142400->23146495] (req 23142400->23146495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760723.322378:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2dba8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23142400->23146495] (req 23142400->23146495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760723.322511:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febffb9c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512582272:12345-192.168.203.56@tcp:4:dd.0 Request processed in 48875us (49179us total) trans 4294972969 rc 0/0 00000100:00100000:3.0:1789760723.322520:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5680 00000100:00100000:3.0:1789760723.322530:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 702787722119 time 702837919226 state 0x2 ref 0 00000100:00100000:3.0:1789760723.325970:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760723.326131:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512582400 00000100:00100000:1.0:1789760723.326141:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512582400 00000100:00100000:1.0:1789760723.326202:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 702787722119 time 702837919226 state 0x0 ref 1 00000100:00100000:1.0:1789760723.326216:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 702838179642 time 702837919226 state 0x1 ref 1 00000100:00100000:1.0:1789760723.326245:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 702838179642 time 702890544536 state 0x0 ref 1 00000100:00100000:1.0:1789760723.326257:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac28f1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512582400:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760723.326315:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23146496, mtime: 0x6aad94d2, atime: 0x0, ctime: 0x6aad94d2, blocks: 0xb807 00010000:00010000:1.0:1789760723.326323:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2dbaf lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.326339:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=702s v=(0 0) 00010000:00010000:1.0:1789760723.326343:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760723.326350:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2dbaf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23146496->23150591] (req 23146496->23150591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.326365:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2dbaf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23146496->23150591] (req 23146496->23150591) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760723.358909:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23146496 -> 23150592 00010000:00010000:1.0:1789760723.358930:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2dbaf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23146496->23150591] (req 23146496->23150591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.358944:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2dbaf lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23146496->23150591] (req 23146496->23150591) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.358955:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2dbaf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23146496->23150591] (req 23146496->23150591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.358963:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760723.358967:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2dbaf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23146496->23150591] (req 23146496->23150591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.358976:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2dbaf lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23146496->23150591] (req 23146496->23150591) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.358995:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2dbaf lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23146496->23150591] (req 23146496->23150591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.359003:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2dbaf lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23146496->23150591] (req 23146496->23150591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760723.359009:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972970 is committed 00000100:00100000:1.0:1789760723.359106:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac28f1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512582400:12345-192.168.203.56@tcp:4:dd.0 Request processed in 32846us (33140us total) trans 4294972970 rc 0/0 00000100:00100000:1.0:1789760723.359117:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5681 00000100:00100000:1.0:1789760723.359126:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 702838179642 time 702890544536 state 0x2 ref 0 00000100:00100000:0.0:1789760723.363461:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760723.363518:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512582528 00000100:00100000:1.0:1789760723.363528:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512582528 00000100:00100000:1.0:1789760723.363580:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 702838179642 time 702890544536 state 0x0 ref 1 00000100:00100000:1.0:1789760723.363597:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 702890804952 time 702890544536 state 0x1 ref 1 00000100:00100000:1.0:1789760723.363676:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 702890804952 time 702927973959 state 0x0 ref 1 00000100:00100000:1.0:1789760723.363691:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febffb9d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512582528:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760723.363775:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23150592, mtime: 0x6aad94d2, atime: 0x0, ctime: 0x6aad94d2, blocks: 0xb807 00010000:00010000:1.0:1789760723.363784:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2dbb6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.363799:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=702s v=(0 0) 00010000:00010000:1.0:1789760723.363803:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760723.363810:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2dbb6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23150592->23154687] (req 23150592->23154687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.363825:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2dbb6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23150592->23154687] (req 23150592->23154687) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760723.412895:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972971 is committed 00002000:00010000:1.0:1789760723.412980:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23150592 -> 23154688 00010000:00010000:1.0:1789760723.412996:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2dbb6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23150592->23154687] (req 23150592->23154687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.413009:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2dbb6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23150592->23154687] (req 23150592->23154687) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.413023:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2dbb6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23150592->23154687] (req 23150592->23154687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.413031:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760723.413036:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2dbb6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23150592->23154687] (req 23150592->23154687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.413045:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2dbb6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23150592->23154687] (req 23150592->23154687) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.413063:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2dbb6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23150592->23154687] (req 23150592->23154687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.413191:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2dbb6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23150592->23154687] (req 23150592->23154687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760723.413318:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febffb9d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512582528:12345-192.168.203.56@tcp:4:dd.0 Request processed in 49625us (49861us total) trans 4294972971 rc 0/0 00000100:00100000:1.0:1789760723.413328:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5682 00000100:00100000:1.0:1789760723.413338:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 702890804952 time 702927973959 state 0x2 ref 0 00000100:00100000:0.0:1789760723.416960:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760723.417047:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512582656 00000100:00100000:3.0:1789760723.417056:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512582656 00000100:00100000:3.0:1789760723.417121:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 702890804952 time 702927973959 state 0x0 ref 1 00000100:00100000:3.0:1789760723.417137:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 702928234375 time 702927973959 state 0x1 ref 1 00000100:00100000:3.0:1789760723.417164:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 702928234375 time 702981463847 state 0x0 ref 1 00000100:00100000:3.0:1789760723.417176:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febffb9f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512582656:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760723.417222:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23154688, mtime: 0x6aad94d2, atime: 0x0, ctime: 0x6aad94d2, blocks: 0xb807 00010000:00010000:3.0:1789760723.417231:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2dbbd lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760723.417247:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=702s v=(0 0) 00010000:00010000:3.0:1789760723.417251:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760723.417258:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2dbbd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23154688->23158783] (req 23154688->23158783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760723.417277:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2dbbd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23154688->23158783] (req 23154688->23158783) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760723.450964:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23154688 -> 23158784 00010000:00010000:1.0:1789760723.450985:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2dbbd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23154688->23158783] (req 23154688->23158783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.451004:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2dbbd lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23154688->23158783] (req 23154688->23158783) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.451013:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2dbbd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23154688->23158783] (req 23154688->23158783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.451020:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760723.451025:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2dbbd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23154688->23158783] (req 23154688->23158783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.451035:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2dbbd lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23154688->23158783] (req 23154688->23158783) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.451056:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2dbbd lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23154688->23158783] (req 23154688->23158783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.451064:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2dbbd lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23154688->23158783] (req 23154688->23158783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760723.451282:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972972 is committed 00000100:00100000:1.0:1789760723.451343:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febffb9f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512582656:12345-192.168.203.56@tcp:4:dd.0 Request processed in 34164us (34387us total) trans 4294972972 rc 0/0 00000100:00100000:1.0:1789760723.451354:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5683 00000100:00100000:1.0:1789760723.451363:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 702928234375 time 702981463847 state 0x2 ref 0 00000100:00100000:2.0:1789760723.457324:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760723.457456:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512582784 00000100:00100000:1.0:1789760723.457482:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512582784 00000100:00100000:1.0:1789760723.457542:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 702928234375 time 702981463847 state 0x0 ref 1 00000100:00100000:1.0:1789760723.457557:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 702981724263 time 702981463847 state 0x1 ref 1 00000100:00100000:1.0:1789760723.457625:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 702981724263 time 703021924274 state 0x0 ref 1 00000100:00100000:1.0:1789760723.457638:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febfbddc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512582784:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760723.457687:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23158784, mtime: 0x6aad94d2, atime: 0x0, ctime: 0x6aad94d2, blocks: 0xb807 00010000:00010000:1.0:1789760723.457715:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2dbc4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.457731:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=703s v=(0 0) 00010000:00010000:1.0:1789760723.457736:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760723.457742:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2dbc4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23158784->23162879] (req 23158784->23162879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.457756:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2dbc4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23158784->23162879] (req 23158784->23162879) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760723.516374:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972973 is committed 00002000:00010000:0.0:1789760723.516976:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23158784 -> 23162880 00010000:00010000:0.0:1789760723.516999:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2dbc4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23158784->23162879] (req 23158784->23162879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.517015:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2dbc4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23158784->23162879] (req 23158784->23162879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.517025:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2dbc4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23158784->23162879] (req 23158784->23162879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.517033:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760723.517039:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2dbc4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23158784->23162879] (req 23158784->23162879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.517050:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2dbc4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23158784->23162879] (req 23158784->23162879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.517075:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2dbc4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23158784->23162879] (req 23158784->23162879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.517084:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2dbc4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23158784->23162879] (req 23158784->23162879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760723.518131:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febfbddc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512582784:12345-192.168.203.56@tcp:4:dd.0 Request processed in 60482us (60808us total) trans 4294972973 rc 0/0 00000100:00100000:0.0:1789760723.518148:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5684 00000100:00100000:0.0:1789760723.518160:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 702981724263 time 703021924274 state 0x2 ref 0 00000100:00100000:3.0:1789760723.522071:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760723.523013:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512582912 00000100:00100000:1.0:1789760723.523031:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512582912 00000100:00100000:1.0:1789760723.523115:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 702981724263 time 703021924274 state 0x0 ref 1 00000100:00100000:1.0:1789760723.523198:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 703022184690 time 703021924274 state 0x1 ref 1 00000100:00100000:1.0:1789760723.523284:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 703022184690 time 703087581673 state 0x0 ref 1 00000100:00100000:1.0:1789760723.523316:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac28f0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512582912:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760723.523396:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23162880, mtime: 0x6aad94d2, atime: 0x0, ctime: 0x6aad94d2, blocks: 0xb807 00010000:00010000:1.0:1789760723.523407:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2dbcb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.523430:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=703s v=(0 0) 00010000:00010000:1.0:1789760723.523436:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760723.523445:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2dbcb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23162880->23166975] (req 23162880->23166975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.523464:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2dbcb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23162880->23166975] (req 23162880->23166975) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760723.593743:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972974 is committed 00002000:00010000:1.0:1789760723.593844:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23162880 -> 23166976 00002000:00010000:1.0:1789760723.593849:0:6708:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760722 -> 1789760723 00002000:00010000:1.0:1789760723.593852:0:6708:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760722 -> 1789760723 00010000:00010000:1.0:1789760723.593881:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2dbcb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23162880->23166975] (req 23162880->23166975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.593897:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2dbcb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23162880->23166975] (req 23162880->23166975) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.593918:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2dbcb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23162880->23166975] (req 23162880->23166975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.593926:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760723.593933:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2dbcb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23162880->23166975] (req 23162880->23166975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.593944:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2dbcb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23162880->23166975] (req 23162880->23166975) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.593969:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2dbcb lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23162880->23166975] (req 23162880->23166975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.593978:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2dbcb lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23162880->23166975] (req 23162880->23166975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760723.594102:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac28f0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512582912:12345-192.168.203.56@tcp:4:dd.0 Request processed in 70785us (72039us total) trans 4294972974 rc 0/0 00000100:00100000:1.0:1789760723.594115:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5685 00000100:00100000:1.0:1789760723.594127:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 703022184690 time 703087581673 state 0x2 ref 0 00000100:00100000:0.0:1789760723.598177:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760723.598292:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512583040 00000100:00100000:1.0:1789760723.598307:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512583040 00000100:00100000:1.0:1789760723.598382:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 703022184690 time 703087581673 state 0x0 ref 1 00000100:00100000:1.0:1789760723.598400:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 703087842089 time 703087581673 state 0x1 ref 1 00000100:00100000:1.0:1789760723.598435:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 703087842089 time 703162734395 state 0x0 ref 1 00000100:00100000:1.0:1789760723.598450:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febffb9f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512583040:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760723.598529:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23166976, mtime: 0x6aad94d3, atime: 0x0, ctime: 0x6aad94d3, blocks: 0xb807 00010000:00010000:1.0:1789760723.598541:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2dbd2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.598561:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=703s v=(0 0) 00010000:00010000:1.0:1789760723.598566:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760723.598574:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2dbd2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23166976->23171071] (req 23166976->23171071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.598591:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2dbd2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23166976->23171071] (req 23166976->23171071) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760723.658200:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972975 is committed 00002000:00010000:1.0:1789760723.658492:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23166976 -> 23171072 00010000:00010000:1.0:1789760723.658512:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2dbd2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23166976->23171071] (req 23166976->23171071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.658528:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2dbd2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23166976->23171071] (req 23166976->23171071) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.658543:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2dbd2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23166976->23171071] (req 23166976->23171071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.658551:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760723.658557:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2dbd2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23166976->23171071] (req 23166976->23171071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.658568:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2dbd2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23166976->23171071] (req 23166976->23171071) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.658600:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2dbd2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23166976->23171071] (req 23166976->23171071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.658609:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2dbd2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23166976->23171071] (req 23166976->23171071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760723.658841:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febffb9f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512583040:12345-192.168.203.56@tcp:4:dd.0 Request processed in 60323us (60609us total) trans 4294972975 rc 0/0 00000100:00100000:1.0:1789760723.658854:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5686 00000100:00100000:1.0:1789760723.658934:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 703087842089 time 703162734395 state 0x2 ref 0 00000100:00100000:3.0:1789760723.668630:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760723.669345:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512583168 00000100:00100000:0.0:1789760723.669549:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512583168 00000100:00100000:2.0:1789760723.669817:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 703087842089 time 703162734395 state 0x0 ref 1 00000100:00100000:2.0:1789760723.669838:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 703162994811 time 703162734395 state 0x1 ref 1 00000100:00100000:2.0:1789760723.670485:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 703162994811 time 703234769820 state 0x0 ref 1 00000100:00100000:2.0:1789760723.670506:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febfb894700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512583168:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760723.670580:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23171072, mtime: 0x6aad94d3, atime: 0x0, ctime: 0x6aad94d3, blocks: 0xb807 00010000:00010000:2.0:1789760723.670591:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2dbd9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.670610:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=703s v=(0 0) 00010000:00010000:2.0:1789760723.670615:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760723.670624:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2dbd9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23171072->23175167] (req 23171072->23175167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.670642:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2dbd9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23171072->23175167] (req 23171072->23175167) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760723.739834:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972976 is committed 00002000:00010000:1.0:1789760723.740556:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23171072 -> 23175168 00010000:00010000:1.0:1789760723.740576:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2dbd9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23171072->23175167] (req 23171072->23175167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.740592:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2dbd9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23171072->23175167] (req 23171072->23175167) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.740602:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2dbd9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23171072->23175167] (req 23171072->23175167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.740610:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760723.740616:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2dbd9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23171072->23175167] (req 23171072->23175167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.740633:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2dbd9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23171072->23175167] (req 23171072->23175167) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.740657:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2dbd9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23171072->23175167] (req 23171072->23175167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.740665:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2dbd9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23171072->23175167] (req 23171072->23175167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760723.740837:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febfb894700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512583168:12345-192.168.203.56@tcp:4:dd.0 Request processed in 70330us (72473us total) trans 4294972976 rc 0/0 00000100:00100000:1.0:1789760723.740849:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5687 00000100:00100000:1.0:1789760723.740871:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 703162994811 time 703234769820 state 0x2 ref 0 00000100:00100000:0.0:1789760723.750267:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760723.750550:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512583296 00000100:00100000:1.0:1789760723.750564:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512583296 00000100:00100000:1.0:1789760723.750646:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 703162994811 time 703234769820 state 0x0 ref 1 00000100:00100000:1.0:1789760723.750665:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 703235030236 time 703234769820 state 0x1 ref 1 00000100:00100000:1.0:1789760723.750736:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 703235030236 time 703315035536 state 0x0 ref 1 00000100:00100000:1.0:1789760723.750749:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febea651500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512583296:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760723.750815:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23175168, mtime: 0x6aad94d3, atime: 0x0, ctime: 0x6aad94d3, blocks: 0xb807 00010000:00010000:1.0:1789760723.750825:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2dbe0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.750846:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=703s v=(0 0) 00010000:00010000:1.0:1789760723.750851:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760723.750871:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2dbe0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23175168->23179263] (req 23175168->23179263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.750888:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2dbe0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23175168->23179263] (req 23175168->23179263) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760723.842867:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972977 is committed 00002000:00010000:1.0:1789760723.843303:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23175168 -> 23179264 00010000:00010000:1.0:1789760723.843322:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2dbe0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23175168->23179263] (req 23175168->23179263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.843333:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2dbe0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23175168->23179263] (req 23175168->23179263) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.843339:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2dbe0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23175168->23179263] (req 23175168->23179263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.843344:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760723.843349:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2dbe0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23175168->23179263] (req 23175168->23179263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.843357:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2dbe0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23175168->23179263] (req 23175168->23179263) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.843375:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2dbe0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23175168->23179263] (req 23175168->23179263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.843381:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2dbe0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23175168->23179263] (req 23175168->23179263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760723.843573:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febea651500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512583296:12345-192.168.203.56@tcp:4:dd.0 Request processed in 92759us (93257us total) trans 4294972977 rc 0/0 00000100:00100000:1.0:1789760723.843584:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5688 00000100:00100000:1.0:1789760723.843596:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 703235030236 time 703315035536 state 0x2 ref 0 00000100:00100000:0.0:1789760723.847400:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760723.847572:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512583424 00000100:00100000:1.0:1789760723.847586:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512583424 00000100:00100000:1.0:1789760723.847658:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 703235030236 time 703315035536 state 0x0 ref 1 00000100:00100000:1.0:1789760723.847678:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 703315295952 time 703315035536 state 0x1 ref 1 00000100:00100000:1.0:1789760723.847792:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 703315295952 time 703412089435 state 0x0 ref 1 00000100:00100000:1.0:1789760723.847807:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febea650380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512583424:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760723.847890:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23179264, mtime: 0x6aad94d3, atime: 0x0, ctime: 0x6aad94d3, blocks: 0xb807 00010000:00010000:1.0:1789760723.847901:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2dbe7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.847922:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=703s v=(0 0) 00010000:00010000:1.0:1789760723.847927:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760723.847936:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2dbe7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23179264->23183359] (req 23179264->23183359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.847959:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2dbe7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23179264->23183359] (req 23179264->23183359) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760723.896558:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972978 is committed 00002000:00010000:0.0:1789760723.896771:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23179264 -> 23183360 00010000:00010000:0.0:1789760723.896793:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2dbe7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23179264->23183359] (req 23179264->23183359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.896810:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2dbe7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23179264->23183359] (req 23179264->23183359) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.896820:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2dbe7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23179264->23183359] (req 23179264->23183359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.896829:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760723.896835:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2dbe7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23179264->23183359] (req 23179264->23183359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.896846:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2dbe7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23179264->23183359] (req 23179264->23183359) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.896945:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2dbe7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23179264->23183359] (req 23179264->23183359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.896956:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2dbe7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23179264->23183359] (req 23179264->23183359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760723.897266:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febea650380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512583424:12345-192.168.203.56@tcp:4:dd.0 Request processed in 49380us (49801us total) trans 4294972978 rc 0/0 00000100:00100000:0.0:1789760723.897280:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5689 00000100:00100000:0.0:1789760723.897294:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 703315295952 time 703412089435 state 0x2 ref 0 00000100:00100000:3.0:1789760723.907825:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760723.908166:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512583552 00000100:00100000:1.0:1789760723.908180:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512583552 00000100:00100000:1.0:1789760723.908265:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 703315295952 time 703412089435 state 0x0 ref 1 00000100:00100000:1.0:1789760723.908284:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 703412349851 time 703412089435 state 0x1 ref 1 00000100:00100000:1.0:1789760723.908729:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 703412349851 time 703473025396 state 0x0 ref 1 00000100:00100000:1.0:1789760723.908759:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febfb897100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512583552:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760723.908827:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23183360, mtime: 0x6aad94d3, atime: 0x0, ctime: 0x6aad94d3, blocks: 0xb807 00010000:00010000:1.0:1789760723.908839:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2dbee lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.908868:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=703s v=(0 0) 00010000:00010000:1.0:1789760723.908874:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760723.908882:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2dbee lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23183360->23187455] (req 23183360->23187455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.908901:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2dbee lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23183360->23187455] (req 23183360->23187455) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760723.963926:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972979 is committed 00002000:00010000:3.0:1789760723.966983:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23183360 -> 23187456 00010000:00010000:3.0:1789760723.967004:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2dbee lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23183360->23187455] (req 23183360->23187455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760723.967017:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2dbee lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23183360->23187455] (req 23183360->23187455) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760723.967027:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2dbee lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23183360->23187455] (req 23183360->23187455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760723.967034:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760723.967039:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2dbee lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23183360->23187455] (req 23183360->23187455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760723.967047:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2dbee lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23183360->23187455] (req 23183360->23187455) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760723.967072:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2dbee lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23183360->23187455] (req 23183360->23187455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760723.967079:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2dbee lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23183360->23187455] (req 23183360->23187455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760723.967193:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febfb897100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512583552:12345-192.168.203.56@tcp:4:dd.0 Request processed in 58435us (59379us total) trans 4294972979 rc 0/0 00000100:00100000:3.0:1789760723.967204:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5690 00000100:00100000:3.0:1789760723.967213:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 703412349851 time 703473025396 state 0x2 ref 0 00000100:00100000:3.0:1789760723.970535:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760723.970635:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512583680 00000100:00100000:1.0:1789760723.970646:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512583680 00000100:00100000:1.0:1789760723.971025:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 703412349851 time 703473025396 state 0x0 ref 1 00000100:00100000:1.0:1789760723.971043:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 703473285812 time 703473025396 state 0x1 ref 1 00000100:00100000:1.0:1789760723.971079:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 703473285812 time 703535377609 state 0x0 ref 1 00000100:00100000:1.0:1789760723.971093:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febfb895880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512583680:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760723.971254:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23187456, mtime: 0x6aad94d3, atime: 0x0, ctime: 0x6aad94d3, blocks: 0xb807 00010000:00010000:1.0:1789760723.971265:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2dbf5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.971282:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=703s v=(0 0) 00010000:00010000:1.0:1789760723.971287:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760723.971294:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2dbf5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23187456->23191551] (req 23187456->23191551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.971309:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2dbf5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23187456->23191551] (req 23187456->23191551) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760724.048356:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972980 is committed 00002000:00010000:2.0:1789760724.048577:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23187456 -> 23191552 00010000:00010000:2.0:1789760724.048600:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2dbf5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23187456->23191551] (req 23187456->23191551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.048613:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2dbf5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23187456->23191551] (req 23187456->23191551) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.048622:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2dbf5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23187456->23191551] (req 23187456->23191551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.048630:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760724.048635:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2dbf5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23187456->23191551] (req 23187456->23191551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.048643:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2dbf5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23187456->23191551] (req 23187456->23191551) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.048663:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2dbf5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23187456->23191551] (req 23187456->23191551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.048670:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2dbf5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23187456->23191551] (req 23187456->23191551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760724.048892:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febfb895880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512583680:12345-192.168.203.56@tcp:4:dd.0 Request processed in 77798us (78362us total) trans 4294972980 rc 0/0 00000100:00100000:2.0:1789760724.048903:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5691 00000100:00100000:2.0:1789760724.048913:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 703473285812 time 703535377609 state 0x2 ref 0 00000100:00100000:0.0:1789760724.056774:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760724.056905:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512583808 00000100:00100000:2.0:1789760724.056919:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512583808 00000100:00100000:2.0:1789760724.056983:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 703473285812 time 703535377609 state 0x0 ref 1 00000100:00100000:2.0:1789760724.056999:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 703535638025 time 703535377609 state 0x1 ref 1 00000100:00100000:2.0:1789760724.057034:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 703535638025 time 703621333521 state 0x0 ref 1 00000100:00100000:2.0:1789760724.057047:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac3352300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512583808:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760724.057095:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23191552, mtime: 0x6aad94d3, atime: 0x0, ctime: 0x6aad94d3, blocks: 0xb807 00010000:00010000:2.0:1789760724.057103:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2dbfc lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.057118:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=703s v=(0 0) 00010000:00010000:2.0:1789760724.057122:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760724.057128:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2dbfc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23191552->23195647] (req 23191552->23195647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.057141:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2dbfc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23191552->23195647] (req 23191552->23195647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760724.127160:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23191552 -> 23195648 00010000:00010000:2.0:1789760724.127186:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2dbfc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23191552->23195647] (req 23191552->23195647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.127200:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2dbfc lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23191552->23195647] (req 23191552->23195647) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.127211:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2dbfc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23191552->23195647] (req 23191552->23195647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.127219:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760724.127225:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2dbfc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23191552->23195647] (req 23191552->23195647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.127234:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2dbfc lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23191552->23195647] (req 23191552->23195647) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.127259:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2dbfc lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23191552->23195647] (req 23191552->23195647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.127268:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2dbfc lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23191552->23195647] (req 23191552->23195647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760724.127390:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac3352300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2359:x1876699512583808:12345-192.168.203.56@tcp:4:dd.0 Request processed in 70337us (70623us total) trans 4294972981 rc 0/0 00000100:00100000:2.0:1789760724.127407:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5692 00000100:00100000:2.0:1789760724.127419:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 703535638025 time 703621333521 state 0x2 ref 0 00000001:00080000:3.0:1789760724.128114:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972981 is committed 00000100:00100000:3.0:1789760724.131939:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760724.132024:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512583936 00000100:00100000:1.0:1789760724.132047:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512583936 00000100:00100000:1.0:1789760724.132121:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 703535638025 time 703621333521 state 0x0 ref 1 00000100:00100000:1.0:1789760724.132154:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 703621593937 time 703621333521 state 0x1 ref 1 00000100:00100000:1.0:1789760724.132186:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 703621593937 time 703696485000 state 0x0 ref 1 00000100:00100000:1.0:1789760724.132201:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2745500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512583936:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760724.132269:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23195648, mtime: 0x6aad94d3, atime: 0x0, ctime: 0x6aad94d3, blocks: 0xb807 00010000:00010000:1.0:1789760724.132279:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2dc03 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760724.132298:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=703s v=(0 0) 00010000:00010000:1.0:1789760724.132304:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760724.132334:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2dc03 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23195648->23199743] (req 23195648->23199743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760724.132353:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2dc03 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23195648->23199743] (req 23195648->23199743) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760724.188411:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972982 is committed 00002000:00010000:1.0:1789760724.188744:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23195648 -> 23199744 00010000:00010000:1.0:1789760724.188766:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2dc03 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23195648->23199743] (req 23195648->23199743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760724.188781:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2dc03 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23195648->23199743] (req 23195648->23199743) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760724.188792:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2dc03 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23195648->23199743] (req 23195648->23199743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760724.188806:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760724.188813:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2dc03 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23195648->23199743] (req 23195648->23199743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760724.188824:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2dc03 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23195648->23199743] (req 23195648->23199743) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760724.188849:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2dc03 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23195648->23199743] (req 23195648->23199743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760724.188858:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2dc03 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23195648->23199743] (req 23195648->23199743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760724.188983:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2745500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512583936:12345-192.168.203.56@tcp:4:dd.0 Request processed in 56778us (57282us total) trans 4294972982 rc 0/0 00000100:00100000:1.0:1789760724.188995:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5693 00000100:00100000:1.0:1789760724.189008:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 703621593937 time 703696485000 state 0x2 ref 0 00000100:00100000:0.0:1789760724.194362:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760724.194484:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512584064 00000100:00100000:2.0:1789760724.194506:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512584064 00000100:00100000:2.0:1789760724.194635:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 703621593937 time 703696485000 state 0x0 ref 1 00000100:00100000:2.0:1789760724.194683:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 703696745416 time 703696485000 state 0x1 ref 1 00000100:00100000:2.0:1789760724.194817:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 703696745416 time 703759116018 state 0x0 ref 1 00000100:00100000:2.0:1789760724.194833:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac3353800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512584064:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760724.194901:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23199744, mtime: 0x6aad94d3, atime: 0x0, ctime: 0x6aad94d3, blocks: 0xb807 00010000:00010000:2.0:1789760724.194912:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2dc0a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.194932:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=703s v=(0 0) 00010000:00010000:2.0:1789760724.194937:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760724.194945:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2dc0a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23199744->23203839] (req 23199744->23203839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.194962:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2dc0a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23199744->23203839] (req 23199744->23203839) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760724.229426:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760724.229502:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760724.229596:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760724.232381:0:5848:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512584320 00000100:00100000:0.0:1789760724.232402:0:5848:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512584320 00000100:00100000:0.0:1789760724.232505:0:5848:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512584448 00000100:00100000:0.0:1789760724.232509:0:5848:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512584448 00000100:00100000:0.0:1789760724.232543:0:5848:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 454 00000100:00100000:0.0:1789760724.232554:0:5848:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2747480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+8:2361:x1876699512584320:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:0.0:1789760724.232676:0:5848:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2747480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+8:2361:x1876699512584320:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 122us (3264us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760724.232685:0:5848:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 454 00000100:00100000:0.0:1789760724.232747:0:5848:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 455 00000100:00100000:0.0:1789760724.232755:0:5848:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2744700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:d2c40150-e97b-40da-952b-cb9f18bf7e96+5:2361:x1876699512584448:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:0.0:1789760724.232783:0:5848:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2744700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:d2c40150-e97b-40da-952b-cb9f18bf7e96+5:2361:x1876699512584448:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 29us (3188us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760724.232790:0:5848:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 455 00000100:00080000:3.0:1789760724.233329:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789760724.236888:0:5842:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512584192 00000100:00100000:0.0:1789760724.236909:0:5842:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512584192 00000100:00100000:0.0:1789760724.236991:0:5842:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 390 00000100:00100000:0.0:1789760724.237006:0:5842:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febe0501450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:e6cf7267-9b19-462a-a263-14115df15dab+13:2361:x1876699512584192:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:0.0:1789760724.237328:0:5842:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febe0501450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:e6cf7267-9b19-462a-a263-14115df15dab+13:2361:x1876699512584192:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 324us (7822us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760724.237339:0:5842:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 390 00000001:00080000:3.0:1789760724.259818:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972983 is committed 00002000:00010000:2.0:1789760724.260262:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23199744 -> 23203840 00010000:00010000:2.0:1789760724.260282:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2dc0a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23199744->23203839] (req 23199744->23203839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.260298:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2dc0a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23199744->23203839] (req 23199744->23203839) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.260310:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2dc0a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23199744->23203839] (req 23199744->23203839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.260320:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760724.260326:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2dc0a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23199744->23203839] (req 23199744->23203839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.260337:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2dc0a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23199744->23203839] (req 23199744->23203839) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.260363:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2dc0a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23199744->23203839] (req 23199744->23203839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.260372:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2dc0a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23199744->23203839] (req 23199744->23203839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760724.260973:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac3353800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512584064:12345-192.168.203.56@tcp:4:dd.0 Request processed in 66132us (66616us total) trans 4294972983 rc 0/0 00000100:00100000:2.0:1789760724.260987:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5694 00000100:00100000:2.0:1789760724.261000:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 703696745416 time 703759116018 state 0x2 ref 0 00000100:00100000:0.0:1789760724.265102:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760724.265205:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512584576 00000100:00100000:2.0:1789760724.265219:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512584576 00000100:00100000:2.0:1789760724.265290:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 703696745416 time 703759116018 state 0x0 ref 1 00000100:00100000:2.0:1789760724.265310:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 703759376434 time 703759116018 state 0x1 ref 1 00000100:00100000:2.0:1789760724.265345:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 703759376434 time 703829643712 state 0x0 ref 1 00000100:00100000:2.0:1789760724.265360:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac3352a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512584576:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760724.265425:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23203840, mtime: 0x6aad94d3, atime: 0x0, ctime: 0x6aad94d3, blocks: 0xb807 00010000:00010000:2.0:1789760724.265436:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2dc11 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.265478:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=703s v=(0 0) 00010000:00010000:2.0:1789760724.265484:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760724.265494:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2dc11 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23203840->23207935] (req 23203840->23207935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.265511:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2dc11 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23203840->23207935] (req 23203840->23207935) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760724.301718:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972984 is committed 00002000:00010000:2.0:1789760724.301870:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23203840 -> 23207936 00010000:00010000:2.0:1789760724.301890:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2dc11 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23203840->23207935] (req 23203840->23207935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.301906:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2dc11 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23203840->23207935] (req 23203840->23207935) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.301921:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2dc11 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23203840->23207935] (req 23203840->23207935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.301928:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760724.301935:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2dc11 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23203840->23207935] (req 23203840->23207935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.301945:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2dc11 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23203840->23207935] (req 23203840->23207935) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.301971:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2dc11 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23203840->23207935] (req 23203840->23207935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.301980:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2dc11 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23203840->23207935] (req 23203840->23207935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760724.302197:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac3352a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512584576:12345-192.168.203.56@tcp:4:dd.0 Request processed in 36774us (37045us total) trans 4294972984 rc 0/0 00000100:00100000:2.0:1789760724.302211:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5695 00000100:00100000:2.0:1789760724.302223:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 703759376434 time 703829643712 state 0x2 ref 0 00000100:00100000:3.0:1789760724.308960:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760724.309565:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512584704 00000100:00100000:1.0:1789760724.309591:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512584704 00000100:00100000:1.0:1789760724.309674:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 703759376434 time 703829643712 state 0x0 ref 1 00000100:00100000:1.0:1789760724.309694:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 703829904128 time 703829643712 state 0x1 ref 1 00000100:00100000:3.0:1789760724.310046:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 703829904128 time 703874340620 state 0x0 ref 1 00000100:00100000:3.0:1789760724.310076:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2746a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512584704:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760724.310157:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23207936, mtime: 0x6aad94d3, atime: 0x0, ctime: 0x6aad94d3, blocks: 0xb807 00010000:00010000:3.0:1789760724.310169:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2dc18 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.310189:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=703s v=(0 0) 00010000:00010000:3.0:1789760724.310194:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760724.310202:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2dc18 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23207936->23212031] (req 23207936->23212031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.310219:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2dc18 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23207936->23212031] (req 23207936->23212031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.339244:0:5835:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:0.0:1789760724.361491:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972985 is committed 00002000:00010000:3.0:1789760724.361651:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23207936 -> 23212032 00010000:00010000:3.0:1789760724.361673:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2dc18 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23207936->23212031] (req 23207936->23212031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.361688:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2dc18 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23207936->23212031] (req 23207936->23212031) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.361733:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2dc18 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23207936->23212031] (req 23207936->23212031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.361743:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760724.361748:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2dc18 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23207936->23212031] (req 23207936->23212031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.361759:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2dc18 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23207936->23212031] (req 23207936->23212031) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.361786:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2dc18 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23207936->23212031] (req 23207936->23212031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.361795:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2dc18 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23207936->23212031] (req 23207936->23212031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760724.361908:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2746a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512584704:12345-192.168.203.56@tcp:4:dd.0 Request processed in 51842us (52955us total) trans 4294972985 rc 0/0 00000100:00100000:3.0:1789760724.361920:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5696 00000100:00100000:3.0:1789760724.361932:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 703829904128 time 703874340620 state 0x2 ref 0 00000100:00100000:0.0:1789760724.367010:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760724.367156:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512584832 00000100:00100000:2.0:1789760724.367169:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512584832 00000100:00100000:2.0:1789760724.367243:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 703829904128 time 703874340620 state 0x0 ref 1 00000100:00100000:2.0:1789760724.367262:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 703874601036 time 703874340620 state 0x1 ref 1 00000100:00100000:2.0:1789760724.367295:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 703874601036 time 703931594710 state 0x0 ref 1 00000100:00100000:2.0:1789760724.367309:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac3353480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512584832:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760724.367380:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23212032, mtime: 0x6aad94d3, atime: 0x0, ctime: 0x6aad94d3, blocks: 0xb807 00010000:00010000:2.0:1789760724.367401:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2dc1f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.367421:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=703s v=(0 0) 00010000:00010000:2.0:1789760724.367426:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760724.367434:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2dc1f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23212032->23216127] (req 23212032->23216127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.367450:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2dc1f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23212032->23216127] (req 23212032->23216127) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760724.401367:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972986 is committed 00002000:00010000:2.0:1789760724.401637:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23212032 -> 23216128 00010000:00010000:2.0:1789760724.401652:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2dc1f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23212032->23216127] (req 23212032->23216127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.401665:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2dc1f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23212032->23216127] (req 23212032->23216127) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.401674:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2dc1f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23212032->23216127] (req 23212032->23216127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.401682:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760724.401687:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2dc1f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23212032->23216127] (req 23212032->23216127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.401713:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2dc1f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23212032->23216127] (req 23212032->23216127) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.401734:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2dc1f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23212032->23216127] (req 23212032->23216127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.401742:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2dc1f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23212032->23216127] (req 23212032->23216127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760724.401835:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac3353480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512584832:12345-192.168.203.56@tcp:4:dd.0 Request processed in 34524us (34836us total) trans 4294972986 rc 0/0 00000100:00100000:2.0:1789760724.401846:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5697 00000100:00100000:2.0:1789760724.401856:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 703874601036 time 703931594710 state 0x2 ref 0 00000100:00100000:0.0:1789760724.417675:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760724.418020:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512584960 00000100:00100000:2.0:1789760724.418030:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512584960 00000100:00100000:2.0:1789760724.418074:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 703874601036 time 703931594710 state 0x0 ref 1 00000100:00100000:2.0:1789760724.418086:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 703931855126 time 703931594710 state 0x1 ref 1 00000100:00100000:2.0:1789760724.418108:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 703931855126 time 703982408890 state 0x0 ref 1 00000100:00100000:2.0:1789760724.418116:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac38b6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512584960:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760724.418155:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23216128, mtime: 0x6aad94d3, atime: 0x0, ctime: 0x6aad94d3, blocks: 0xb807 00010000:00010000:2.0:1789760724.418162:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2dc26 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.418184:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=703s v=(0 0) 00010000:00010000:2.0:1789760724.418188:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760724.418193:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2dc26 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23216128->23220223] (req 23216128->23220223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.418203:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2dc26 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23216128->23220223] (req 23216128->23220223) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760724.453966:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972987 is committed 00002000:00010000:1.0:1789760724.455973:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23216128 -> 23220224 00010000:00010000:1.0:1789760724.455993:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2dc26 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23216128->23220223] (req 23216128->23220223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760724.456006:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2dc26 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23216128->23220223] (req 23216128->23220223) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760724.456015:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2dc26 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23216128->23220223] (req 23216128->23220223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760724.456022:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760724.456027:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2dc26 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23216128->23220223] (req 23216128->23220223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760724.456036:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2dc26 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23216128->23220223] (req 23216128->23220223) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760724.456057:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2dc26 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23216128->23220223] (req 23216128->23220223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760724.456063:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2dc26 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23216128->23220223] (req 23216128->23220223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760724.456318:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac38b6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512584960:12345-192.168.203.56@tcp:4:dd.0 Request processed in 38195us (38646us total) trans 4294972987 rc 0/0 00000100:00100000:1.0:1789760724.456328:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5698 00000100:00100000:1.0:1789760724.456338:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 703931855126 time 703982408890 state 0x2 ref 0 00000100:00100000:3.0:1789760724.460028:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760724.460320:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512585088 00000100:00100000:0.0:1789760724.460331:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512585088 00000100:00100000:0.0:1789760724.460388:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 703931855126 time 703982408890 state 0x0 ref 1 00000100:00100000:0.0:1789760724.460405:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 703982669306 time 703982408890 state 0x1 ref 1 00000100:00100000:0.0:1789760724.460435:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 703982669306 time 704024734951 state 0x0 ref 1 00000100:00100000:0.0:1789760724.460449:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2745880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512585088:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760724.460508:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23220224, mtime: 0x6aad94d3, atime: 0x0, ctime: 0x6aad94d3, blocks: 0xb807 00010000:00010000:0.0:1789760724.460518:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2dc2d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.460535:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=704s v=(0 0) 00010000:00010000:0.0:1789760724.460539:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760724.460545:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2dc2d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23220224->23224319] (req 23220224->23224319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.460557:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2dc2d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23220224->23224319] (req 23220224->23224319) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760724.511620:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972988 is committed 00002000:00010000:1.0:1789760724.512232:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23220224 -> 23224320 00010000:00010000:1.0:1789760724.512254:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2dc2d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23220224->23224319] (req 23220224->23224319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760724.512269:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2dc2d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23220224->23224319] (req 23220224->23224319) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760724.512279:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2dc2d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23220224->23224319] (req 23220224->23224319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760724.512286:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760724.512291:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2dc2d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23220224->23224319] (req 23220224->23224319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760724.512300:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2dc2d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23220224->23224319] (req 23220224->23224319) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760724.512324:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2dc2d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23220224->23224319] (req 23220224->23224319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760724.512332:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2dc2d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23220224->23224319] (req 23220224->23224319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760724.512472:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2745880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512585088:12345-192.168.203.56@tcp:4:dd.0 Request processed in 52008us (52434us total) trans 4294972988 rc 0/0 00000100:00100000:1.0:1789760724.512484:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5699 00000100:00100000:1.0:1789760724.512496:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 703982669306 time 704024734951 state 0x2 ref 0 00000100:00100000:0.0:1789760724.518298:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760724.518390:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512585216 00000100:00100000:2.0:1789760724.518408:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512585216 00000100:00100000:2.0:1789760724.518495:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 703982669306 time 704024734951 state 0x0 ref 1 00000100:00100000:2.0:1789760724.518514:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 704024995367 time 704024734951 state 0x1 ref 1 00000100:00100000:2.0:1789760724.518547:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 704024995367 time 704082846923 state 0x0 ref 1 00000100:00100000:2.0:1789760724.518560:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2fb9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512585216:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760724.518629:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23224320, mtime: 0x6aad94d3, atime: 0x0, ctime: 0x6aad94d3, blocks: 0xb807 00010000:00010000:2.0:1789760724.518641:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2dc34 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.518661:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=704s v=(0 0) 00010000:00010000:2.0:1789760724.518667:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760724.518675:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2dc34 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23224320->23228415] (req 23224320->23228415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.518693:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2dc34 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23224320->23228415] (req 23224320->23228415) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760724.592627:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972989 is committed 00002000:00010000:2.0:1789760724.592836:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23224320 -> 23228416 00002000:00010000:2.0:1789760724.592841:0:6708:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760723 -> 1789760724 00002000:00010000:2.0:1789760724.592844:0:6708:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760723 -> 1789760724 00010000:00010000:2.0:1789760724.592873:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2dc34 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23224320->23228415] (req 23224320->23228415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.592889:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2dc34 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23224320->23228415] (req 23224320->23228415) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.592899:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2dc34 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23224320->23228415] (req 23224320->23228415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.592906:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760724.592912:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2dc34 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23224320->23228415] (req 23224320->23228415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.592921:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2dc34 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23224320->23228415] (req 23224320->23228415) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.592946:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2dc34 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23224320->23228415] (req 23224320->23228415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.592959:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2dc34 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23224320->23228415] (req 23224320->23228415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760724.593090:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2fb9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512585216:12345-192.168.203.56@tcp:4:dd.0 Request processed in 74525us (74793us total) trans 4294972989 rc 0/0 00000100:00100000:2.0:1789760724.593102:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5700 00000100:00100000:2.0:1789760724.593115:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 704024995367 time 704082846923 state 0x2 ref 0 00000100:00100000:3.0:1789760724.597178:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760724.598805:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512585344 00000100:00100000:1.0:1789760724.598828:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512585344 00000100:00100000:1.0:1789760724.599246:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 704024995367 time 704082846923 state 0x0 ref 1 00000100:00100000:1.0:1789760724.599268:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 704083107339 time 704082846923 state 0x1 ref 1 00000100:00100000:1.0:1789760724.599362:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 704083107339 time 704163661154 state 0x0 ref 1 00000100:00100000:1.0:1789760724.599390:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2746300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512585344:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760724.599467:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23228416, mtime: 0x6aad94d4, atime: 0x0, ctime: 0x6aad94d4, blocks: 0xb807 00010000:00010000:1.0:1789760724.599479:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2dc3b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760724.599499:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=704s v=(0 0) 00010000:00010000:1.0:1789760724.599503:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760724.599511:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2dc3b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23228416->23232511] (req 23228416->23232511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760724.599528:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2dc3b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23228416->23232511] (req 23228416->23232511) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760724.653935:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23228416 -> 23232512 00010000:00010000:1.0:1789760724.653964:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2dc3b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23228416->23232511] (req 23228416->23232511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760724.653987:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2dc3b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23228416->23232511] (req 23228416->23232511) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760724.653997:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2dc3b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23228416->23232511] (req 23228416->23232511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760724.654005:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760724.654011:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2dc3b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23228416->23232511] (req 23228416->23232511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760724.654022:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2dc3b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23228416->23232511] (req 23228416->23232511) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760724.654045:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2dc3b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23228416->23232511] (req 23228416->23232511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760724.654053:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2dc3b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23228416->23232511] (req 23228416->23232511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760724.654286:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972990 is committed 00000100:00100000:1.0:1789760724.654346:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2746300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512585344:12345-192.168.203.56@tcp:4:dd.0 Request processed in 54963us (57176us total) trans 4294972990 rc 0/0 00000100:00100000:1.0:1789760724.654360:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5701 00000100:00100000:1.0:1789760724.654373:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 704083107339 time 704163661154 state 0x2 ref 0 00000100:00100000:2.0:1789760724.658066:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760724.658215:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512585472 00000100:00100000:1.0:1789760724.658232:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512585472 00000100:00100000:1.0:1789760724.658314:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 704083107339 time 704163661154 state 0x0 ref 1 00000100:00100000:1.0:1789760724.658335:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 704163921570 time 704163661154 state 0x1 ref 1 00000100:00100000:1.0:1789760724.658412:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 704163921570 time 704222711061 state 0x0 ref 1 00000100:00100000:1.0:1789760724.658427:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febd02e0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512585472:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760724.658508:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23232512, mtime: 0x6aad94d4, atime: 0x0, ctime: 0x6aad94d4, blocks: 0xb807 00010000:00010000:1.0:1789760724.658519:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2dc42 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760724.658538:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=704s v=(0 0) 00010000:00010000:1.0:1789760724.658544:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760724.658551:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2dc42 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23232512->23236607] (req 23232512->23236607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760724.658568:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2dc42 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23232512->23236607] (req 23232512->23236607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760724.695212:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23232512 -> 23236608 00010000:00010000:1.0:1789760724.695242:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2dc42 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23232512->23236607] (req 23232512->23236607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760724.695258:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2dc42 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23232512->23236607] (req 23232512->23236607) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760724.695273:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2dc42 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23232512->23236607] (req 23232512->23236607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760724.695281:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760724.695287:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2dc42 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23232512->23236607] (req 23232512->23236607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760724.695297:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2dc42 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23232512->23236607] (req 23232512->23236607) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760724.695322:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2dc42 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23232512->23236607] (req 23232512->23236607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760724.695332:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2dc42 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23232512->23236607] (req 23232512->23236607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760724.695467:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febd02e0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2359:x1876699512585472:12345-192.168.203.56@tcp:4:dd.0 Request processed in 37026us (37403us total) trans 4294972991 rc 0/0 00000100:00100000:1.0:1789760724.695481:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5702 00000100:00100000:1.0:1789760724.695493:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 704163921570 time 704222711061 state 0x2 ref 0 00000001:00080000:0.0:1789760724.698673:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972991 is committed 00000100:00100000:0.0:1789760724.699685:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760724.699940:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512585600 00000100:00100000:2.0:1789760724.699953:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512585600 00000100:00100000:2.0:1789760724.700025:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 704163921570 time 704222711061 state 0x0 ref 1 00000100:00100000:2.0:1789760724.700043:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 704222971477 time 704222711061 state 0x1 ref 1 00000100:00100000:2.0:1789760724.700075:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 704222971477 time 704264374686 state 0x0 ref 1 00000100:00100000:2.0:1789760724.700088:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2fb8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512585600:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760724.700255:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23236608, mtime: 0x6aad94d4, atime: 0x0, ctime: 0x6aad94d4, blocks: 0xb807 00010000:00010000:2.0:1789760724.700266:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2dc49 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.700286:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=704s v=(0 0) 00010000:00010000:2.0:1789760724.700291:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760724.700298:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2dc49 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23236608->23240703] (req 23236608->23240703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.700316:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2dc49 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23236608->23240703] (req 23236608->23240703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760724.754292:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972992 is committed 00002000:00010000:3.0:1789760724.754488:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23236608 -> 23240704 00010000:00010000:3.0:1789760724.754510:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2dc49 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23236608->23240703] (req 23236608->23240703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.754526:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2dc49 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23236608->23240703] (req 23236608->23240703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.754535:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2dc49 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23236608->23240703] (req 23236608->23240703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.754542:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760724.754550:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2dc49 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23236608->23240703] (req 23236608->23240703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.754560:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2dc49 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23236608->23240703] (req 23236608->23240703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.754584:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2dc49 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23236608->23240703] (req 23236608->23240703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.754592:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2dc49 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23236608->23240703] (req 23236608->23240703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760724.754771:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2fb8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512585600:12345-192.168.203.56@tcp:4:dd.0 Request processed in 54669us (55079us total) trans 4294972992 rc 0/0 00000100:00100000:3.0:1789760724.754785:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5703 00000100:00100000:3.0:1789760724.754797:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 704222971477 time 704264374686 state 0x2 ref 0 00000100:00100000:3.0:1789760724.758101:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760724.758668:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512585728 00000100:00100000:1.0:1789760724.758684:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512585728 00000100:00100000:1.0:1789760724.759953:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 704222971477 time 704264374686 state 0x0 ref 1 00000100:00100000:1.0:1789760724.759980:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 704264635102 time 704264374686 state 0x1 ref 1 00000100:00100000:1.0:1789760724.760014:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 704264635102 time 704324313768 state 0x0 ref 1 00000100:00100000:1.0:1789760724.760031:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2745180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512585728:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760724.760098:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23240704, mtime: 0x6aad94d4, atime: 0x0, ctime: 0x6aad94d4, blocks: 0xb807 00010000:00010000:1.0:1789760724.760110:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2dc50 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760724.760202:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=704s v=(0 0) 00010000:00010000:1.0:1789760724.760210:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760724.760219:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2dc50 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23240704->23244799] (req 23240704->23244799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760724.760238:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2dc50 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23240704->23244799] (req 23240704->23244799) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760724.811059:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972993 is committed 00002000:00010000:1.0:1789760724.811605:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23240704 -> 23244800 00010000:00010000:1.0:1789760724.811628:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2dc50 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23240704->23244799] (req 23240704->23244799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760724.811644:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2dc50 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23240704->23244799] (req 23240704->23244799) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760724.811654:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2dc50 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23240704->23244799] (req 23240704->23244799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760724.811668:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760724.811674:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2dc50 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23240704->23244799] (req 23240704->23244799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760724.811683:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2dc50 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23240704->23244799] (req 23240704->23244799) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760724.811837:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2dc50 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23240704->23244799] (req 23240704->23244799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760724.811904:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2dc50 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23240704->23244799] (req 23240704->23244799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760724.812072:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2745180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512585728:12345-192.168.203.56@tcp:4:dd.0 Request processed in 52038us (53978us total) trans 4294972993 rc 0/0 00000100:00100000:1.0:1789760724.812085:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5704 00000100:00100000:1.0:1789760724.812098:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 704264635102 time 704324313768 state 0x2 ref 0 00000100:00100000:2.0:1789760724.817020:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760724.817188:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512585856 00000100:00100000:1.0:1789760724.817201:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512585856 00000100:00100000:1.0:1789760724.817275:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 704264635102 time 704324313768 state 0x0 ref 1 00000100:00100000:1.0:1789760724.817293:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 704324574184 time 704324313768 state 0x1 ref 1 00000100:00100000:1.0:1789760724.817371:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 704324574184 time 704381669049 state 0x0 ref 1 00000100:00100000:1.0:1789760724.817390:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febea899180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512585856:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760724.817457:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23244800, mtime: 0x6aad94d4, atime: 0x0, ctime: 0x6aad94d4, blocks: 0xb807 00010000:00010000:1.0:1789760724.817479:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2dc57 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760724.817500:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=704s v=(0 0) 00010000:00010000:1.0:1789760724.817505:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760724.817514:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2dc57 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23244800->23248895] (req 23244800->23248895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760724.817530:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2dc57 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23244800->23248895] (req 23244800->23248895) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760724.872173:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23244800 -> 23248896 00010000:00010000:2.0:1789760724.872203:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2dc57 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23244800->23248895] (req 23244800->23248895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.872220:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2dc57 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23244800->23248895] (req 23244800->23248895) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.872230:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2dc57 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23244800->23248895] (req 23244800->23248895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.872239:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760724.872245:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2dc57 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23244800->23248895] (req 23244800->23248895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.872255:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2dc57 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23244800->23248895] (req 23244800->23248895) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.872280:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2dc57 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23244800->23248895] (req 23244800->23248895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.872288:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2dc57 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23244800->23248895] (req 23244800->23248895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760724.872420:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febea899180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2358:x1876699512585856:12345-192.168.203.56@tcp:4:dd.0 Request processed in 55026us (55408us total) trans 4294972994 rc 0/0 00000100:00100000:2.0:1789760724.872432:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5705 00000100:00100000:2.0:1789760724.872444:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 704324574184 time 704381669049 state 0x2 ref 0 00000001:00080000:3.0:1789760724.876079:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972994 is committed 00000100:00100000:1.0:1789760724.876410:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760724.876508:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512585984 00000100:00100000:3.0:1789760724.876519:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512585984 00000100:00100000:3.0:1789760724.876587:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 704324574184 time 704381669049 state 0x0 ref 1 00000100:00100000:3.0:1789760724.876605:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 704381929465 time 704381669049 state 0x1 ref 1 00000100:00100000:3.0:1789760724.876672:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 704381929465 time 704440970786 state 0x0 ref 1 00000100:00100000:3.0:1789760724.876690:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4130a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512585984:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760724.876785:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23248896, mtime: 0x6aad94d4, atime: 0x0, ctime: 0x6aad94d4, blocks: 0xb807 00010000:00010000:3.0:1789760724.876796:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2dc5e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.876816:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=704s v=(0 0) 00010000:00010000:3.0:1789760724.876822:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760724.876830:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2dc5e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23248896->23252991] (req 23248896->23252991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.876847:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2dc5e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23248896->23252991] (req 23248896->23252991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760724.913614:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972995 is committed 00002000:00010000:3.0:1789760724.913791:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23248896 -> 23252992 00010000:00010000:3.0:1789760724.913820:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2dc5e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23248896->23252991] (req 23248896->23252991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.913837:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2dc5e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23248896->23252991] (req 23248896->23252991) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.913847:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2dc5e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23248896->23252991] (req 23248896->23252991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.913863:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760724.913869:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2dc5e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23248896->23252991] (req 23248896->23252991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.913879:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2dc5e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23248896->23252991] (req 23248896->23252991) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.913902:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2dc5e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23248896->23252991] (req 23248896->23252991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.913911:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2dc5e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23248896->23252991] (req 23248896->23252991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760724.914030:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4130a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512585984:12345-192.168.203.56@tcp:4:dd.0 Request processed in 37334us (37617us total) trans 4294972995 rc 0/0 00000100:00100000:3.0:1789760724.914042:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5706 00000100:00100000:3.0:1789760724.914054:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 704381929465 time 704440970786 state 0x2 ref 0 00000100:00100000:1.0:1789760724.918594:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760724.922642:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512586112 00000100:00100000:2.0:1789760724.922663:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512586112 00000100:00100000:2.0:1789760724.922928:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 704381929465 time 704440970786 state 0x0 ref 1 00000100:00100000:2.0:1789760724.922949:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 704441231202 time 704440970786 state 0x1 ref 1 00000100:00100000:2.0:1789760724.922990:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 704441231202 time 704487289106 state 0x0 ref 1 00000100:00100000:2.0:1789760724.923006:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4132d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512586112:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760724.923074:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23252992, mtime: 0x6aad94d4, atime: 0x0, ctime: 0x6aad94d4, blocks: 0xb807 00010000:00010000:2.0:1789760724.923085:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2dc65 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.923106:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=704s v=(0 0) 00010000:00010000:2.0:1789760724.923112:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760724.923120:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2dc65 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23252992->23257087] (req 23252992->23257087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.923147:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2dc65 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23252992->23257087] (req 23252992->23257087) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760724.960413:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972996 is committed 00002000:00010000:1.0:1789760724.960530:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23252992 -> 23257088 00010000:00010000:1.0:1789760724.960550:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2dc65 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23252992->23257087] (req 23252992->23257087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760724.960567:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2dc65 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23252992->23257087] (req 23252992->23257087) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760724.960577:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2dc65 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23252992->23257087] (req 23252992->23257087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760724.960587:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760724.960593:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2dc65 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23252992->23257087] (req 23252992->23257087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760724.960605:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2dc65 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23252992->23257087] (req 23252992->23257087) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760724.960630:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2dc65 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23252992->23257087] (req 23252992->23257087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760724.960639:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2dc65 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23252992->23257087] (req 23252992->23257087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760724.960969:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4132d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512586112:12345-192.168.203.56@tcp:4:dd.0 Request processed in 37958us (42383us total) trans 4294972996 rc 0/0 00000100:00100000:0.0:1789760724.960982:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5707 00000100:00100000:0.0:1789760724.960994:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 704441231202 time 704487289106 state 0x2 ref 0 00000100:00100000:0.0:1789760724.964229:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760724.964332:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512586240 00000100:00100000:2.0:1789760724.964344:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512586240 00000100:00100000:2.0:1789760724.964409:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 704441231202 time 704487289106 state 0x0 ref 1 00000100:00100000:2.0:1789760724.964427:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 704487549522 time 704487289106 state 0x1 ref 1 00000100:00100000:2.0:1789760724.964495:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 704487549522 time 704528794788 state 0x0 ref 1 00000100:00100000:2.0:1789760724.964508:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2fba680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512586240:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760724.964570:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23257088, mtime: 0x6aad94d4, atime: 0x0, ctime: 0x6aad94d4, blocks: 0xb807 00010000:00010000:2.0:1789760724.964580:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2dc6c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.964598:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=704s v=(0 0) 00010000:00010000:2.0:1789760724.964603:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760724.964611:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2dc6c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23257088->23261183] (req 23257088->23261183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.964626:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2dc6c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23257088->23261183] (req 23257088->23261183) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760725.007018:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972997 is committed 00002000:00010000:1.0:1789760725.007778:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23257088 -> 23261184 00010000:00010000:1.0:1789760725.007800:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2dc6c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23257088->23261183] (req 23257088->23261183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.007815:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2dc6c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23257088->23261183] (req 23257088->23261183) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.007825:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2dc6c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23257088->23261183] (req 23257088->23261183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.007834:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760725.007840:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2dc6c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23257088->23261183] (req 23257088->23261183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.007860:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2dc6c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23257088->23261183] (req 23257088->23261183) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.007886:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2dc6c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23257088->23261183] (req 23257088->23261183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.007896:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2dc6c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23257088->23261183] (req 23257088->23261183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760725.008010:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2fba680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512586240:12345-192.168.203.56@tcp:4:dd.0 Request processed in 43495us (43787us total) trans 4294972997 rc 0/0 00000100:00100000:1.0:1789760725.008022:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5708 00000100:00100000:1.0:1789760725.008034:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 704487549522 time 704528794788 state 0x2 ref 0 00000100:00100000:1.0:1789760725.012964:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760725.013047:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512586368 00000100:00100000:3.0:1789760725.013069:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512586368 00000100:00100000:3.0:1789760725.013175:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 704487549522 time 704528794788 state 0x0 ref 1 00000100:00100000:3.0:1789760725.013195:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 704529055204 time 704528794788 state 0x1 ref 1 00000100:00100000:3.0:1789760725.013228:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 704529055204 time 704577526861 state 0x0 ref 1 00000100:00100000:3.0:1789760725.013247:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4132300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512586368:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760725.013313:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23261184, mtime: 0x6aad94d4, atime: 0x0, ctime: 0x6aad94d4, blocks: 0xb807 00010000:00010000:3.0:1789760725.013324:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2dc73 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.013348:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=704s v=(0 0) 00010000:00010000:3.0:1789760725.013354:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760725.013363:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2dc73 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23261184->23265279] (req 23261184->23265279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.013380:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2dc73 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23261184->23265279] (req 23261184->23265279) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1789760725.014061:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:0.0:1789760725.048572:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972998 is committed 00002000:00010000:1.0:1789760725.048716:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23261184 -> 23265280 00010000:00010000:1.0:1789760725.048739:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2dc73 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23261184->23265279] (req 23261184->23265279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.048753:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2dc73 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23261184->23265279] (req 23261184->23265279) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.048763:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2dc73 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23261184->23265279] (req 23261184->23265279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.048772:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760725.048778:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2dc73 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23261184->23265279] (req 23261184->23265279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.048793:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2dc73 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23261184->23265279] (req 23261184->23265279) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.048817:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2dc73 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23261184->23265279] (req 23261184->23265279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.048826:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2dc73 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23261184->23265279] (req 23261184->23265279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760725.048963:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4132300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512586368:12345-192.168.203.56@tcp:4:dd.0 Request processed in 35715us (36004us total) trans 4294972998 rc 0/0 00000100:00100000:1.0:1789760725.048975:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5709 00000100:00100000:1.0:1789760725.048987:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 704529055204 time 704577526861 state 0x2 ref 0 00000100:00100000:2.0:1789760725.053357:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760725.053467:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512586496 00000100:00100000:0.0:1789760725.053482:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512586496 00000100:00100000:0.0:1789760725.053563:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 704529055204 time 704577526861 state 0x0 ref 1 00000100:00100000:0.0:1789760725.053583:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 704577787277 time 704577526861 state 0x1 ref 1 00000100:00100000:0.0:1789760725.053652:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 704577787277 time 704617950068 state 0x0 ref 1 00000100:00100000:0.0:1789760725.053669:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febea89aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512586496:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760725.053777:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23265280, mtime: 0x6aad94d4, atime: 0x0, ctime: 0x6aad94d4, blocks: 0xb807 00010000:00010000:0.0:1789760725.053788:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2dc7a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.053808:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=704s v=(0 0) 00010000:00010000:0.0:1789760725.053813:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760725.053821:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2dc7a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23265280->23269375] (req 23265280->23269375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.053837:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2dc7a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23265280->23269375] (req 23265280->23269375) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760725.095934:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23265280 -> 23269376 00010000:00010000:1.0:1789760725.095962:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2dc7a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23265280->23269375] (req 23265280->23269375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.095979:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2dc7a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23265280->23269375] (req 23265280->23269375) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.095988:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2dc7a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23265280->23269375] (req 23265280->23269375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.095996:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760725.096002:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2dc7a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23265280->23269375] (req 23265280->23269375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.096013:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2dc7a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23265280->23269375] (req 23265280->23269375) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.096037:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2dc7a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23265280->23269375] (req 23265280->23269375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.096044:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2dc7a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23265280->23269375] (req 23265280->23269375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760725.096180:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febea89aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2361:x1876699512586496:12345-192.168.203.56@tcp:4:dd.0 Request processed in 42506us (42830us total) trans 4294972999 rc 0/0 00000100:00100000:1.0:1789760725.096193:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5710 00000100:00100000:1.0:1789760725.096205:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 704577787277 time 704617950068 state 0x2 ref 0 00000100:00100000:1.0:1789760725.099566:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760725.099664:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512586624 00000100:00100000:3.0:1789760725.099678:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512586624 00000100:00100000:3.0:1789760725.099790:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 704577787277 time 704617950068 state 0x0 ref 1 00000100:00100000:3.0:1789760725.099811:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 704618210484 time 704617950068 state 0x1 ref 1 00000100:00100000:3.0:1789760725.099839:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 704618210484 time 704664138259 state 0x0 ref 1 00000100:00100000:3.0:1789760725.099863:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4131f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2360:x1876699512586624:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760725.099931:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23269376, mtime: 0x6aad94d4, atime: 0x0, ctime: 0x6aad94d4, blocks: 0xb807 00010000:00010000:3.0:1789760725.099942:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2dc81 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.099964:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=704s v=(0 0) 00010000:00010000:3.0:1789760725.099970:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760725.099978:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2dc81 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23269376->23273471] (req 23269376->23273471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.099999:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2dc81 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23269376->23273471] (req 23269376->23273471) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760725.105331:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972999 is committed 00000001:00080000:0.0:1789760725.145517:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973000 is committed 00002000:00010000:2.0:1789760725.145924:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23269376 -> 23273472 00010000:00010000:2.0:1789760725.145945:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2dc81 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23269376->23273471] (req 23269376->23273471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.145968:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2dc81 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23269376->23273471] (req 23269376->23273471) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.145978:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2dc81 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23269376->23273471] (req 23269376->23273471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.145987:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760725.145993:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2dc81 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23269376->23273471] (req 23269376->23273471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.146004:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2dc81 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23269376->23273471] (req 23269376->23273471) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.146028:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2dc81 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23269376->23273471] (req 23269376->23273471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.146035:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2dc81 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23269376->23273471] (req 23269376->23273471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760725.146292:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4131f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512586624:12345-192.168.203.56@tcp:4:dd.0 Request processed in 46433us (46735us total) trans 4294973000 rc 0/0 00000100:00100000:2.0:1789760725.146306:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5711 00000100:00100000:2.0:1789760725.146319:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 704618210484 time 704664138259 state 0x2 ref 0 00000100:00100000:3.0:1789760725.149473:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760725.149585:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512586752 00000100:00100000:1.0:1789760725.149598:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512586752 00000100:00100000:1.0:1789760725.149673:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 704618210484 time 704664138259 state 0x0 ref 1 00000100:00100000:1.0:1789760725.149693:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 704664398675 time 704664138259 state 0x1 ref 1 00000100:00100000:0.0:1789760725.150008:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 704664398675 time 704714304253 state 0x0 ref 1 00000100:00100000:0.0:1789760725.150031:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2744a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512586752:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760725.150099:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23273472, mtime: 0x6aad94d4, atime: 0x0, ctime: 0x6aad94d4, blocks: 0xb807 00010000:00010000:0.0:1789760725.150109:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2dc88 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.150138:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=704s v=(0 0) 00010000:00010000:0.0:1789760725.150144:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760725.150152:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2dc88 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23273472->23277567] (req 23273472->23277567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.150168:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2dc88 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23273472->23277567] (req 23273472->23277567) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760725.189294:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973001 is committed 00002000:00010000:3.0:1789760725.189422:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23273472 -> 23277568 00010000:00010000:3.0:1789760725.189447:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2dc88 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23273472->23277567] (req 23273472->23277567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.189475:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2dc88 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23273472->23277567] (req 23273472->23277567) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.189485:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2dc88 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23273472->23277567] (req 23273472->23277567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.189493:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760725.189499:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2dc88 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23273472->23277567] (req 23273472->23277567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.189510:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2dc88 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23273472->23277567] (req 23273472->23277567) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.189536:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2dc88 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23273472->23277567] (req 23273472->23277567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.189545:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2dc88 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23273472->23277567] (req 23273472->23277567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760725.190447:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2744a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512586752:12345-192.168.203.56@tcp:4:dd.0 Request processed in 40410us (40984us total) trans 4294973001 rc 0/0 00000100:00100000:3.0:1789760725.190474:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5712 00000100:00100000:3.0:1789760725.190488:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 704664398675 time 704714304253 state 0x2 ref 0 00000100:00100000:0.0:1789760725.196041:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760725.196318:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512586880 00000100:00100000:1.0:1789760725.196330:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512586880 00000100:00100000:1.0:1789760725.196396:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 704664398675 time 704714304253 state 0x0 ref 1 00000100:00100000:1.0:1789760725.196415:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 704714564669 time 704714304253 state 0x1 ref 1 00000100:00100000:1.0:1789760725.196478:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 704714564669 time 704760776879 state 0x0 ref 1 00000100:00100000:1.0:1789760725.196491:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2fb8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512586880:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760725.196550:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23277568, mtime: 0x6aad94d4, atime: 0x0, ctime: 0x6aad94d4, blocks: 0xb807 00010000:00010000:1.0:1789760725.196560:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2dc8f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.196579:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=704s v=(0 0) 00010000:00010000:1.0:1789760725.196584:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760725.196591:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2dc8f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23277568->23281663] (req 23277568->23281663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.196608:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2dc8f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23277568->23281663] (req 23277568->23281663) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760725.228063:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973002 is committed 00002000:00010000:0.0:1789760725.228104:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23277568 -> 23281664 00010000:00010000:0.0:1789760725.228124:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2dc8f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23277568->23281663] (req 23277568->23281663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.228139:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2dc8f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23277568->23281663] (req 23277568->23281663) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.228150:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2dc8f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23277568->23281663] (req 23277568->23281663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.228158:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760725.228164:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2dc8f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23277568->23281663] (req 23277568->23281663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.228176:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2dc8f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23277568->23281663] (req 23277568->23281663) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.228202:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2dc8f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23277568->23281663] (req 23277568->23281663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.228211:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2dc8f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23277568->23281663] (req 23277568->23281663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760725.228334:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2fb8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512586880:12345-192.168.203.56@tcp:4:dd.0 Request processed in 31838us (32299us total) trans 4294973002 rc 0/0 00000100:00100000:0.0:1789760725.228346:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5713 00000100:00100000:0.0:1789760725.228358:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 704714564669 time 704760776879 state 0x2 ref 0 00000100:00100000:1.0:1789760725.232365:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760725.232539:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512587008 00000100:00100000:2.0:1789760725.232555:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512587008 00000100:00100000:2.0:1789760725.232629:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 704714564669 time 704760776879 state 0x0 ref 1 00000100:00100000:2.0:1789760725.232662:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 704761037295 time 704760776879 state 0x1 ref 1 00000100:00100000:2.0:1789760725.233068:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 704761037295 time 704797364156 state 0x0 ref 1 00000100:00100000:2.0:1789760725.233086:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4130380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512587008:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760725.233152:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23281664, mtime: 0x6aad94d4, atime: 0x0, ctime: 0x6aad94d4, blocks: 0xb807 00010000:00010000:2.0:1789760725.233163:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2dc96 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.233182:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=704s v=(0 0) 00010000:00010000:2.0:1789760725.233187:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760725.233195:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2dc96 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23281664->23285759] (req 23281664->23285759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.234570:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2dc96 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23281664->23285759] (req 23281664->23285759) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760725.295576:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23281664 -> 23285760 00010000:00010000:3.0:1789760725.295604:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2dc96 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23281664->23285759] (req 23281664->23285759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.295621:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2dc96 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23281664->23285759] (req 23281664->23285759) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.295632:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2dc96 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23281664->23285759] (req 23281664->23285759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.295640:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760725.295646:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2dc96 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23281664->23285759] (req 23281664->23285759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.295655:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2dc96 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23281664->23285759] (req 23281664->23285759) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.295680:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2dc96 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23281664->23285759] (req 23281664->23285759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.295727:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2dc96 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23281664->23285759] (req 23281664->23285759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760725.295854:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4130380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2361:x1876699512587008:12345-192.168.203.56@tcp:4:dd.0 Request processed in 62758us (63490us total) trans 4294973003 rc 0/0 00000100:00100000:3.0:1789760725.295867:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5714 00000100:00100000:3.0:1789760725.295879:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 704761037295 time 704797364156 state 0x2 ref 0 00000001:00080000:0.0:1789760725.297837:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973003 is committed 00000100:00100000:3.0:1789760725.299048:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760725.299851:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512587136 00000100:00100000:0.0:1789760725.299867:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512587136 00000100:00100000:1.0:1789760725.300007:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 704761037295 time 704797364156 state 0x0 ref 1 00000100:00100000:1.0:1789760725.300026:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 704797624572 time 704797364156 state 0x1 ref 1 00000100:00100000:1.0:1789760725.300068:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 704797624572 time 704864367493 state 0x0 ref 1 00000100:00100000:1.0:1789760725.300082:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2746d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512587136:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760725.300159:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23285760, mtime: 0x6aad94d4, atime: 0x0, ctime: 0x6aad94d4, blocks: 0xb807 00010000:00010000:1.0:1789760725.300170:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2dc9d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.300189:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=704s v=(0 0) 00010000:00010000:1.0:1789760725.300193:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760725.300201:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2dc9d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23285760->23289855] (req 23285760->23289855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.300219:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2dc9d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23285760->23289855] (req 23285760->23289855) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760725.328400:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23285760 -> 23289856 00010000:00010000:2.0:1789760725.328429:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2dc9d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23285760->23289855] (req 23285760->23289855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.328445:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2dc9d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23285760->23289855] (req 23285760->23289855) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.328474:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2dc9d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23285760->23289855] (req 23285760->23289855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.328483:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760725.328488:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2dc9d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23285760->23289855] (req 23285760->23289855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.328497:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2dc9d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23285760->23289855] (req 23285760->23289855) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.328521:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2dc9d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23285760->23289855] (req 23285760->23289855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.328530:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2dc9d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23285760->23289855] (req 23285760->23289855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760725.328653:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2746d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2360:x1876699512587136:12345-192.168.203.56@tcp:4:dd.0 Request processed in 28566us (29609us total) trans 4294973004 rc 0/0 00000100:00100000:2.0:1789760725.328665:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5715 00000100:00100000:2.0:1789760725.328676:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 704797624572 time 704864367493 state 0x2 ref 0 00000001:00080000:0.0:1789760725.328987:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973004 is committed 00000100:00100000:2.0:1789760725.337493:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760725.337559:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512587264 00000100:00100000:0.0:1789760725.337573:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512587264 00000100:00100000:0.0:1789760725.337651:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 704797624572 time 704864367493 state 0x0 ref 1 00000100:00100000:0.0:1789760725.337670:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 704864627909 time 704864367493 state 0x1 ref 1 00000100:00100000:3.0:1789760725.337739:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 704864627909 time 704902034934 state 0x0 ref 1 00000100:00100000:3.0:1789760725.337760:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febea89b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512587264:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760725.337895:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23289856, mtime: 0x6aad94d4, atime: 0x0, ctime: 0x6aad94d4, blocks: 0xb807 00010000:00010000:2.0:1789760725.337917:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2dca4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.337946:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=704s v=(0 0) 00010000:00010000:2.0:1789760725.337952:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760725.337961:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2dca4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23289856->23293951] (req 23289856->23293951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.337978:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2dca4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23289856->23293951] (req 23289856->23293951) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.363360:0:7647:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:0.0:1789760725.400970:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973005 is committed 00002000:00010000:2.0:1789760725.401175:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23289856 -> 23293952 00010000:00010000:2.0:1789760725.401198:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2dca4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23289856->23293951] (req 23289856->23293951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.401215:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2dca4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23289856->23293951] (req 23289856->23293951) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.401226:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2dca4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23289856->23293951] (req 23289856->23293951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.401236:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760725.401243:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2dca4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23289856->23293951] (req 23289856->23293951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.401253:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2dca4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23289856->23293951] (req 23289856->23293951) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.401277:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2dca4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23289856->23293951] (req 23289856->23293951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.401286:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2dca4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23289856->23293951] (req 23289856->23293951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760725.401410:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febea89b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512587264:12345-192.168.203.56@tcp:4:dd.0 Request processed in 63650us (63923us total) trans 4294973005 rc 0/0 00000100:00100000:2.0:1789760725.401423:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5716 00000100:00100000:2.0:1789760725.401436:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 704864627909 time 704902034934 state 0x2 ref 0 00000100:00100000:1.0:1789760725.405995:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760725.406084:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512587392 00000100:00100000:3.0:1789760725.406102:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512587392 00000100:00100000:3.0:1789760725.406280:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 704864627909 time 704902034934 state 0x0 ref 1 00000100:00100000:3.0:1789760725.406298:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 704902295350 time 704902034934 state 0x1 ref 1 00000100:00100000:3.0:1789760725.406329:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 704902295350 time 704970628407 state 0x0 ref 1 00000100:00100000:3.0:1789760725.406343:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4133b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512587392:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760725.406412:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23293952, mtime: 0x6aad94d4, atime: 0x0, ctime: 0x6aad94d4, blocks: 0xb807 00010000:00010000:3.0:1789760725.406424:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2dcab lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.406451:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=704s v=(0 0) 00010000:00010000:3.0:1789760725.406465:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760725.406473:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2dcab lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23293952->23298047] (req 23293952->23298047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.406490:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2dcab lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23293952->23298047] (req 23293952->23298047) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760725.440111:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973006 is committed 00002000:00010000:2.0:1789760725.440481:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23293952 -> 23298048 00010000:00010000:2.0:1789760725.440503:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2dcab lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23293952->23298047] (req 23293952->23298047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.440517:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2dcab lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23293952->23298047] (req 23293952->23298047) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.440526:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2dcab lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23293952->23298047] (req 23293952->23298047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.440533:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760725.440539:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2dcab lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23293952->23298047] (req 23293952->23298047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.440550:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2dcab lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23293952->23298047] (req 23293952->23298047) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.440585:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2dcab lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23293952->23298047] (req 23293952->23298047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.440594:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2dcab lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23293952->23298047] (req 23293952->23298047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760725.441003:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4133b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512587392:12345-192.168.203.56@tcp:4:dd.0 Request processed in 34655us (35014us total) trans 4294973006 rc 0/0 00000100:00100000:2.0:1789760725.441018:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5717 00000100:00100000:2.0:1789760725.441031:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 704902295350 time 704970628407 state 0x2 ref 0 00000100:00100000:3.0:1789760725.451336:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760725.451561:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512587520 00000100:00100000:2.0:1789760725.451575:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512587520 00000100:00100000:2.0:1789760725.451927:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 704902295350 time 704970628407 state 0x0 ref 1 00000100:00100000:2.0:1789760725.451947:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 704970888823 time 704970628407 state 0x1 ref 1 00000100:00100000:2.0:1789760725.452025:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 704970888823 time 705016324083 state 0x0 ref 1 00000100:00100000:2.0:1789760725.452042:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2744380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512587520:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760725.452115:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23298048, mtime: 0x6aad94d4, atime: 0x0, ctime: 0x6aad94d4, blocks: 0xb807 00010000:00010000:2.0:1789760725.452135:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2dcb2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.452156:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=705s v=(0 0) 00010000:00010000:2.0:1789760725.452163:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760725.452170:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2dcb2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23298048->23302143] (req 23298048->23302143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.452188:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2dcb2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23298048->23302143] (req 23298048->23302143) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760725.538834:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973007 is committed 00002000:00010000:2.0:1789760725.539053:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23298048 -> 23302144 00010000:00010000:2.0:1789760725.539076:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2dcb2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23298048->23302143] (req 23298048->23302143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.539091:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2dcb2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23298048->23302143] (req 23298048->23302143) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.539102:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2dcb2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23298048->23302143] (req 23298048->23302143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.539110:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760725.539116:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2dcb2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23298048->23302143] (req 23298048->23302143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.539244:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2dcb2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23298048->23302143] (req 23298048->23302143) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.539270:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2dcb2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23298048->23302143] (req 23298048->23302143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.539280:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2dcb2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23298048->23302143] (req 23298048->23302143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760725.539444:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2744380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512587520:12345-192.168.203.56@tcp:4:dd.0 Request processed in 87399us (88116us total) trans 4294973007 rc 0/0 00000100:00100000:2.0:1789760725.539467:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5718 00000100:00100000:2.0:1789760725.539480:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 704970888823 time 705016324083 state 0x2 ref 0 00000100:00100000:0.0:1789760725.547035:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760725.547294:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512587648 00000100:00100000:2.0:1789760725.547313:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512587648 00000100:00100000:2.0:1789760725.547393:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 704970888823 time 705016324083 state 0x0 ref 1 00000100:00100000:2.0:1789760725.547413:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 705016584499 time 705016324083 state 0x1 ref 1 00000100:00100000:2.0:1789760725.547452:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 705016584499 time 705111751571 state 0x0 ref 1 00000100:00100000:2.0:1789760725.547477:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2fbb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512587648:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760725.547545:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23302144, mtime: 0x6aad94d4, atime: 0x0, ctime: 0x6aad94d4, blocks: 0xb807 00010000:00010000:2.0:1789760725.547556:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2dcb9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.547578:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=705s v=(0 0) 00010000:00010000:2.0:1789760725.547584:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760725.547592:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2dcb9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23302144->23306239] (req 23302144->23306239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.547610:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2dcb9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23302144->23306239] (req 23302144->23306239) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760725.639889:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973008 is committed 00002000:00010000:2.0:1789760725.640027:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23302144 -> 23306240 00002000:00010000:2.0:1789760725.640031:0:13063:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760724 -> 1789760725 00002000:00010000:2.0:1789760725.640034:0:13063:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760724 -> 1789760725 00010000:00010000:2.0:1789760725.640046:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2dcb9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23302144->23306239] (req 23302144->23306239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.640060:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2dcb9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23302144->23306239] (req 23302144->23306239) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.640069:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2dcb9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23302144->23306239] (req 23302144->23306239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.640076:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760725.640081:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2dcb9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23302144->23306239] (req 23302144->23306239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.640090:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2dcb9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23302144->23306239] (req 23302144->23306239) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.640108:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2dcb9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23302144->23306239] (req 23302144->23306239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.640115:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2dcb9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23302144->23306239] (req 23302144->23306239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760725.640217:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2fbb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512587648:12345-192.168.203.56@tcp:4:dd.0 Request processed in 92738us (93197us total) trans 4294973008 rc 0/0 00000100:00100000:2.0:1789760725.640227:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5719 00000100:00100000:2.0:1789760725.640236:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 705016584499 time 705111751571 state 0x2 ref 0 00000100:00100000:0.0:1789760725.647615:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760725.647746:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512587776 00000100:00100000:2.0:1789760725.647764:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512587776 00000100:00100000:0.0:1789760725.648040:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 705016584499 time 705111751571 state 0x0 ref 1 00000100:00100000:0.0:1789760725.648061:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 705112011987 time 705111751571 state 0x1 ref 1 00000100:00100000:0.0:1789760725.648098:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 705112011987 time 705212397651 state 0x0 ref 1 00000100:00100000:0.0:1789760725.648114:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2fb8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512587776:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760725.648196:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23306240, mtime: 0x6aad94d5, atime: 0x0, ctime: 0x6aad94d5, blocks: 0xb807 00010000:00010000:0.0:1789760725.648208:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2dcc0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.648228:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=705s v=(0 0) 00010000:00010000:0.0:1789760725.648234:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760725.648241:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2dcc0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23306240->23310335] (req 23306240->23310335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.648257:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2dcc0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23306240->23310335] (req 23306240->23310335) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760725.719025:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973009 is committed 00002000:00010000:2.0:1789760725.719341:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23306240 -> 23310336 00010000:00010000:2.0:1789760725.719356:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2dcc0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23306240->23310335] (req 23306240->23310335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.719369:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2dcc0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23306240->23310335] (req 23306240->23310335) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.719377:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2dcc0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23306240->23310335] (req 23306240->23310335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.719384:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760725.719388:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2dcc0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23306240->23310335] (req 23306240->23310335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.719396:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2dcc0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23306240->23310335] (req 23306240->23310335) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.719414:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2dcc0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23306240->23310335] (req 23306240->23310335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.719426:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2dcc0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23306240->23310335] (req 23306240->23310335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760725.719524:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2fb8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512587776:12345-192.168.203.56@tcp:4:dd.0 Request processed in 71409us (71921us total) trans 4294973009 rc 0/0 00000100:00100000:2.0:1789760725.719533:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5720 00000100:00100000:2.0:1789760725.719542:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 705112011987 time 705212397651 state 0x2 ref 0 00000100:00100000:0.0:1789760725.724318:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760725.724440:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512587904 00000100:00100000:2.0:1789760725.724450:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512587904 00000100:00100000:2.0:1789760725.724508:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 705112011987 time 705212397651 state 0x0 ref 1 00000100:00100000:2.0:1789760725.724521:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 705212658067 time 705212397651 state 0x1 ref 1 00000100:00100000:2.0:1789760725.724549:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 705212658067 time 705288849803 state 0x0 ref 1 00000100:00100000:2.0:1789760725.724560:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2fb9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512587904:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760725.724609:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23310336, mtime: 0x6aad94d5, atime: 0x0, ctime: 0x6aad94d5, blocks: 0xb807 00010000:00010000:2.0:1789760725.724618:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2dcc7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.724633:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=705s v=(0 0) 00010000:00010000:2.0:1789760725.724637:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760725.724645:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2dcc7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23310336->23314431] (req 23310336->23314431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.724665:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2dcc7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23310336->23314431] (req 23310336->23314431) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760725.771902:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23310336 -> 23314432 00010000:00010000:2.0:1789760725.771922:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2dcc7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23310336->23314431] (req 23310336->23314431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.771935:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2dcc7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23310336->23314431] (req 23310336->23314431) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.771944:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2dcc7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23310336->23314431] (req 23310336->23314431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.771951:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760725.771955:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2dcc7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23310336->23314431] (req 23310336->23314431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.771963:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2dcc7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23310336->23314431] (req 23310336->23314431) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.771981:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2dcc7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23310336->23314431] (req 23310336->23314431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.771990:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2dcc7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23310336->23314431] (req 23310336->23314431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760725.772085:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2fb9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2361:x1876699512587904:12345-192.168.203.56@tcp:4:dd.0 Request processed in 47522us (47772us total) trans 4294973010 rc 0/0 00000100:00100000:2.0:1789760725.772095:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5721 00000100:00100000:2.0:1789760725.772105:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 705212658067 time 705288849803 state 0x2 ref 0 00000001:00080000:3.0:1789760725.772748:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973010 is committed 00000100:00100000:0.0:1789760725.779213:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760725.779658:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512588032 00000100:00100000:2.0:1789760725.779668:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512588032 00000100:00100000:2.0:1789760725.779764:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 705212658067 time 705288849803 state 0x0 ref 1 00000100:00100000:2.0:1789760725.779781:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 705289110219 time 705288849803 state 0x1 ref 1 00000100:00100000:2.0:1789760725.779812:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 705289110219 time 705344112008 state 0x0 ref 1 00000100:00100000:2.0:1789760725.779825:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2fbaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512588032:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760725.779874:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23314432, mtime: 0x6aad94d5, atime: 0x0, ctime: 0x6aad94d5, blocks: 0xb807 00010000:00010000:2.0:1789760725.779884:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2dcce lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.779901:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=705s v=(0 0) 00010000:00010000:2.0:1789760725.779906:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760725.779913:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2dcce lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23314432->23318527] (req 23314432->23318527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.779927:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2dcce lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23314432->23318527] (req 23314432->23318527) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760725.823427:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973011 is committed 00002000:00010000:1.0:1789760725.823681:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23314432 -> 23318528 00010000:00010000:1.0:1789760725.823719:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2dcce lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23314432->23318527] (req 23314432->23318527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.823731:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2dcce lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23314432->23318527] (req 23314432->23318527) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.823740:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2dcce lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23314432->23318527] (req 23314432->23318527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.823746:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760725.823752:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2dcce lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23314432->23318527] (req 23314432->23318527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.823760:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2dcce lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23314432->23318527] (req 23314432->23318527) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.823779:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2dcce lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23314432->23318527] (req 23314432->23318527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.823787:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2dcce lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23314432->23318527] (req 23314432->23318527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760725.823945:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2fbaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512588032:12345-192.168.203.56@tcp:4:dd.0 Request processed in 44118us (44737us total) trans 4294973011 rc 0/0 00000100:00100000:1.0:1789760725.823956:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5722 00000100:00100000:1.0:1789760725.823966:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 705289110219 time 705344112008 state 0x2 ref 0 00000100:00100000:3.0:1789760725.829041:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760725.829108:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512588160 00000100:00100000:2.0:1789760725.829123:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512588160 00000100:00100000:2.0:1789760725.829182:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 705289110219 time 705344112008 state 0x0 ref 1 00000100:00100000:2.0:1789760725.829197:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 705344372424 time 705344112008 state 0x1 ref 1 00000100:00100000:2.0:1789760725.829230:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 705344372424 time 705393530344 state 0x0 ref 1 00000100:00100000:2.0:1789760725.829242:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2746680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512588160:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760725.829498:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23318528, mtime: 0x6aad94d5, atime: 0x0, ctime: 0x6aad94d5, blocks: 0xb807 00010000:00010000:2.0:1789760725.829509:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2dcd5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.829523:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=705s v=(0 0) 00010000:00010000:2.0:1789760725.829527:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760725.829533:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2dcd5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23318528->23322623] (req 23318528->23322623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.829546:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2dcd5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23318528->23322623] (req 23318528->23322623) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760725.912387:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23318528 -> 23322624 00010000:00010000:0.0:1789760725.912406:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2dcd5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23318528->23322623] (req 23318528->23322623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.912425:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2dcd5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23318528->23322623] (req 23318528->23322623) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.912434:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2dcd5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23318528->23322623] (req 23318528->23322623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.912441:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760725.912445:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2dcd5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23318528->23322623] (req 23318528->23322623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.912464:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2dcd5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23318528->23322623] (req 23318528->23322623) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.912481:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2dcd5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23318528->23322623] (req 23318528->23322623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.912498:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2dcd5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23318528->23322623] (req 23318528->23322623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760725.912794:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2746680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2359:x1876699512588160:12345-192.168.203.56@tcp:4:dd.0 Request processed in 83548us (83760us total) trans 4294973012 rc 0/0 00000100:00100000:0.0:1789760725.912805:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5723 00000100:00100000:0.0:1789760725.912814:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 705344372424 time 705393530344 state 0x2 ref 0 00000001:00080000:3.0:1789760725.913465:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973012 is committed 00000100:00100000:0.0:1789760725.919540:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760725.919779:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512588288 00000100:00100000:2.0:1789760725.919793:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512588288 00000100:00100000:2.0:1789760725.919879:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 705344372424 time 705393530344 state 0x0 ref 1 00000100:00100000:2.0:1789760725.919896:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 705393790760 time 705393530344 state 0x1 ref 1 00000100:00100000:2.0:1789760725.919978:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 705393790760 time 705484277096 state 0x0 ref 1 00000100:00100000:2.0:1789760725.919991:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5305880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512588288:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760725.920039:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23322624, mtime: 0x6aad94d5, atime: 0x0, ctime: 0x6aad94d5, blocks: 0xb807 00010000:00010000:2.0:1789760725.920048:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2dcdc lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.920062:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=705s v=(0 0) 00010000:00010000:2.0:1789760725.920078:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760725.920085:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2dcdc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23322624->23326719] (req 23322624->23326719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.920100:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2dcdc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23322624->23326719] (req 23322624->23326719) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760725.939012:0:3309:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8feac38b3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3309:1876699540220928:0@lo:400:kworker.0 00000100:00100000:1.0:1789760725.939079:0:3309:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760725.939147:0:3309:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:1.0:1789760725.939153:0:3309:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8feac4133800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:6df98d54-eb52-4c5c-8fc9-ee9d74f90f98:3309:1876699540220544:0@lo:400:kworker.0 00000100:00100000:1.0:1789760725.939192:0:3309:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760725.939215:0:3309:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8feac38b2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3309:1876699540221056:0@lo:400:kworker.0 00000100:00100000:1.0:1789760725.939251:0:3309:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760725.939281:0:3311:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8feac4217100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3311:1876699540220672:0@lo:400:kworker.0 00000100:00100000:1.0:1789760725.939302:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760725.939319:0:3311:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8feac38b1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3311:1876699540221184:0@lo:400:kworker.0 00000100:00100000:1.0:1789760725.939341:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760725.939360:0:3311:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1789760725.939365:0:3311:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8feac38b0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3311:1876699540220800:0@lo:400:kworker.0 00000100:00100000:1.0:1789760725.939386:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760725.939415:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540220928 00000100:00100000:1.0:1789760725.939423:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540220928 00000100:00100000:1.0:1789760725.939488:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540221184 00000100:00100000:1.0:1789760725.939503:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540221184 00000100:00100000:1.0:1789760725.939529:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 175 00000100:00100000:1.0:1789760725.939948:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac38b1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3309:x1876699540220928:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760725.940004:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac38b1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3309:x1876699540220928:12345-0@lo:400:kworker.0 Request processed in 60us (924us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760725.940010:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 175 00000100:00100000:1.0:1789760725.940025:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 176 00000100:00100000:1.0:1789760725.940030:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febff933b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3311:x1876699540221184:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760725.940055:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febff933b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3311:x1876699540221184:12345-0@lo:400:kworker.0 Request processed in 25us (713us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760725.940060:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 176 00000100:00100000:1.0:1789760725.940078:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540220544 00000100:00100000:1.0:1789760725.940082:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540220544 00000100:00100000:1.0:1789760725.940107:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 391 00000100:00100000:1.0:1789760725.940113:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febe0501050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:6df98d54-eb52-4c5c-8fc9-ee9d74f90f98+19:3309:x1876699540220544:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760725.940152:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febe0501050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:6df98d54-eb52-4c5c-8fc9-ee9d74f90f98+19:3309:x1876699540220544:12345-0@lo:400:kworker.0 Request processed in 39us (969us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760725.940157:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 391 00000100:00100000:1.0:1789760725.940334:0:5848:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540221056 00000100:00100000:1.0:1789760725.940340:0:5848:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540221056 00000100:00100000:1.0:1789760725.940370:0:5848:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540220672 00000100:00100000:1.0:1789760725.940373:0:5848:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540220672 00000100:00100000:1.0:1789760725.940383:0:5848:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540220800 00000100:00100000:1.0:1789760725.940385:0:5848:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540220800 00000100:00100000:1.0:1789760725.940400:0:5848:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 456 00000100:00100000:1.0:1789760725.940406:0:5848:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febc6bd6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3309:x1876699540221056:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760725.940438:0:5848:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febc6bd6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3309:x1876699540221056:12345-0@lo:400:kworker.0 Request processed in 32us (1187us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760725.940445:0:5848:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 456 00000100:00100000:1.0:1789760725.940467:0:5848:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 457 00000100:00100000:1.0:1789760725.940472:0:5848:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febea771880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3311:x1876699540220672:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760725.940619:0:5848:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febea771880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3311:x1876699540220672:12345-0@lo:400:kworker.0 Request processed in 145us (1314us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760725.940626:0:5848:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 457 00000100:00100000:1.0:1789760725.940638:0:5848:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 458 00000100:00100000:1.0:1789760725.940645:0:5848:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febff930380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3311:x1876699540220800:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760725.940673:0:5848:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febff930380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3311:x1876699540220800:12345-0@lo:400:kworker.0 Request processed in 29us (1286us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760725.940678:0:5848:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 458 00000100:00080000:1.0:1789760725.940726:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:1.0:1789760725.940731:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:1.0:1789760725.940734:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1789760725.940774:0:3311:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8feac38b0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3311:1876699540220800:0@lo:400:kworker.0 00000100:00100000:1.0:1789760725.940785:0:3311:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8feac4217100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3311:1876699540220672:0@lo:400:kworker.0 00000100:00100000:1.0:1789760725.940794:0:3311:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8feac38b1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3311:1876699540221184:0@lo:400:kworker.0 00000100:00100000:1.0:1789760725.940825:0:3309:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8feac4133800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:6df98d54-eb52-4c5c-8fc9-ee9d74f90f98:3309:1876699540220544:0@lo:400:kworker.0 00000100:00100000:1.0:1789760725.940835:0:3309:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8feac38b2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3309:1876699540221056:0@lo:400:kworker.0 00000100:00100000:1.0:1789760725.940844:0:3309:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8feac38b3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3309:1876699540220928:0@lo:400:kworker.0 00000100:00100000:0.0:1789760725.943432:0:3311:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1789760725.943467:0:3311:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8feac2747b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3311:1876699540221312:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1789760725.943546:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760725.943743:0:6706:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540221312 00000100:00100000:3.0:1789760725.943751:0:6706:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540221312 00000100:00100000:3.0:1789760725.943794:0:6706:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 190 00000100:00100000:3.0:1789760725.943803:0:6706:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5305180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3311:x1876699540221312:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1789760725.943825:0:6706:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 528416768 pending 1703936 free 3834642432 avail 3806330880 00000100:00100000:3.0:1789760725.943864:0:6706:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5305180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3311:x1876699540221312:12345-0@lo:13:osp-pre-0-0.0 Request processed in 62us (318us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760725.943870:0:6706:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 190 00000100:00100000:0.0:1789760725.943958:0:3311:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8feac2747b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3311:1876699540221312:0@lo:13:osp-pre-0-0.0 00000001:00080000:0.0:1789760725.973389:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973013 is committed 00002000:00010000:3.0:1789760725.973502:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23322624 -> 23326720 00010000:00010000:3.0:1789760725.973517:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2dcdc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23322624->23326719] (req 23322624->23326719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.973543:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2dcdc lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23322624->23326719] (req 23322624->23326719) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.973556:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2dcdc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23322624->23326719] (req 23322624->23326719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.973564:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760725.973568:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2dcdc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23322624->23326719] (req 23322624->23326719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.973577:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2dcdc lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23322624->23326719] (req 23322624->23326719) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.973594:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2dcdc lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23322624->23326719] (req 23322624->23326719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.973602:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2dcdc lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23322624->23326719] (req 23322624->23326719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760725.973730:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5305880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512588288:12345-192.168.203.56@tcp:4:dd.0 Request processed in 53703us (54167us total) trans 4294973013 rc 0/0 00000100:00100000:3.0:1789760725.973740:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5724 00000100:00100000:3.0:1789760725.973750:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 705393790760 time 705484277096 state 0x2 ref 0 00000100:00100000:2.0:1789760725.979751:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760725.979841:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512588416 00000100:00100000:1.0:1789760725.979851:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512588416 00000100:00100000:1.0:1789760725.979904:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 705393790760 time 705484277096 state 0x0 ref 1 00000100:00100000:1.0:1789760725.979919:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 705484537512 time 705484277096 state 0x1 ref 1 00000100:00100000:1.0:1789760725.979948:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 705484537512 time 705544247903 state 0x0 ref 1 00000100:00100000:1.0:1789760725.979960:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febea89b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512588416:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760725.980008:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23326720, mtime: 0x6aad94d5, atime: 0x0, ctime: 0x6aad94d5, blocks: 0xb807 00010000:00010000:1.0:1789760725.980017:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2dce3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.980033:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=705s v=(0 0) 00010000:00010000:1.0:1789760725.980037:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760725.980044:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2dce3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23326720->23330815] (req 23326720->23330815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.980057:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2dce3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23326720->23330815] (req 23326720->23330815) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760726.016847:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973014 is committed 00002000:00010000:1.0:1789760726.017557:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23326720 -> 23330816 00010000:00010000:1.0:1789760726.017573:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2dce3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23326720->23330815] (req 23326720->23330815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.017586:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2dce3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23326720->23330815] (req 23326720->23330815) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.017595:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2dce3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23326720->23330815] (req 23326720->23330815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.017602:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760726.017607:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2dce3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23326720->23330815] (req 23326720->23330815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.017615:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2dce3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23326720->23330815] (req 23326720->23330815) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.017634:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2dce3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23326720->23330815] (req 23326720->23330815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.017642:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2dce3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23326720->23330815] (req 23326720->23330815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760726.017781:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febea89b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512588416:12345-192.168.203.56@tcp:4:dd.0 Request processed in 37819us (38035us total) trans 4294973014 rc 0/0 00000100:00100000:1.0:1789760726.017791:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5725 00000100:00100000:1.0:1789760726.017801:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 705484537512 time 705544247903 state 0x2 ref 0 00000100:00100000:0.0:1789760726.022781:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760726.026611:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512588544 00000100:00100000:2.0:1789760726.026626:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512588544 00000100:00100000:2.0:1789760726.026690:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 705484537512 time 705544247903 state 0x0 ref 1 00000100:00100000:2.0:1789760726.026754:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 705544508319 time 705544247903 state 0x1 ref 1 00000100:00100000:2.0:1789760726.026785:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 705544508319 time 705591084874 state 0x0 ref 1 00000100:00100000:2.0:1789760726.026797:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febea4bb450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512588544:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760726.026848:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23330816, mtime: 0x6aad94d5, atime: 0x0, ctime: 0x6aad94d5, blocks: 0xb807 00010000:00010000:2.0:1789760726.026857:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2dcea lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760726.026873:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=705s v=(0 0) 00010000:00010000:2.0:1789760726.026877:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760726.026883:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2dcea lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23330816->23334911] (req 23330816->23334911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760726.026897:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2dcea lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23330816->23334911] (req 23330816->23334911) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760726.067335:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973015 is committed 00002000:00010000:0.0:1789760726.067487:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23330816 -> 23334912 00010000:00010000:0.0:1789760726.067503:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2dcea lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23330816->23334911] (req 23330816->23334911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.067516:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2dcea lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23330816->23334911] (req 23330816->23334911) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.067525:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2dcea lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23330816->23334911] (req 23330816->23334911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.067532:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760726.067546:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2dcea lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23330816->23334911] (req 23330816->23334911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.067557:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2dcea lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23330816->23334911] (req 23330816->23334911) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.067576:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2dcea lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23330816->23334911] (req 23330816->23334911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.067590:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2dcea lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23330816->23334911] (req 23330816->23334911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760726.067681:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febea4bb450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512588544:12345-192.168.203.56@tcp:4:dd.0 Request processed in 40881us (44904us total) trans 4294973015 rc 0/0 00000100:00100000:0.0:1789760726.067692:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5726 00000100:00100000:0.0:1789760726.067731:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 705544508319 time 705591084874 state 0x2 ref 0 00000100:00100000:0.0:1789760726.080224:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760726.080686:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512588672 00000100:00100000:3.0:1789760726.080739:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512588672 00000100:00100000:3.0:1789760726.080815:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 705544508319 time 705591084874 state 0x0 ref 1 00000100:00100000:3.0:1789760726.080833:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 705591345290 time 705591084874 state 0x1 ref 1 00000100:00100000:3.0:1789760726.080940:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 705591345290 time 705645238006 state 0x0 ref 1 00000100:00100000:3.0:1789760726.080956:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5305880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512588672:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760726.081019:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23334912, mtime: 0x6aad94d5, atime: 0x0, ctime: 0x6aad94d5, blocks: 0xb807 00010000:00010000:3.0:1789760726.081033:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2dcf1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.081052:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=705s v=(0 0) 00010000:00010000:3.0:1789760726.081057:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760726.081065:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2dcf1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23334912->23339007] (req 23334912->23339007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.081081:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2dcf1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23334912->23339007] (req 23334912->23339007) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760726.139402:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973016 is committed 00002000:00010000:0.0:1789760726.139576:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23334912 -> 23339008 00010000:00010000:0.0:1789760726.139593:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2dcf1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23334912->23339007] (req 23334912->23339007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.139611:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2dcf1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23334912->23339007] (req 23334912->23339007) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.139620:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2dcf1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23334912->23339007] (req 23334912->23339007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.139627:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760726.139633:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2dcf1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23334912->23339007] (req 23334912->23339007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.139642:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2dcf1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23334912->23339007] (req 23334912->23339007) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.139667:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2dcf1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23334912->23339007] (req 23334912->23339007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.139675:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2dcf1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23334912->23339007] (req 23334912->23339007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760726.139939:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5305880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512588672:12345-192.168.203.56@tcp:4:dd.0 Request processed in 58981us (59724us total) trans 4294973016 rc 0/0 00000100:00100000:0.0:1789760726.139949:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5727 00000100:00100000:0.0:1789760726.139958:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 705591345290 time 705645238006 state 0x2 ref 0 00000100:00100000:1.0:1789760726.146079:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760726.146166:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512588800 00000100:00100000:2.0:1789760726.146176:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512588800 00000100:00100000:2.0:1789760726.146239:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 705591345290 time 705645238006 state 0x0 ref 1 00000100:00100000:2.0:1789760726.146255:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 705645498422 time 705645238006 state 0x1 ref 1 00000100:00100000:2.0:1789760726.146289:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 705645498422 time 705710587621 state 0x0 ref 1 00000100:00100000:2.0:1789760726.146303:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febff932a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512588800:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760726.146357:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23339008, mtime: 0x6aad94d5, atime: 0x0, ctime: 0x6aad94d5, blocks: 0xb807 00010000:00010000:2.0:1789760726.146366:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2dcf8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760726.146388:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=705s v=(0 0) 00010000:00010000:2.0:1789760726.146393:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760726.146399:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2dcf8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23339008->23343103] (req 23339008->23343103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760726.146413:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2dcf8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23339008->23343103] (req 23339008->23343103) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760726.209414:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973017 is committed 00002000:00010000:0.0:1789760726.209661:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23339008 -> 23343104 00010000:00010000:0.0:1789760726.209773:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2dcf8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23339008->23343103] (req 23339008->23343103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.209789:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2dcf8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23339008->23343103] (req 23339008->23343103) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.209800:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2dcf8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23339008->23343103] (req 23339008->23343103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.209809:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760726.209815:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2dcf8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23339008->23343103] (req 23339008->23343103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.209832:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2dcf8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23339008->23343103] (req 23339008->23343103) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.210214:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2dcf8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23339008->23343103] (req 23339008->23343103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.210225:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2dcf8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23339008->23343103] (req 23339008->23343103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760726.210358:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febff932a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512588800:12345-192.168.203.56@tcp:4:dd.0 Request processed in 64050us (64291us total) trans 4294973017 rc 0/0 00000100:00100000:0.0:1789760726.210370:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5728 00000100:00100000:0.0:1789760726.210392:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 705645498422 time 705710587621 state 0x2 ref 0 00000100:00100000:2.0:1789760726.214304:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760726.214444:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512588928 00000100:00100000:0.0:1789760726.214478:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512588928 00000100:00100000:0.0:1789760726.214554:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 705645498422 time 705710587621 state 0x0 ref 1 00000100:00100000:0.0:1789760726.214574:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 705710848037 time 705710587621 state 0x1 ref 1 00000100:00100000:0.0:1789760726.214611:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 705710848037 time 705778910414 state 0x0 ref 1 00000100:00100000:0.0:1789760726.214625:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febea899180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512588928:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760726.214689:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23343104, mtime: 0x6aad94d5, atime: 0x0, ctime: 0x6aad94d5, blocks: 0xb807 00010000:00010000:0.0:1789760726.214739:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2dcff lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.214765:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=705s v=(0 0) 00010000:00010000:0.0:1789760726.214771:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760726.214778:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2dcff lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23343104->23347199] (req 23343104->23347199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.214803:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2dcff lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23343104->23347199] (req 23343104->23347199) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760726.290773:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973018 is committed 00002000:00010000:3.0:1789760726.290875:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23343104 -> 23347200 00010000:00010000:3.0:1789760726.290896:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2dcff lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23343104->23347199] (req 23343104->23347199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.290911:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2dcff lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23343104->23347199] (req 23343104->23347199) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.290922:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2dcff lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23343104->23347199] (req 23343104->23347199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.290936:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760726.290942:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2dcff lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23343104->23347199] (req 23343104->23347199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.290952:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2dcff lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23343104->23347199] (req 23343104->23347199) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.290977:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2dcff lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23343104->23347199] (req 23343104->23347199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.290986:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2dcff lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23343104->23347199] (req 23343104->23347199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760726.291108:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febea899180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512588928:12345-192.168.203.56@tcp:4:dd.0 Request processed in 76479us (76812us total) trans 4294973018 rc 0/0 00000100:00100000:3.0:1789760726.291558:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5729 00000100:00100000:3.0:1789760726.291572:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 705710848037 time 705778910414 state 0x2 ref 0 00000100:00100000:3.0:1789760726.297458:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760726.297577:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512589056 00000100:00100000:0.0:1789760726.297591:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512589056 00000100:00100000:0.0:1789760726.297672:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 705710848037 time 705778910414 state 0x0 ref 1 00000100:00100000:0.0:1789760726.297789:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 705779170830 time 705778910414 state 0x1 ref 1 00000100:00100000:0.0:1789760726.297820:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 705779170830 time 705862118908 state 0x0 ref 1 00000100:00100000:0.0:1789760726.297834:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2747800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512589056:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760726.297900:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23347200, mtime: 0x6aad94d5, atime: 0x0, ctime: 0x6aad94d5, blocks: 0xb807 00010000:00010000:0.0:1789760726.297911:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2dd06 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.297931:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=705s v=(0 0) 00010000:00010000:0.0:1789760726.297937:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760726.297944:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2dd06 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23347200->23351295] (req 23347200->23351295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.297966:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2dd06 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23347200->23351295] (req 23347200->23351295) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760726.371431:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973019 is committed 00002000:00010000:0.0:1789760726.371550:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23347200 -> 23351296 00010000:00010000:0.0:1789760726.371571:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2dd06 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23347200->23351295] (req 23347200->23351295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.371588:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2dd06 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23347200->23351295] (req 23347200->23351295) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.371600:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2dd06 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23347200->23351295] (req 23347200->23351295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.371609:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760726.371616:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2dd06 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23347200->23351295] (req 23347200->23351295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.371627:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2dd06 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23347200->23351295] (req 23347200->23351295) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.371652:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2dd06 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23347200->23351295] (req 23347200->23351295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.371660:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2dd06 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23347200->23351295] (req 23347200->23351295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760726.371810:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2747800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512589056:12345-192.168.203.56@tcp:4:dd.0 Request processed in 73972us (74370us total) trans 4294973019 rc 0/0 00000100:00100000:0.0:1789760726.371823:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5730 00000100:00100000:0.0:1789760726.371835:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 705779170830 time 705862118908 state 0x2 ref 0 00000100:00100000:3.0:1789760726.380422:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760726.380661:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512589184 00000100:00100000:0.0:1789760726.380676:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512589184 00000100:00100000:0.0:1789760726.380790:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 705779170830 time 705862118908 state 0x0 ref 1 00000100:00100000:0.0:1789760726.380808:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 705862379324 time 705862118908 state 0x1 ref 1 00000100:00100000:0.0:1789760726.380853:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 705862379324 time 705945151310 state 0x0 ref 1 00000100:00100000:0.0:1789760726.380868:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2745500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512589184:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760726.380938:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23351296, mtime: 0x6aad94d5, atime: 0x0, ctime: 0x6aad94d5, blocks: 0xb807 00010000:00010000:0.0:1789760726.380949:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2dd0d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.380968:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=705s v=(0 0) 00010000:00010000:0.0:1789760726.380973:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760726.380981:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2dd0d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23351296->23355391] (req 23351296->23355391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.380997:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2dd0d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23351296->23355391] (req 23351296->23355391) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.388654:0:5836:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789760726.457730:0:3309:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8febea89aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3309:1876699540221440:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1789760726.457824:0:3309:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760726.457952:0:6706:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540221440 00000100:00100000:2.0:1789760726.457963:0:6706:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540221440 00000100:00100000:2.0:1789760726.458023:0:6706:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 191 00000100:00100000:2.0:1789760726.458033:0:6706:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5305500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3309:x1876699540221440:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1789760726.458077:0:6706:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3859808256 avail 3857711104 00000100:00100000:2.0:1789760726.458142:0:6706:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5305500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3309:x1876699540221440:12345-0@lo:13:osp-pre-1-0.0 Request processed in 110us (317us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760726.458150:0:6706:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 191 00000100:00100000:0.0:1789760726.458356:0:3309:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8febea89aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3309:1876699540221440:0@lo:13:osp-pre-1-0.0 00000001:00080000:0.0:1789760726.462871:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973020 is committed 00002000:00010000:0.0:1789760726.463081:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23351296 -> 23355392 00010000:00010000:0.0:1789760726.463100:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2dd0d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23351296->23355391] (req 23351296->23355391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.463115:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2dd0d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23351296->23355391] (req 23351296->23355391) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.463469:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2dd0d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23351296->23355391] (req 23351296->23355391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.463477:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760726.463483:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2dd0d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23351296->23355391] (req 23351296->23355391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.463494:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2dd0d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23351296->23355391] (req 23351296->23355391) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.463522:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2dd0d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23351296->23355391] (req 23351296->23355391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.463537:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2dd0d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23351296->23355391] (req 23351296->23355391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760726.463659:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2745500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512589184:12345-192.168.203.56@tcp:4:dd.0 Request processed in 82786us (83245us total) trans 4294973020 rc 0/0 00000100:00100000:0.0:1789760726.463671:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5731 00000100:00100000:0.0:1789760726.463684:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 705862379324 time 705945151310 state 0x2 ref 0 00000100:00100000:2.0:1789760726.472250:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760726.478285:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512589312 00000100:00100000:3.0:1789760726.478305:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512589312 00000100:00100000:3.0:1789760726.478512:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 705862379324 time 705945151310 state 0x0 ref 1 00000100:00100000:3.0:1789760726.478534:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 705945411726 time 705945151310 state 0x1 ref 1 00000100:00100000:3.0:1789760726.478685:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 705945411726 time 706042982803 state 0x0 ref 1 00000100:00100000:3.0:1789760726.478721:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febea89b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512589312:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760726.478785:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23355392, mtime: 0x6aad94d5, atime: 0x0, ctime: 0x6aad94d5, blocks: 0xb807 00010000:00010000:3.0:1789760726.478796:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2dd14 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.478817:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=706s v=(0 0) 00010000:00010000:3.0:1789760726.478822:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760726.478830:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2dd14 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23355392->23359487] (req 23355392->23359487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.478847:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2dd14 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23355392->23359487] (req 23355392->23359487) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760726.543203:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23355392 -> 23359488 00010000:00010000:3.0:1789760726.543234:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2dd14 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23355392->23359487] (req 23355392->23359487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.543252:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2dd14 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23355392->23359487] (req 23355392->23359487) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.543262:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2dd14 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23355392->23359487] (req 23355392->23359487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.543271:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760726.543278:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2dd14 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23355392->23359487] (req 23355392->23359487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.543288:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2dd14 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23355392->23359487] (req 23355392->23359487) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.543313:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2dd14 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23355392->23359487] (req 23355392->23359487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.543320:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2dd14 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23355392->23359487] (req 23355392->23359487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760726.543460:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febea89b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2360:x1876699512589312:12345-192.168.203.56@tcp:4:dd.0 Request processed in 64747us (71319us total) trans 4294973021 rc 0/0 00000100:00100000:3.0:1789760726.543472:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5732 00000100:00100000:3.0:1789760726.543485:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 705945411726 time 706042982803 state 0x2 ref 0 00000001:00080000:2.0:1789760726.544051:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973021 is committed 00000100:00100000:0.0:1789760726.549407:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760726.549533:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512589440 00000100:00100000:2.0:1789760726.549548:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512589440 00000100:00100000:2.0:1789760726.549633:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 705945411726 time 706042982803 state 0x0 ref 1 00000100:00100000:2.0:1789760726.549653:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 706043243219 time 706042982803 state 0x1 ref 1 00000100:00100000:2.0:1789760726.549690:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 706043243219 time 706113989018 state 0x0 ref 1 00000100:00100000:2.0:1789760726.549743:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5304a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512589440:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760726.549815:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23359488, mtime: 0x6aad94d5, atime: 0x0, ctime: 0x6aad94d5, blocks: 0xb807 00010000:00010000:2.0:1789760726.549827:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2dd1b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760726.549847:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=706s v=(0 0) 00010000:00010000:2.0:1789760726.549853:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760726.549861:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2dd1b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23359488->23363583] (req 23359488->23363583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760726.549878:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2dd1b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23359488->23363583] (req 23359488->23363583) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760726.596922:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973022 is committed 00002000:00010000:3.0:1789760726.597071:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23359488 -> 23363584 00002000:00010000:3.0:1789760726.597077:0:6708:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760725 -> 1789760726 00002000:00010000:3.0:1789760726.597080:0:6708:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760725 -> 1789760726 00010000:00010000:3.0:1789760726.597098:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2dd1b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23359488->23363583] (req 23359488->23363583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.597114:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2dd1b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23359488->23363583] (req 23359488->23363583) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.597133:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2dd1b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23359488->23363583] (req 23359488->23363583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.597142:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760726.597148:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2dd1b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23359488->23363583] (req 23359488->23363583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.597159:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2dd1b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23359488->23363583] (req 23359488->23363583) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.597185:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2dd1b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23359488->23363583] (req 23359488->23363583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.597194:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2dd1b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23359488->23363583] (req 23359488->23363583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760726.597349:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5304a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512589440:12345-192.168.203.56@tcp:4:dd.0 Request processed in 47603us (47948us total) trans 4294973022 rc 0/0 00000100:00100000:3.0:1789760726.597361:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5733 00000100:00100000:3.0:1789760726.597373:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 706043243219 time 706113989018 state 0x2 ref 0 00000100:00100000:3.0:1789760726.602638:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760726.602887:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512589568 00000100:00100000:1.0:1789760726.602902:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512589568 00000100:00100000:1.0:1789760726.602988:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 706043243219 time 706113989018 state 0x0 ref 1 00000100:00100000:1.0:1789760726.603008:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 706114249434 time 706113989018 state 0x1 ref 1 00000100:00100000:1.0:1789760726.603046:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 706114249434 time 706167345829 state 0x0 ref 1 00000100:00100000:1.0:1789760726.603060:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2746a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512589568:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760726.603131:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23363584, mtime: 0x6aad94d6, atime: 0x0, ctime: 0x6aad94d6, blocks: 0xb807 00010000:00010000:1.0:1789760726.603143:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2dd22 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.603164:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=706s v=(0 0) 00010000:00010000:1.0:1789760726.603170:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760726.603179:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2dd22 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23363584->23367679] (req 23363584->23367679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.603196:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2dd22 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23363584->23367679] (req 23363584->23367679) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760726.669718:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23363584 -> 23367680 00010000:00010000:2.0:1789760726.669745:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2dd22 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23363584->23367679] (req 23363584->23367679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760726.669761:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2dd22 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23363584->23367679] (req 23363584->23367679) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760726.669771:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2dd22 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23363584->23367679] (req 23363584->23367679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760726.669779:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760726.669785:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2dd22 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23363584->23367679] (req 23363584->23367679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760726.669797:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2dd22 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23363584->23367679] (req 23363584->23367679) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760726.669821:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2dd22 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23363584->23367679] (req 23363584->23367679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760726.669829:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2dd22 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23363584->23367679] (req 23363584->23367679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760726.669988:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2746a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2358:x1876699512589568:12345-192.168.203.56@tcp:4:dd.0 Request processed in 66923us (67358us total) trans 4294973023 rc 0/0 00000100:00100000:2.0:1789760726.670002:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5734 00000100:00100000:2.0:1789760726.670014:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 706114249434 time 706167345829 state 0x2 ref 0 00000001:00080000:2.0:1789760726.670063:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973023 is committed 00000100:00100000:2.0:1789760726.674483:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760726.675049:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512589696 00000100:00100000:3.0:1789760726.675064:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512589696 00000100:00100000:3.0:1789760726.675154:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 706114249434 time 706167345829 state 0x0 ref 1 00000100:00100000:3.0:1789760726.675176:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 706167606245 time 706167345829 state 0x1 ref 1 00000100:00100000:3.0:1789760726.675219:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 706167606245 time 706239518491 state 0x0 ref 1 00000100:00100000:3.0:1789760726.675233:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febea89b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512589696:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760726.675312:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23367680, mtime: 0x6aad94d6, atime: 0x0, ctime: 0x6aad94d6, blocks: 0xb807 00010000:00010000:3.0:1789760726.675324:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2dd29 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.675348:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=706s v=(0 0) 00010000:00010000:3.0:1789760726.675354:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760726.675361:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2dd29 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23367680->23371775] (req 23367680->23371775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.675377:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2dd29 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23367680->23371775] (req 23367680->23371775) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760726.740773:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973024 is committed 00002000:00010000:3.0:1789760726.741317:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23367680 -> 23371776 00010000:00010000:3.0:1789760726.741337:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2dd29 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23367680->23371775] (req 23367680->23371775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.741352:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2dd29 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23367680->23371775] (req 23367680->23371775) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.741362:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2dd29 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23367680->23371775] (req 23367680->23371775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.741371:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760726.741377:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2dd29 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23367680->23371775] (req 23367680->23371775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.741387:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2dd29 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23367680->23371775] (req 23367680->23371775) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.741410:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2dd29 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23367680->23371775] (req 23367680->23371775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.741418:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2dd29 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23367680->23371775] (req 23367680->23371775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760726.741553:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febea89b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512589696:12345-192.168.203.56@tcp:4:dd.0 Request processed in 66314us (67081us total) trans 4294973024 rc 0/0 00000100:00100000:3.0:1789760726.741565:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5735 00000100:00100000:3.0:1789760726.741578:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 706167606245 time 706239518491 state 0x2 ref 0 00000100:00100000:0.0:1789760726.746104:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760726.746205:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512589824 00000100:00100000:3.0:1789760726.746218:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512589824 00000100:00100000:3.0:1789760726.746294:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 706167606245 time 706239518491 state 0x0 ref 1 00000100:00100000:3.0:1789760726.746315:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 706239778907 time 706239518491 state 0x1 ref 1 00000100:00100000:3.0:1789760726.746351:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 706239778907 time 706310649951 state 0x0 ref 1 00000100:00100000:3.0:1789760726.746366:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5306d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512589824:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760726.746434:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23371776, mtime: 0x6aad94d6, atime: 0x0, ctime: 0x6aad94d6, blocks: 0xb807 00010000:00010000:3.0:1789760726.746453:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2dd30 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.746474:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=706s v=(0 0) 00010000:00010000:3.0:1789760726.746480:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760726.746487:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2dd30 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23371776->23375871] (req 23371776->23375871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.746505:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2dd30 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23371776->23375871] (req 23371776->23375871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760726.799287:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973025 is committed 00002000:00010000:3.0:1789760726.799426:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23371776 -> 23375872 00010000:00010000:3.0:1789760726.799454:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2dd30 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23371776->23375871] (req 23371776->23375871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.799469:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2dd30 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23371776->23375871] (req 23371776->23375871) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.799480:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2dd30 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23371776->23375871] (req 23371776->23375871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.799488:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760726.799494:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2dd30 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23371776->23375871] (req 23371776->23375871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.799504:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2dd30 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23371776->23375871] (req 23371776->23375871) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.799527:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2dd30 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23371776->23375871] (req 23371776->23375871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.799536:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2dd30 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23371776->23375871] (req 23371776->23375871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760726.799658:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5306d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512589824:12345-192.168.203.56@tcp:4:dd.0 Request processed in 53288us (53563us total) trans 4294973025 rc 0/0 00000100:00100000:3.0:1789760726.799669:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5736 00000100:00100000:3.0:1789760726.799681:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 706239778907 time 706310649951 state 0x2 ref 0 00000100:00100000:0.0:1789760726.804623:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760726.804735:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512589952 00000100:00100000:2.0:1789760726.804748:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512589952 00000100:00100000:2.0:1789760726.805028:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 706239778907 time 706310649951 state 0x0 ref 1 00000100:00100000:2.0:1789760726.805048:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 706310910367 time 706310649951 state 0x1 ref 1 00000100:00100000:2.0:1789760726.805086:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 706310910367 time 706369385197 state 0x0 ref 1 00000100:00100000:2.0:1789760726.805102:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5306300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512589952:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760726.805238:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23375872, mtime: 0x6aad94d6, atime: 0x0, ctime: 0x6aad94d6, blocks: 0xb807 00010000:00010000:2.0:1789760726.805250:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2dd37 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760726.805270:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=706s v=(0 0) 00010000:00010000:2.0:1789760726.805276:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760726.805284:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2dd37 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23375872->23379967] (req 23375872->23379967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760726.805303:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2dd37 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23375872->23379967] (req 23375872->23379967) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760726.842572:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23375872 -> 23379968 00010000:00010000:1.0:1789760726.842601:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2dd37 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23375872->23379967] (req 23375872->23379967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.842616:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2dd37 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23375872->23379967] (req 23375872->23379967) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.842627:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2dd37 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23375872->23379967] (req 23375872->23379967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.842635:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760726.842641:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2dd37 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23375872->23379967] (req 23375872->23379967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.842652:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2dd37 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23375872->23379967] (req 23375872->23379967) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.842676:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2dd37 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23375872->23379967] (req 23375872->23379967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.842684:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2dd37 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23375872->23379967] (req 23375872->23379967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760726.842862:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5306300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2360:x1876699512589952:12345-192.168.203.56@tcp:4:dd.0 Request processed in 37756us (38245us total) trans 4294973026 rc 0/0 00000100:00100000:1.0:1789760726.842875:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5737 00000100:00100000:1.0:1789760726.842886:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 706310910367 time 706369385197 state 0x2 ref 0 00000001:00080000:0.0:1789760726.843279:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973026 is committed 00000100:00100000:1.0:1789760726.850672:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760726.850921:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512590080 00000100:00100000:3.0:1789760726.850935:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512590080 00000100:00100000:3.0:1789760726.851007:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 706310910367 time 706369385197 state 0x0 ref 1 00000100:00100000:3.0:1789760726.851027:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 706369645613 time 706369385197 state 0x1 ref 1 00000100:00100000:3.0:1789760726.851113:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 706369645613 time 706415411589 state 0x0 ref 1 00000100:00100000:3.0:1789760726.851128:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febff930e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512590080:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760726.851202:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23379968, mtime: 0x6aad94d6, atime: 0x0, ctime: 0x6aad94d6, blocks: 0xb807 00010000:00010000:3.0:1789760726.851214:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2dd3e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.851233:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=706s v=(0 0) 00010000:00010000:3.0:1789760726.851238:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760726.851245:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2dd3e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23379968->23384063] (req 23379968->23384063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.851265:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2dd3e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23379968->23384063] (req 23379968->23384063) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760726.899620:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23379968 -> 23384064 00010000:00010000:1.0:1789760726.899647:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2dd3e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23379968->23384063] (req 23379968->23384063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.899663:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2dd3e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23379968->23384063] (req 23379968->23384063) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.899673:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2dd3e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23379968->23384063] (req 23379968->23384063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.899682:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760726.899687:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2dd3e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23379968->23384063] (req 23379968->23384063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.899725:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2dd3e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23379968->23384063] (req 23379968->23384063) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.899750:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2dd3e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23379968->23384063] (req 23379968->23384063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.899759:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2dd3e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23379968->23384063] (req 23379968->23384063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760726.899884:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febff930e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2359:x1876699512590080:12345-192.168.203.56@tcp:4:dd.0 Request processed in 48753us (49219us total) trans 4294973027 rc 0/0 00000100:00100000:1.0:1789760726.899900:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5738 00000100:00100000:1.0:1789760726.899913:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 706369645613 time 706415411589 state 0x2 ref 0 00000001:00080000:2.0:1789760726.900039:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973027 is committed 00000100:00100000:3.0:1789760726.908365:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760726.908460:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512590208 00000100:00100000:2.0:1789760726.908474:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512590208 00000100:00100000:2.0:1789760726.908548:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 706369645613 time 706415411589 state 0x0 ref 1 00000100:00100000:2.0:1789760726.908567:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 706415672005 time 706415411589 state 0x1 ref 1 00000100:00100000:2.0:1789760726.908600:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 706415672005 time 706472899783 state 0x0 ref 1 00000100:00100000:2.0:1789760726.908613:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2745880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512590208:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760726.908674:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23384064, mtime: 0x6aad94d6, atime: 0x0, ctime: 0x6aad94d6, blocks: 0xb807 00010000:00010000:2.0:1789760726.908684:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2dd45 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760726.908765:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=706s v=(0 0) 00010000:00010000:2.0:1789760726.908771:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760726.908779:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2dd45 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23384064->23388159] (req 23384064->23388159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760726.908796:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2dd45 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23384064->23388159] (req 23384064->23388159) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760726.973002:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973028 is committed 00002000:00010000:2.0:1789760726.973366:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23384064 -> 23388160 00010000:00010000:2.0:1789760726.973388:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2dd45 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23384064->23388159] (req 23384064->23388159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760726.973404:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2dd45 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23384064->23388159] (req 23384064->23388159) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760726.973414:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2dd45 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23384064->23388159] (req 23384064->23388159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760726.973423:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760726.973429:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2dd45 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23384064->23388159] (req 23384064->23388159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760726.973459:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2dd45 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23384064->23388159] (req 23384064->23388159) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760726.973486:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2dd45 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23384064->23388159] (req 23384064->23388159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760726.973496:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2dd45 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23384064->23388159] (req 23384064->23388159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760726.973643:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2745880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512590208:12345-192.168.203.56@tcp:4:dd.0 Request processed in 65024us (65296us total) trans 4294973028 rc 0/0 00000100:00100000:2.0:1789760726.973656:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5739 00000100:00100000:2.0:1789760726.973668:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 706415672005 time 706472899783 state 0x2 ref 0 00000100:00100000:0.0:1789760726.977529:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760726.982074:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512590336 00000100:00100000:1.0:1789760726.982098:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512590336 00000100:00100000:0.0:1789760726.990691:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 706415672005 time 706472899783 state 0x0 ref 1 00000100:00100000:0.0:1789760726.990733:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 706473160199 time 706472899783 state 0x1 ref 1 00000100:00100000:0.0:1789760726.990856:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 706473160199 time 706555154225 state 0x0 ref 1 00000100:00100000:0.0:1789760726.990874:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5307800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512590336:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760726.990940:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23388160, mtime: 0x6aad94d6, atime: 0x0, ctime: 0x6aad94d6, blocks: 0xb807 00010000:00010000:0.0:1789760726.990952:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2dd4c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.990971:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=706s v=(0 0) 00010000:00010000:0.0:1789760726.990978:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760726.990984:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2dd4c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23388160->23392255] (req 23388160->23392255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.991001:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2dd4c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23388160->23392255] (req 23388160->23392255) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760727.040829:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973029 is committed 00002000:00010000:2.0:1789760727.040961:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23388160 -> 23392256 00010000:00010000:2.0:1789760727.040983:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2dd4c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23388160->23392255] (req 23388160->23392255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.040998:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2dd4c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23388160->23392255] (req 23388160->23392255) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.041007:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2dd4c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23388160->23392255] (req 23388160->23392255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.041015:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760727.041020:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2dd4c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23388160->23392255] (req 23388160->23392255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.041030:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2dd4c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23388160->23392255] (req 23388160->23392255) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.041059:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2dd4c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23388160->23392255] (req 23388160->23392255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.041067:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2dd4c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23388160->23392255] (req 23388160->23392255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760727.041326:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5307800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512590336:12345-192.168.203.56@tcp:4:dd.0 Request processed in 50450us (63815us total) trans 4294973029 rc 0/0 00000100:00100000:2.0:1789760727.041339:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5740 00000100:00100000:2.0:1789760727.041352:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 706473160199 time 706555154225 state 0x2 ref 0 00000100:00100000:0.0:1789760727.049898:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760727.050019:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512590464 00000100:00100000:1.0:1789760727.050033:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512590464 00000100:00100000:1.0:1789760727.050134:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 706473160199 time 706555154225 state 0x0 ref 1 00000100:00100000:1.0:1789760727.050154:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 706555414641 time 706555154225 state 0x1 ref 1 00000100:00100000:1.0:1789760727.050189:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 706555414641 time 706614488747 state 0x0 ref 1 00000100:00100000:1.0:1789760727.050204:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5304000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512590464:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760727.050273:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23392256, mtime: 0x6aad94d6, atime: 0x0, ctime: 0x6aad94d6, blocks: 0xb807 00010000:00010000:1.0:1789760727.050285:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2dd53 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.050306:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=706s v=(0 0) 00010000:00010000:1.0:1789760727.050312:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760727.050320:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2dd53 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23392256->23396351] (req 23392256->23396351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.050337:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2dd53 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23392256->23396351] (req 23392256->23396351) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1789760727.051141:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00010000:2.0:1789760727.125075:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23392256 -> 23396352 00010000:00010000:2.0:1789760727.125107:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2dd53 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23392256->23396351] (req 23392256->23396351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.125288:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2dd53 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23392256->23396351] (req 23392256->23396351) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.125299:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2dd53 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23392256->23396351] (req 23392256->23396351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.125308:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760727.125314:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2dd53 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23392256->23396351] (req 23392256->23396351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.125331:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2dd53 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23392256->23396351] (req 23392256->23396351) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.125357:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2dd53 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23392256->23396351] (req 23392256->23396351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.125366:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2dd53 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23392256->23396351] (req 23392256->23396351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760727.125503:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5304000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2359:x1876699512590464:12345-192.168.203.56@tcp:4:dd.0 Request processed in 75295us (75804us total) trans 4294973030 rc 0/0 00000100:00100000:2.0:1789760727.125515:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5741 00000100:00100000:2.0:1789760727.125528:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 706555414641 time 706614488747 state 0x2 ref 0 00000001:00080000:0.0:1789760727.126054:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973030 is committed 00000100:00100000:3.0:1789760727.136507:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760727.136814:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512590592 00000100:00100000:2.0:1789760727.136827:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512590592 00000100:00100000:2.0:1789760727.136903:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 706555414641 time 706614488747 state 0x0 ref 1 00000100:00100000:2.0:1789760727.136922:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 706614749163 time 706614488747 state 0x1 ref 1 00000100:00100000:2.0:1789760727.137009:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 706614749163 time 706701307146 state 0x0 ref 1 00000100:00100000:2.0:1789760727.137023:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2746300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512590592:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760727.137084:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23396352, mtime: 0x6aad94d6, atime: 0x0, ctime: 0x6aad94d6, blocks: 0xb807 00010000:00010000:2.0:1789760727.137095:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2dd5a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.137115:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=706s v=(0 0) 00010000:00010000:2.0:1789760727.137120:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760727.137128:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2dd5a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23396352->23400447] (req 23396352->23400447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.137144:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2dd5a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23396352->23400447] (req 23396352->23400447) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760727.202824:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973031 is committed 00002000:00010000:0.0:1789760727.202999:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23396352 -> 23400448 00010000:00010000:0.0:1789760727.203015:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2dd5a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23396352->23400447] (req 23396352->23400447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.203028:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2dd5a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23396352->23400447] (req 23396352->23400447) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.203037:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2dd5a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23396352->23400447] (req 23396352->23400447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.203044:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760727.203048:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2dd5a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23396352->23400447] (req 23396352->23400447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.203057:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2dd5a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23396352->23400447] (req 23396352->23400447) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.203074:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2dd5a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23396352->23400447] (req 23396352->23400447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.203082:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2dd5a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23396352->23400447] (req 23396352->23400447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760727.203195:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2746300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512590592:12345-192.168.203.56@tcp:4:dd.0 Request processed in 66169us (66699us total) trans 4294973031 rc 0/0 00000100:00100000:0.0:1789760727.203205:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5742 00000100:00100000:0.0:1789760727.203214:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 706614749163 time 706701307146 state 0x2 ref 0 00000100:00100000:1.0:1789760727.208294:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760727.208380:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512590720 00000100:00100000:2.0:1789760727.208392:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512590720 00000100:00100000:2.0:1789760727.208455:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 706614749163 time 706701307146 state 0x0 ref 1 00000100:00100000:2.0:1789760727.208471:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 706701567562 time 706701307146 state 0x1 ref 1 00000100:00100000:2.0:1789760727.208499:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 706701567562 time 706772799197 state 0x0 ref 1 00000100:00100000:2.0:1789760727.208511:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4130000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512590720:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760727.208557:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23400448, mtime: 0x6aad94d6, atime: 0x0, ctime: 0x6aad94d6, blocks: 0xb807 00010000:00010000:2.0:1789760727.208566:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2dd61 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.208581:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=706s v=(0 0) 00010000:00010000:2.0:1789760727.208586:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760727.208592:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2dd61 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23400448->23404543] (req 23400448->23404543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.208612:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2dd61 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23400448->23404543] (req 23400448->23404543) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760727.239891:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973032 is committed 00002000:00010000:2.0:1789760727.240043:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23400448 -> 23404544 00010000:00010000:2.0:1789760727.240059:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2dd61 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23400448->23404543] (req 23400448->23404543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.240072:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2dd61 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23400448->23404543] (req 23400448->23404543) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.240081:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2dd61 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23400448->23404543] (req 23400448->23404543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.240087:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760727.240092:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2dd61 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23400448->23404543] (req 23400448->23404543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.240101:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2dd61 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23400448->23404543] (req 23400448->23404543) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.240129:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2dd61 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23400448->23404543] (req 23400448->23404543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.240138:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2dd61 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23400448->23404543] (req 23400448->23404543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760727.240515:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4130000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512590720:12345-192.168.203.56@tcp:4:dd.0 Request processed in 31720us (31944us total) trans 4294973032 rc 0/0 00000100:00100000:2.0:1789760727.240527:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5743 00000100:00100000:2.0:1789760727.240538:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 706701567562 time 706772799197 state 0x2 ref 0 00000100:00100000:1.0:1789760727.244910:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760727.245160:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512590848 00000100:00100000:2.0:1789760727.245171:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512590848 00000100:00100000:2.0:1789760727.245237:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 706701567562 time 706772799197 state 0x0 ref 1 00000100:00100000:2.0:1789760727.245255:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 706773059613 time 706772799197 state 0x1 ref 1 00000100:00100000:2.0:1789760727.245287:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 706773059613 time 706809586524 state 0x0 ref 1 00000100:00100000:2.0:1789760727.245300:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4133480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512590848:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760727.245359:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23404544, mtime: 0x6aad94d6, atime: 0x0, ctime: 0x6aad94d6, blocks: 0xb807 00010000:00010000:2.0:1789760727.245368:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2dd68 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.245384:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=706s v=(0 0) 00010000:00010000:2.0:1789760727.245389:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760727.245395:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2dd68 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23404544->23408639] (req 23404544->23408639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.245408:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2dd68 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23404544->23408639] (req 23404544->23408639) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760727.288751:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23404544 -> 23408640 00010000:00010000:1.0:1789760727.288771:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2dd68 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23404544->23408639] (req 23404544->23408639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.288784:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2dd68 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23404544->23408639] (req 23404544->23408639) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.288794:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2dd68 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23404544->23408639] (req 23404544->23408639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.288801:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760727.288806:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2dd68 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23404544->23408639] (req 23404544->23408639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.288815:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2dd68 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23404544->23408639] (req 23404544->23408639) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.288903:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2dd68 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23404544->23408639] (req 23404544->23408639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.288912:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2dd68 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23404544->23408639] (req 23404544->23408639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760727.289062:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4133480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2358:x1876699512590848:12345-192.168.203.56@tcp:4:dd.0 Request processed in 43760us (44156us total) trans 4294973033 rc 0/0 00000100:00100000:1.0:1789760727.289072:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5744 00000100:00100000:1.0:1789760727.289081:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 706773059613 time 706809586524 state 0x2 ref 0 00000001:00080000:0.0:1789760727.290891:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973033 is committed 00000100:00100000:3.0:1789760727.297586:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760727.301059:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512590976 00000100:00100000:0.0:1789760727.301077:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512590976 00000100:00100000:0.0:1789760727.301375:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 706773059613 time 706809586524 state 0x0 ref 1 00000100:00100000:0.0:1789760727.301542:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 706809846940 time 706809586524 state 0x1 ref 1 00000100:00100000:2.0:1789760727.302388:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 706809846940 time 706866680777 state 0x0 ref 1 00000100:00100000:2.0:1789760727.302421:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2745180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512590976:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760727.302483:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23408640, mtime: 0x6aad94d6, atime: 0x0, ctime: 0x6aad94d6, blocks: 0xb807 00010000:00010000:2.0:1789760727.302493:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2dd6f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.302510:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=706s v=(0 0) 00010000:00010000:2.0:1789760727.302514:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760727.302521:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2dd6f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23408640->23412735] (req 23408640->23412735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.302536:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2dd6f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23408640->23412735] (req 23408640->23412735) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760727.338940:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973034 is committed 00002000:00010000:0.0:1789760727.338967:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23408640 -> 23412736 00010000:00010000:0.0:1789760727.338984:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2dd6f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23408640->23412735] (req 23408640->23412735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.338998:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2dd6f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23408640->23412735] (req 23408640->23412735) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.339007:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2dd6f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23408640->23412735] (req 23408640->23412735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.339014:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760727.339020:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2dd6f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23408640->23412735] (req 23408640->23412735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.339029:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2dd6f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23408640->23412735] (req 23408640->23412735) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.339052:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2dd6f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23408640->23412735] (req 23408640->23412735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.339060:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2dd6f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23408640->23412735] (req 23408640->23412735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760727.339162:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2745180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512590976:12345-192.168.203.56@tcp:4:dd.0 Request processed in 36745us (41581us total) trans 4294973034 rc 0/0 00000100:00100000:0.0:1789760727.339173:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5745 00000100:00100000:0.0:1789760727.339183:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 706809846940 time 706866680777 state 0x2 ref 0 00000100:00100000:2.0:1789760727.353299:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760727.353611:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512591104 00000100:00100000:3.0:1789760727.353621:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512591104 00000100:00100000:3.0:1789760727.353675:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 706809846940 time 706866680777 state 0x0 ref 1 00000100:00100000:3.0:1789760727.353690:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 706866941193 time 706866680777 state 0x1 ref 1 00000100:00100000:3.0:1789760727.353802:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 706866941193 time 706918100634 state 0x0 ref 1 00000100:00100000:3.0:1789760727.353818:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febea89aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512591104:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760727.353865:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23412736, mtime: 0x6aad94d6, atime: 0x0, ctime: 0x6aad94d6, blocks: 0xb807 00010000:00010000:3.0:1789760727.353874:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2dd76 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.353891:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=706s v=(0 0) 00010000:00010000:3.0:1789760727.353895:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760727.353902:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2dd76 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23412736->23416831] (req 23412736->23416831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.353916:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2dd76 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23412736->23416831] (req 23412736->23416831) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760727.388418:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973035 is committed 00002000:00010000:1.0:1789760727.388495:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23412736 -> 23416832 00010000:00010000:1.0:1789760727.388509:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2dd76 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23412736->23416831] (req 23412736->23416831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.388523:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2dd76 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23412736->23416831] (req 23412736->23416831) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.388533:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2dd76 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23412736->23416831] (req 23412736->23416831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.388541:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760727.388545:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2dd76 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23412736->23416831] (req 23412736->23416831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.388554:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2dd76 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23412736->23416831] (req 23412736->23416831) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.388572:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2dd76 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23412736->23416831] (req 23412736->23416831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.388580:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2dd76 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23412736->23416831] (req 23412736->23416831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760727.388672:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febea89aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512591104:12345-192.168.203.56@tcp:4:dd.0 Request processed in 34854us (35378us total) trans 4294973035 rc 0/0 00000100:00100000:1.0:1789760727.388682:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5746 00000100:00100000:1.0:1789760727.388692:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 706866941193 time 706918100634 state 0x2 ref 0 00000100:00100000:1.0:1789760727.400032:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760727.402780:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512591232 00000100:00100000:2.0:1789760727.402805:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512591232 00000100:00100000:2.0:1789760727.402883:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 706866941193 time 706918100634 state 0x0 ref 1 00000100:00100000:2.0:1789760727.402898:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 706918361050 time 706918100634 state 0x1 ref 1 00000100:00100000:2.0:1789760727.402936:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 706918361050 time 706967236108 state 0x0 ref 1 00000100:00100000:2.0:1789760727.402948:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4130700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512591232:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760727.402996:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23416832, mtime: 0x6aad94d6, atime: 0x0, ctime: 0x6aad94d6, blocks: 0xb807 00010000:00010000:2.0:1789760727.403005:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2dd7d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.403022:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=706s v=(0 0) 00010000:00010000:2.0:1789760727.403026:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760727.403032:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2dd7d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23416832->23420927] (req 23416832->23420927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.403045:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2dd7d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23416832->23420927] (req 23416832->23420927) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.410846:0:5835:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:1.0:1789760727.457579:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973036 is committed 00002000:00010000:2.0:1789760727.457811:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23416832 -> 23420928 00010000:00010000:2.0:1789760727.457827:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2dd7d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23416832->23420927] (req 23416832->23420927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.457840:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2dd7d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23416832->23420927] (req 23416832->23420927) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.457849:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2dd7d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23416832->23420927] (req 23416832->23420927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.457856:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760727.457861:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2dd7d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23416832->23420927] (req 23416832->23420927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.457869:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2dd7d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23416832->23420927] (req 23416832->23420927) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.457890:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2dd7d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23416832->23420927] (req 23416832->23420927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.457898:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2dd7d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23416832->23420927] (req 23416832->23420927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760727.458642:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4130700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512591232:12345-192.168.203.56@tcp:4:dd.0 Request processed in 55040us (57963us total) trans 4294973036 rc 0/0 00000100:00100000:2.0:1789760727.458654:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5747 00000100:00100000:2.0:1789760727.458665:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 706918361050 time 706967236108 state 0x2 ref 0 00000100:00100000:3.0:1789760727.464098:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760727.464237:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512591360 00000100:00100000:0.0:1789760727.464248:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512591360 00000100:00100000:0.0:1789760727.464303:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 706918361050 time 706967236108 state 0x0 ref 1 00000100:00100000:0.0:1789760727.464318:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 706967496524 time 706967236108 state 0x1 ref 1 00000100:00100000:0.0:1789760727.464350:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 706967496524 time 707028649042 state 0x0 ref 1 00000100:00100000:0.0:1789760727.464363:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2744a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512591360:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760727.464414:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23420928, mtime: 0x6aad94d6, atime: 0x0, ctime: 0x6aad94d6, blocks: 0xb807 00010000:00010000:0.0:1789760727.464424:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2dd84 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.464460:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=707s v=(0 0) 00010000:00010000:0.0:1789760727.464468:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760727.464474:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2dd84 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23420928->23425023] (req 23420928->23425023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.464488:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2dd84 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23420928->23425023] (req 23420928->23425023) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760727.507670:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23420928 -> 23425024 00010000:00010000:0.0:1789760727.507689:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2dd84 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23420928->23425023] (req 23420928->23425023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.507735:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2dd84 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23420928->23425023] (req 23420928->23425023) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.507745:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2dd84 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23420928->23425023] (req 23420928->23425023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.507753:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760727.507757:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2dd84 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23420928->23425023] (req 23420928->23425023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.507767:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2dd84 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23420928->23425023] (req 23420928->23425023) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.507787:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2dd84 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23420928->23425023] (req 23420928->23425023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.507795:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2dd84 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23420928->23425023] (req 23420928->23425023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760727.507892:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2744a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2359:x1876699512591360:12345-192.168.203.56@tcp:4:dd.0 Request processed in 43528us (43798us total) trans 4294973037 rc 0/0 00000100:00100000:0.0:1789760727.507903:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5748 00000100:00100000:0.0:1789760727.507912:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 706967496524 time 707028649042 state 0x2 ref 0 00000001:00080000:2.0:1789760727.508763:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973037 is committed 00000100:00100000:3.0:1789760727.510885:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760727.511029:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512591488 00000100:00100000:0.0:1789760727.511040:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512591488 00000100:00100000:0.0:1789760727.511092:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 706967496524 time 707028649042 state 0x0 ref 1 00000100:00100000:0.0:1789760727.511107:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 707028909458 time 707028649042 state 0x1 ref 1 00000100:00100000:0.0:1789760727.511179:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 707028909458 time 707075477549 state 0x0 ref 1 00000100:00100000:0.0:1789760727.511191:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2746d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512591488:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760727.511237:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23425024, mtime: 0x6aad94d6, atime: 0x0, ctime: 0x6aad94d6, blocks: 0xb807 00010000:00010000:0.0:1789760727.511245:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2dd8b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.511260:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=707s v=(0 0) 00010000:00010000:0.0:1789760727.511265:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760727.511271:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2dd8b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23425024->23429119] (req 23425024->23429119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.511285:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2dd8b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23425024->23429119] (req 23425024->23429119) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760727.544010:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973038 is committed 00002000:00010000:2.0:1789760727.544097:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23425024 -> 23429120 00002000:00010000:2.0:1789760727.544101:0:13063:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760726 -> 1789760727 00002000:00010000:2.0:1789760727.544104:0:13063:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760726 -> 1789760727 00010000:00010000:2.0:1789760727.544149:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2dd8b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23425024->23429119] (req 23425024->23429119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.544163:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2dd8b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23425024->23429119] (req 23425024->23429119) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.544172:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2dd8b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23425024->23429119] (req 23425024->23429119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.544179:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760727.544185:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2dd8b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23425024->23429119] (req 23425024->23429119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.544196:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2dd8b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23425024->23429119] (req 23425024->23429119) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.544213:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2dd8b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23425024->23429119] (req 23425024->23429119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.544220:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2dd8b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23425024->23429119] (req 23425024->23429119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760727.544310:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2746d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512591488:12345-192.168.203.56@tcp:4:dd.0 Request processed in 33116us (33428us total) trans 4294973038 rc 0/0 00000100:00100000:2.0:1789760727.544320:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5749 00000100:00100000:2.0:1789760727.544328:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 707028909458 time 707075477549 state 0x2 ref 0 00000100:00100000:3.0:1789760727.546644:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760727.546805:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512591616 00000100:00100000:2.0:1789760727.546814:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512591616 00000100:00100000:2.0:1789760727.546869:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 707028909458 time 707075477549 state 0x0 ref 1 00000100:00100000:2.0:1789760727.546883:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 707075737965 time 707075477549 state 0x1 ref 1 00000100:00100000:2.0:1789760727.546912:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 707075737965 time 707111211448 state 0x0 ref 1 00000100:00100000:2.0:1789760727.546924:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2744380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512591616:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760727.546974:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23429120, mtime: 0x6aad94d7, atime: 0x0, ctime: 0x6aad94d7, blocks: 0xb807 00010000:00010000:2.0:1789760727.546982:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2dd92 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.546997:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=707s v=(0 0) 00010000:00010000:2.0:1789760727.547001:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760727.547007:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2dd92 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23429120->23433215] (req 23429120->23433215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.547021:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2dd92 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23429120->23433215] (req 23429120->23433215) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760727.592587:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23429120 -> 23433216 00010000:00010000:2.0:1789760727.592607:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2dd92 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23429120->23433215] (req 23429120->23433215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.592622:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2dd92 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23429120->23433215] (req 23429120->23433215) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.592632:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2dd92 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23429120->23433215] (req 23429120->23433215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.592639:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760727.592644:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2dd92 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23429120->23433215] (req 23429120->23433215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.592653:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2dd92 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23429120->23433215] (req 23429120->23433215) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.592680:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2dd92 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23429120->23433215] (req 23429120->23433215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.592689:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2dd92 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23429120->23433215] (req 23429120->23433215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760727.592804:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2744380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2359:x1876699512591616:12345-192.168.203.56@tcp:4:dd.0 Request processed in 45878us (46165us total) trans 4294973039 rc 0/0 00000100:00100000:2.0:1789760727.592815:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5750 00000100:00100000:2.0:1789760727.592825:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 707075737965 time 707111211448 state 0x2 ref 0 00000001:00080000:3.0:1789760727.592916:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973039 is committed 00000100:00100000:3.0:1789760727.595031:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760727.595209:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512591744 00000100:00100000:0.0:1789760727.595219:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512591744 00000100:00100000:0.0:1789760727.595275:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 707075737965 time 707111211448 state 0x0 ref 1 00000100:00100000:0.0:1789760727.595292:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 707111471864 time 707111211448 state 0x1 ref 1 00000100:00100000:0.0:1789760727.595318:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 707111471864 time 707159618004 state 0x0 ref 1 00000100:00100000:0.0:1789760727.595330:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2746680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512591744:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760727.595375:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23433216, mtime: 0x6aad94d7, atime: 0x0, ctime: 0x6aad94d7, blocks: 0xb807 00010000:00010000:0.0:1789760727.595383:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2dd99 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.595398:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=707s v=(0 0) 00010000:00010000:0.0:1789760727.595402:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760727.595409:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2dd99 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23433216->23437311] (req 23433216->23437311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.595422:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2dd99 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23433216->23437311] (req 23433216->23437311) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760727.630396:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23433216 -> 23437312 00010000:00010000:2.0:1789760727.630416:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2dd99 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23433216->23437311] (req 23433216->23437311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.630429:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2dd99 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23433216->23437311] (req 23433216->23437311) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.630447:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2dd99 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23433216->23437311] (req 23433216->23437311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.630454:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760727.630459:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2dd99 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23433216->23437311] (req 23433216->23437311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.630472:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2dd99 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23433216->23437311] (req 23433216->23437311) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.630490:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2dd99 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23433216->23437311] (req 23433216->23437311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.630498:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2dd99 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23433216->23437311] (req 23433216->23437311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760727.630661:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2746680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2361:x1876699512591744:12345-192.168.203.56@tcp:4:dd.0 Request processed in 35327us (35632us total) trans 4294973040 rc 0/0 00000100:00100000:2.0:1789760727.630671:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5751 00000100:00100000:2.0:1789760727.630680:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 707111471864 time 707159618004 state 0x2 ref 0 00000001:00080000:3.0:1789760727.633533:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973040 is committed 00000100:00100000:0.0:1789760727.634184:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760727.634259:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512591872 00000100:00100000:2.0:1789760727.634268:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512591872 00000100:00100000:2.0:1789760727.634319:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 707111471864 time 707159618004 state 0x0 ref 1 00000100:00100000:2.0:1789760727.634334:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 707159878420 time 707159618004 state 0x1 ref 1 00000100:00100000:2.0:1789760727.634362:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 707159878420 time 707198662313 state 0x0 ref 1 00000100:00100000:2.0:1789760727.634375:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5304380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512591872:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760727.634422:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23437312, mtime: 0x6aad94d7, atime: 0x0, ctime: 0x6aad94d7, blocks: 0xb807 00010000:00010000:2.0:1789760727.634430:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2dda0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.634455:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=707s v=(0 0) 00010000:00010000:2.0:1789760727.634460:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760727.634468:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2dda0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23437312->23441407] (req 23437312->23441407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.634481:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2dda0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23437312->23441407] (req 23437312->23441407) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760727.700258:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23437312 -> 23441408 00010000:00010000:0.0:1789760727.700278:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2dda0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23437312->23441407] (req 23437312->23441407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.700290:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2dda0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23437312->23441407] (req 23437312->23441407) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.700299:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2dda0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23437312->23441407] (req 23437312->23441407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.700307:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760727.700312:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2dda0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23437312->23441407] (req 23437312->23441407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.700321:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2dda0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23437312->23441407] (req 23437312->23441407) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.700339:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2dda0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23437312->23441407] (req 23437312->23441407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.700346:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2dda0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23437312->23441407] (req 23437312->23441407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760727.700434:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5304380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2358:x1876699512591872:12345-192.168.203.56@tcp:4:dd.0 Request processed in 66057us (66250us total) trans 4294973041 rc 0/0 00000100:00100000:0.0:1789760727.700455:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5752 00000100:00100000:0.0:1789760727.700464:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 707159878420 time 707198662313 state 0x2 ref 0 00000001:00080000:1.0:1789760727.703232:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973041 is committed 00000100:00100000:0.0:1789760727.703466:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760727.703540:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512592000 00000100:00100000:3.0:1789760727.703550:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512592000 00000100:00100000:3.0:1789760727.703600:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 707159878420 time 707198662313 state 0x0 ref 1 00000100:00100000:3.0:1789760727.703615:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 707198922729 time 707198662313 state 0x1 ref 1 00000100:00100000:3.0:1789760727.703644:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 707198922729 time 707267944244 state 0x0 ref 1 00000100:00100000:3.0:1789760727.703655:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5306680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512592000:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760727.703729:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23441408, mtime: 0x6aad94d7, atime: 0x0, ctime: 0x6aad94d7, blocks: 0xb807 00010000:00010000:3.0:1789760727.703738:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2dda7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.703754:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=707s v=(0 0) 00010000:00010000:3.0:1789760727.703759:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760727.703765:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2dda7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23441408->23445503] (req 23441408->23445503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.703778:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2dda7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23441408->23445503] (req 23441408->23445503) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760727.741083:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973042 is committed 00002000:00010000:0.0:1789760727.741197:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23441408 -> 23445504 00010000:00010000:0.0:1789760727.741214:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2dda7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23441408->23445503] (req 23441408->23445503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.741227:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2dda7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23441408->23445503] (req 23441408->23445503) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.741249:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2dda7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23441408->23445503] (req 23441408->23445503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.741258:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760727.741264:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2dda7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23441408->23445503] (req 23441408->23445503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.741274:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2dda7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23441408->23445503] (req 23441408->23445503) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.741297:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2dda7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23441408->23445503] (req 23441408->23445503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.741312:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2dda7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23441408->23445503] (req 23441408->23445503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760727.741422:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5306680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512592000:12345-192.168.203.56@tcp:4:dd.0 Request processed in 37762us (37953us total) trans 4294973042 rc 0/0 00000100:00100000:0.0:1789760727.741434:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5753 00000100:00100000:0.0:1789760727.741451:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 707198922729 time 707267944244 state 0x2 ref 0 00000100:00100000:3.0:1789760727.743933:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760727.744219:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512592128 00000100:00100000:2.0:1789760727.744229:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512592128 00000100:00100000:2.0:1789760727.744280:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 707198922729 time 707267944244 state 0x0 ref 1 00000100:00100000:2.0:1789760727.744294:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 707268204660 time 707267944244 state 0x1 ref 1 00000100:00100000:2.0:1789760727.744318:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 707268204660 time 707308618319 state 0x0 ref 1 00000100:00100000:2.0:1789760727.744331:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2747b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512592128:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760727.744376:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23445504, mtime: 0x6aad94d7, atime: 0x0, ctime: 0x6aad94d7, blocks: 0xb807 00010000:00010000:2.0:1789760727.744384:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2ddae lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.744399:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=707s v=(0 0) 00010000:00010000:2.0:1789760727.744404:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760727.744411:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2ddae lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23445504->23449599] (req 23445504->23449599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.744428:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2ddae lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23445504->23449599] (req 23445504->23449599) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760727.791578:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973043 is committed 00002000:00010000:3.0:1789760727.791664:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23445504 -> 23449600 00010000:00010000:3.0:1789760727.791680:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2ddae lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23445504->23449599] (req 23445504->23449599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.791692:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2ddae lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23445504->23449599] (req 23445504->23449599) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.791723:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2ddae lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23445504->23449599] (req 23445504->23449599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.791730:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760727.791735:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2ddae lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23445504->23449599] (req 23445504->23449599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.791745:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2ddae lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23445504->23449599] (req 23445504->23449599) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.791765:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2ddae lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23445504->23449599] (req 23445504->23449599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.791777:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2ddae lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23445504->23449599] (req 23445504->23449599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760727.791904:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2747b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512592128:12345-192.168.203.56@tcp:4:dd.0 Request processed in 47569us (47975us total) trans 4294973043 rc 0/0 00000100:00100000:3.0:1789760727.791915:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5754 00000100:00100000:3.0:1789760727.791924:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 707268204660 time 707308618319 state 0x2 ref 0 00000100:00100000:3.0:1789760727.794974:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760727.795133:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512592256 00000100:00100000:0.0:1789760727.795143:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512592256 00000100:00100000:0.0:1789760727.795206:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 707268204660 time 707308618319 state 0x0 ref 1 00000100:00100000:0.0:1789760727.795221:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 707308878735 time 707308618319 state 0x1 ref 1 00000100:00100000:0.0:1789760727.795250:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 707308878735 time 707359550312 state 0x0 ref 1 00000100:00100000:0.0:1789760727.795262:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febfb374380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512592256:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760727.795319:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23449600, mtime: 0x6aad94d7, atime: 0x0, ctime: 0x6aad94d7, blocks: 0xb807 00010000:00010000:0.0:1789760727.795329:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2ddb5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.795343:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=707s v=(0 0) 00010000:00010000:0.0:1789760727.795348:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760727.795360:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2ddb5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23449600->23453695] (req 23449600->23453695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.795374:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2ddb5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23449600->23453695] (req 23449600->23453695) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760727.825613:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973044 is committed 00002000:00010000:0.0:1789760727.825718:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23449600 -> 23453696 00010000:00010000:0.0:1789760727.825734:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2ddb5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23449600->23453695] (req 23449600->23453695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.825749:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2ddb5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23449600->23453695] (req 23449600->23453695) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.825759:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2ddb5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23449600->23453695] (req 23449600->23453695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.825766:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760727.825771:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2ddb5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23449600->23453695] (req 23449600->23453695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.825781:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2ddb5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23449600->23453695] (req 23449600->23453695) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.825801:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2ddb5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23449600->23453695] (req 23449600->23453695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.825809:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2ddb5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23449600->23453695] (req 23449600->23453695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760727.826031:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febfb374380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512592256:12345-192.168.203.56@tcp:4:dd.0 Request processed in 30766us (31061us total) trans 4294973044 rc 0/0 00000100:00100000:0.0:1789760727.826046:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5755 00000100:00100000:0.0:1789760727.826056:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 707308878735 time 707359550312 state 0x2 ref 0 00000100:00100000:0.0:1789760727.831166:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760727.831353:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512592384 00000100:00100000:3.0:1789760727.831363:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512592384 00000100:00100000:3.0:1789760727.831470:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 707308878735 time 707359550312 state 0x0 ref 1 00000100:00100000:3.0:1789760727.831491:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 707359810728 time 707359550312 state 0x1 ref 1 00000100:00100000:3.0:1789760727.831521:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 707359810728 time 707395821092 state 0x0 ref 1 00000100:00100000:3.0:1789760727.831535:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5307b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512592384:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760727.831586:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23453696, mtime: 0x6aad94d7, atime: 0x0, ctime: 0x6aad94d7, blocks: 0xb807 00010000:00010000:3.0:1789760727.831596:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2ddbc lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.831612:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=707s v=(0 0) 00010000:00010000:3.0:1789760727.831617:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760727.831624:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2ddbc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23453696->23457791] (req 23453696->23457791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.831639:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2ddbc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23453696->23457791] (req 23453696->23457791) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760727.870399:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973045 is committed 00002000:00010000:0.0:1789760727.870487:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23453696 -> 23457792 00010000:00010000:0.0:1789760727.870501:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2ddbc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23453696->23457791] (req 23453696->23457791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.870512:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2ddbc lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23453696->23457791] (req 23453696->23457791) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.870521:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2ddbc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23453696->23457791] (req 23453696->23457791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.870529:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760727.870534:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2ddbc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23453696->23457791] (req 23453696->23457791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.870545:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2ddbc lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23453696->23457791] (req 23453696->23457791) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.870562:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2ddbc lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23453696->23457791] (req 23453696->23457791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.870570:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2ddbc lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23453696->23457791] (req 23453696->23457791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760727.870669:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5307b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512592384:12345-192.168.203.56@tcp:4:dd.0 Request processed in 39133us (39507us total) trans 4294973045 rc 0/0 00000100:00100000:0.0:1789760727.870679:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5756 00000100:00100000:0.0:1789760727.870690:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 707359810728 time 707395821092 state 0x2 ref 0 00000100:00100000:2.0:1789760727.883308:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760727.883457:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512592512 00000100:00100000:3.0:1789760727.883471:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512592512 00000100:00100000:3.0:1789760727.883537:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 707359810728 time 707395821092 state 0x0 ref 1 00000100:00100000:3.0:1789760727.883552:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 707396081508 time 707395821092 state 0x1 ref 1 00000100:00100000:0.0:1789760727.883742:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 707396081508 time 707448039157 state 0x0 ref 1 00000100:00100000:2.0:1789760727.886239:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4058380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512592512:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760727.886320:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23457792, mtime: 0x6aad94d7, atime: 0x0, ctime: 0x6aad94d7, blocks: 0xb807 00010000:00010000:2.0:1789760727.886330:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2ddc3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.886346:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=707s v=(0 0) 00010000:00010000:2.0:1789760727.886351:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760727.886357:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2ddc3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23457792->23461887] (req 23457792->23461887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.886372:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2ddc3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23457792->23461887] (req 23457792->23461887) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760727.952382:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973046 is committed 00002000:00010000:1.0:1789760727.952513:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23457792 -> 23461888 00010000:00010000:1.0:1789760727.952529:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2ddc3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23457792->23461887] (req 23457792->23461887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.952552:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2ddc3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23457792->23461887] (req 23457792->23461887) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.952562:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2ddc3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23457792->23461887] (req 23457792->23461887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.952569:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760727.952575:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2ddc3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23457792->23461887] (req 23457792->23461887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.952589:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2ddc3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23457792->23461887] (req 23457792->23461887) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.952611:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2ddc3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23457792->23461887] (req 23457792->23461887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.952619:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2ddc3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23457792->23461887] (req 23457792->23461887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760727.952745:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4058380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512592512:12345-192.168.203.56@tcp:4:dd.0 Request processed in 67753us (69442us total) trans 4294973046 rc 0/0 00000100:00100000:1.0:1789760727.952755:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5757 00000100:00100000:1.0:1789760727.952764:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 707396081508 time 707448039157 state 0x2 ref 0 00000100:00100000:1.0:1789760727.958203:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760727.958335:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512592640 00000100:00100000:0.0:1789760727.958345:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512592640 00000100:00100000:0.0:1789760727.958395:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 707396081508 time 707448039157 state 0x0 ref 1 00000100:00100000:0.0:1789760727.958410:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 707448299573 time 707448039157 state 0x1 ref 1 00000100:00100000:0.0:1789760727.958450:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 707448299573 time 707522748585 state 0x0 ref 1 00000100:00100000:0.0:1789760727.958462:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4131180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512592640:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760727.958512:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23461888, mtime: 0x6aad94d7, atime: 0x0, ctime: 0x6aad94d7, blocks: 0xb807 00010000:00010000:0.0:1789760727.958521:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2ddca lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.958540:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=707s v=(0 0) 00010000:00010000:0.0:1789760727.958545:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760727.958551:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2ddca lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23461888->23465983] (req 23461888->23465983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.958568:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2ddca lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23461888->23465983] (req 23461888->23465983) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760727.991542:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973047 is committed 00002000:00010000:1.0:1789760727.991686:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23461888 -> 23465984 00010000:00010000:1.0:1789760727.991736:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2ddca lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23461888->23465983] (req 23461888->23465983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.991749:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2ddca lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23461888->23465983] (req 23461888->23465983) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.991758:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2ddca lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23461888->23465983] (req 23461888->23465983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.991772:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760727.991778:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2ddca lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23461888->23465983] (req 23461888->23465983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.991787:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2ddca lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23461888->23465983] (req 23461888->23465983) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.991812:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2ddca lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23461888->23465983] (req 23461888->23465983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.991819:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2ddca lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23461888->23465983] (req 23461888->23465983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760727.991958:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4131180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512592640:12345-192.168.203.56@tcp:4:dd.0 Request processed in 33491us (33757us total) trans 4294973047 rc 0/0 00000100:00100000:1.0:1789760727.991970:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5758 00000100:00100000:1.0:1789760727.991980:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 707448299573 time 707522748585 state 0x2 ref 0 00000100:00100000:0.0:1789760727.997011:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760727.997211:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512592768 00000100:00100000:3.0:1789760727.997222:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512592768 00000100:00100000:3.0:1789760727.997275:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 707448299573 time 707522748585 state 0x0 ref 1 00000100:00100000:3.0:1789760727.997290:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 707523009001 time 707522748585 state 0x1 ref 1 00000100:00100000:3.0:1789760727.997351:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 707523009001 time 707561650762 state 0x0 ref 1 00000100:00100000:3.0:1789760727.997364:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5307100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512592768:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760727.997409:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23465984, mtime: 0x6aad94d7, atime: 0x0, ctime: 0x6aad94d7, blocks: 0xb807 00010000:00010000:3.0:1789760727.997418:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2ddd1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.997433:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=707s v=(0 0) 00010000:00010000:3.0:1789760727.997448:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760727.997454:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2ddd1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23465984->23470079] (req 23465984->23470079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.997475:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2ddd1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23465984->23470079] (req 23465984->23470079) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760728.027731:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973048 is committed 00002000:00010000:3.0:1789760728.027835:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23465984 -> 23470080 00010000:00010000:3.0:1789760728.027853:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2ddd1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23465984->23470079] (req 23465984->23470079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.027866:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2ddd1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23465984->23470079] (req 23465984->23470079) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.027876:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2ddd1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23465984->23470079] (req 23465984->23470079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.027886:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760728.027891:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2ddd1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23465984->23470079] (req 23465984->23470079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.027899:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2ddd1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23465984->23470079] (req 23465984->23470079) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.027919:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2ddd1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23465984->23470079] (req 23465984->23470079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.027927:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2ddd1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23465984->23470079] (req 23465984->23470079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760728.028025:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5307100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512592768:12345-192.168.203.56@tcp:4:dd.0 Request processed in 30658us (31019us total) trans 4294973048 rc 0/0 00000100:00100000:3.0:1789760728.028036:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5759 00000100:00100000:3.0:1789760728.028045:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 707523009001 time 707561650762 state 0x2 ref 0 00000100:00100000:2.0:1789760728.032259:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760728.032350:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512592896 00000100:00100000:1.0:1789760728.032360:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512592896 00000100:00100000:1.0:1789760728.032411:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 707523009001 time 707561650762 state 0x0 ref 1 00000100:00100000:1.0:1789760728.032426:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 707561911178 time 707561650762 state 0x1 ref 1 00000100:00100000:1.0:1789760728.032495:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 707561911178 time 707596794138 state 0x0 ref 1 00000100:00100000:1.0:1789760728.032512:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4059180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512592896:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760728.032570:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23470080, mtime: 0x6aad94d7, atime: 0x0, ctime: 0x6aad94d7, blocks: 0xb807 00010000:00010000:1.0:1789760728.032579:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2ddd8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.032596:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=707s v=(0 0) 00010000:00010000:1.0:1789760728.032601:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760728.032607:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2ddd8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23470080->23474175] (req 23470080->23474175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.032621:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2ddd8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23470080->23474175] (req 23470080->23474175) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760728.076592:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23470080 -> 23474176 00010000:00010000:3.0:1789760728.076612:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2ddd8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23470080->23474175] (req 23470080->23474175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.076626:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2ddd8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23470080->23474175] (req 23470080->23474175) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.076635:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2ddd8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23470080->23474175] (req 23470080->23474175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.076642:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760728.076648:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2ddd8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23470080->23474175] (req 23470080->23474175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.076658:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2ddd8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23470080->23474175] (req 23470080->23474175) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.076677:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2ddd8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23470080->23474175] (req 23470080->23474175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.076689:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2ddd8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23470080->23474175] (req 23470080->23474175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760728.076876:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973049 is committed 00000100:00100000:3.0:1789760728.077134:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4059180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512592896:12345-192.168.203.56@tcp:4:dd.0 Request processed in 44618us (44879us total) trans 4294973049 rc 0/0 00000100:00100000:3.0:1789760728.077144:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5760 00000100:00100000:3.0:1789760728.077154:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 707561911178 time 707596794138 state 0x2 ref 0 00000100:00100000:2.0:1789760728.082556:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760728.082627:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512593024 00000100:00100000:1.0:1789760728.082637:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512593024 00000100:00100000:1.0:1789760728.082689:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 707561911178 time 707596794138 state 0x0 ref 1 00000100:00100000:1.0:1789760728.082731:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 707597054554 time 707596794138 state 0x1 ref 1 00000100:00100000:1.0:1789760728.082757:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 707597054554 time 707647057197 state 0x0 ref 1 00000100:00100000:1.0:1789760728.082769:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4059500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512593024:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760728.082821:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23474176, mtime: 0x6aad94d7, atime: 0x0, ctime: 0x6aad94d7, blocks: 0xb807 00010000:00010000:1.0:1789760728.082841:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2dddf lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.082861:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=707s v=(0 0) 00010000:00010000:1.0:1789760728.082866:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760728.082875:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2dddf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23474176->23478271] (req 23474176->23478271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.082891:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2dddf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23474176->23478271] (req 23474176->23478271) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760728.125460:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973050 is committed 00002000:00010000:1.0:1789760728.125616:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23474176 -> 23478272 00010000:00010000:1.0:1789760728.125633:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2dddf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23474176->23478271] (req 23474176->23478271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.125647:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2dddf lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23474176->23478271] (req 23474176->23478271) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.125657:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2dddf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23474176->23478271] (req 23474176->23478271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.125665:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760728.125670:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2dddf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23474176->23478271] (req 23474176->23478271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.125680:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2dddf lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23474176->23478271] (req 23474176->23478271) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.126419:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2dddf lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23474176->23478271] (req 23474176->23478271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.126432:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2dddf lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23474176->23478271] (req 23474176->23478271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760728.126654:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4059500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512593024:12345-192.168.203.56@tcp:4:dd.0 Request processed in 43882us (44101us total) trans 4294973050 rc 0/0 00000100:00100000:1.0:1789760728.126665:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5761 00000100:00100000:1.0:1789760728.126675:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 707597054554 time 707647057197 state 0x2 ref 0 00000100:00100000:0.0:1789760728.131466:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760728.131546:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512593152 00000100:00100000:3.0:1789760728.131563:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512593152 00000100:00100000:3.0:1789760728.131617:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 707597054554 time 707647057197 state 0x0 ref 1 00000100:00100000:3.0:1789760728.131632:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 707647317613 time 707647057197 state 0x1 ref 1 00000100:00100000:3.0:1789760728.131662:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 707647317613 time 707695961873 state 0x0 ref 1 00000100:00100000:3.0:1789760728.131675:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5305f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512593152:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760728.131759:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23478272, mtime: 0x6aad94d7, atime: 0x0, ctime: 0x6aad94d7, blocks: 0xb807 00010000:00010000:3.0:1789760728.131768:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2dde6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.131794:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=707s v=(0 0) 00010000:00010000:3.0:1789760728.131799:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760728.131806:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2dde6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23478272->23482367] (req 23478272->23482367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.131823:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2dde6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23478272->23482367] (req 23478272->23482367) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760728.169515:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973051 is committed 00002000:00010000:3.0:1789760728.169670:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23478272 -> 23482368 00010000:00010000:3.0:1789760728.169685:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2dde6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23478272->23482367] (req 23478272->23482367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.169716:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2dde6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23478272->23482367] (req 23478272->23482367) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.169726:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2dde6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23478272->23482367] (req 23478272->23482367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.169734:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760728.169739:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2dde6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23478272->23482367] (req 23478272->23482367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.169748:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2dde6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23478272->23482367] (req 23478272->23482367) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.169766:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2dde6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23478272->23482367] (req 23478272->23482367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.169774:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2dde6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23478272->23482367] (req 23478272->23482367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760728.169880:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5305f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512593152:12345-192.168.203.56@tcp:4:dd.0 Request processed in 38203us (38420us total) trans 4294973051 rc 0/0 00000100:00100000:3.0:1789760728.169891:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5762 00000100:00100000:3.0:1789760728.169900:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 707647317613 time 707695961873 state 0x2 ref 0 00000100:00100000:2.0:1789760728.172958:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760728.173031:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512593280 00000100:00100000:3.0:1789760728.173040:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512593280 00000100:00100000:3.0:1789760728.173091:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 707647317613 time 707695961873 state 0x0 ref 1 00000100:00100000:3.0:1789760728.173105:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 707696222289 time 707695961873 state 0x1 ref 1 00000100:00100000:3.0:1789760728.173146:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 707696222289 time 707737445800 state 0x0 ref 1 00000100:00100000:3.0:1789760728.173158:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4058a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512593280:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760728.173206:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23482368, mtime: 0x6aad94d7, atime: 0x0, ctime: 0x6aad94d7, blocks: 0xb807 00010000:00010000:3.0:1789760728.173215:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2dded lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.173231:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=707s v=(0 0) 00010000:00010000:3.0:1789760728.173236:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760728.173241:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2dded lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23482368->23486463] (req 23482368->23486463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.173256:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2dded lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23482368->23486463] (req 23482368->23486463) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760728.220535:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973052 is committed 00002000:00010000:1.0:1789760728.221098:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23482368 -> 23486464 00010000:00010000:1.0:1789760728.221249:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2dded lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23482368->23486463] (req 23482368->23486463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.221397:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2dded lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23482368->23486463] (req 23482368->23486463) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.221406:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2dded lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23482368->23486463] (req 23482368->23486463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.221413:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760728.221418:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2dded lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23482368->23486463] (req 23482368->23486463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.221426:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2dded lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23482368->23486463] (req 23482368->23486463) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.221460:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2dded lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23482368->23486463] (req 23482368->23486463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.221467:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2dded lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23482368->23486463] (req 23482368->23486463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760728.221662:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4058a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512593280:12345-192.168.203.56@tcp:4:dd.0 Request processed in 48500us (48708us total) trans 4294973052 rc 0/0 00000100:00100000:1.0:1789760728.221673:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5763 00000100:00100000:1.0:1789760728.221683:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 707696222289 time 707737445800 state 0x2 ref 0 00000100:00100000:0.0:1789760728.227582:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760728.227772:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512593408 00000100:00100000:1.0:1789760728.227782:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512593408 00000100:00100000:1.0:1789760728.227901:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 707696222289 time 707737445800 state 0x0 ref 1 00000100:00100000:1.0:1789760728.227916:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 707737706216 time 707737445800 state 0x1 ref 1 00000100:00100000:1.0:1789760728.227978:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 707737706216 time 707792277408 state 0x0 ref 1 00000100:00100000:1.0:1789760728.227992:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5307480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512593408:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760728.228039:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23486464, mtime: 0x6aad94d7, atime: 0x0, ctime: 0x6aad94d7, blocks: 0xb807 00010000:00010000:1.0:1789760728.228048:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2ddf4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.228064:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=707s v=(0 0) 00010000:00010000:1.0:1789760728.228286:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760728.228295:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2ddf4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23486464->23490559] (req 23486464->23490559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.228311:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2ddf4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23486464->23490559] (req 23486464->23490559) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760728.296463:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973053 is committed 00002000:00010000:1.0:1789760728.296676:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23486464 -> 23490560 00010000:00010000:1.0:1789760728.296693:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2ddf4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23486464->23490559] (req 23486464->23490559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.296740:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2ddf4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23486464->23490559] (req 23486464->23490559) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.296749:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2ddf4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23486464->23490559] (req 23486464->23490559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.296767:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760728.296773:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2ddf4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23486464->23490559] (req 23486464->23490559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.296783:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2ddf4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23486464->23490559] (req 23486464->23490559) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.296802:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2ddf4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23486464->23490559] (req 23486464->23490559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.296815:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2ddf4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23486464->23490559] (req 23486464->23490559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760728.297235:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5307480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512593408:12345-192.168.203.56@tcp:4:dd.0 Request processed in 69239us (69656us total) trans 4294973053 rc 0/0 00000100:00100000:1.0:1789760728.297246:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5764 00000100:00100000:1.0:1789760728.297265:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 707737706216 time 707792277408 state 0x2 ref 0 00000100:00100000:0.0:1789760728.301467:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760728.301819:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512593536 00000100:00100000:3.0:1789760728.301839:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512593536 00000100:00100000:3.0:1789760728.301890:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 707737706216 time 707792277408 state 0x0 ref 1 00000100:00100000:3.0:1789760728.301905:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 707792537824 time 707792277408 state 0x1 ref 1 00000100:00100000:3.0:1789760728.301980:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 707792537824 time 707866279089 state 0x0 ref 1 00000100:00100000:3.0:1789760728.301993:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5304700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512593536:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760728.302057:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23490560, mtime: 0x6aad94d7, atime: 0x0, ctime: 0x6aad94d7, blocks: 0xb807 00010000:00010000:3.0:1789760728.302067:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2ddfb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.302083:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=707s v=(0 0) 00010000:00010000:3.0:1789760728.302088:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760728.302096:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2ddfb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23490560->23494655] (req 23490560->23494655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.302122:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2ddfb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23490560->23494655] (req 23490560->23494655) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760728.337897:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973054 is committed 00002000:00010000:1.0:1789760728.337999:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23490560 -> 23494656 00010000:00010000:1.0:1789760728.338014:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2ddfb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23490560->23494655] (req 23490560->23494655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.338027:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2ddfb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23490560->23494655] (req 23490560->23494655) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.338036:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2ddfb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23490560->23494655] (req 23490560->23494655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.338044:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760728.338049:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2ddfb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23490560->23494655] (req 23490560->23494655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.338058:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2ddfb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23490560->23494655] (req 23490560->23494655) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.338083:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2ddfb lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23490560->23494655] (req 23490560->23494655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.338091:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2ddfb lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23490560->23494655] (req 23490560->23494655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760728.338290:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5304700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512593536:12345-192.168.203.56@tcp:4:dd.0 Request processed in 36294us (36827us total) trans 4294973054 rc 0/0 00000100:00100000:1.0:1789760728.338301:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5765 00000100:00100000:1.0:1789760728.338311:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 707792537824 time 707866279089 state 0x2 ref 0 00000100:00100000:3.0:1789760728.341996:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760728.342056:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512593664 00000100:00100000:1.0:1789760728.342066:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512593664 00000100:00100000:1.0:1789760728.342130:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 707792537824 time 707866279089 state 0x0 ref 1 00000100:00100000:1.0:1789760728.342146:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 707866539505 time 707866279089 state 0x1 ref 1 00000100:00100000:1.0:1789760728.342175:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 707866539505 time 707906474760 state 0x0 ref 1 00000100:00100000:1.0:1789760728.342187:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac437e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512593664:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760728.342232:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23494656, mtime: 0x6aad94d7, atime: 0x0, ctime: 0x6aad94d7, blocks: 0xb807 00010000:00010000:1.0:1789760728.342241:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2de02 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.342267:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=707s v=(0 0) 00010000:00010000:1.0:1789760728.342273:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760728.342279:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2de02 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23494656->23498751] (req 23494656->23498751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.342294:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2de02 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23494656->23498751] (req 23494656->23498751) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760728.379980:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973055 is committed 00002000:00010000:0.0:1789760728.380618:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23494656 -> 23498752 00010000:00010000:0.0:1789760728.380634:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2de02 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23494656->23498751] (req 23494656->23498751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.380646:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2de02 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23494656->23498751] (req 23494656->23498751) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.380655:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2de02 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23494656->23498751] (req 23494656->23498751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.380662:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760728.380667:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2de02 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23494656->23498751] (req 23494656->23498751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.380675:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2de02 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23494656->23498751] (req 23494656->23498751) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.380713:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2de02 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23494656->23498751] (req 23494656->23498751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.380721:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2de02 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23494656->23498751] (req 23494656->23498751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760728.380858:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac437e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512593664:12345-192.168.203.56@tcp:4:dd.0 Request processed in 38668us (38866us total) trans 4294973055 rc 0/0 00000100:00100000:0.0:1789760728.380869:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5766 00000100:00100000:0.0:1789760728.380879:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 707866539505 time 707906474760 state 0x2 ref 0 00000100:00100000:2.0:1789760728.383955:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760728.384176:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512593792 00000100:00100000:1.0:1789760728.384188:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512593792 00000100:00100000:1.0:1789760728.384248:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 707866539505 time 707906474760 state 0x0 ref 1 00000100:00100000:1.0:1789760728.384264:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 707906735176 time 707906474760 state 0x1 ref 1 00000100:00100000:1.0:1789760728.384326:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 707906735176 time 707948625773 state 0x0 ref 1 00000100:00100000:1.0:1789760728.384340:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac405ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512593792:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760728.384410:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23498752, mtime: 0x6aad94d7, atime: 0x0, ctime: 0x6aad94d7, blocks: 0xb807 00010000:00010000:1.0:1789760728.384598:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2de09 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.384619:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=707s v=(0 0) 00010000:00010000:1.0:1789760728.384623:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760728.384630:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2de09 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23498752->23502847] (req 23498752->23502847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.384646:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2de09 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23498752->23502847] (req 23498752->23502847) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760728.428556:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973056 is committed 00002000:00010000:1.0:1789760728.433610:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23498752 -> 23502848 00010000:00010000:1.0:1789760728.433630:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2de09 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23498752->23502847] (req 23498752->23502847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.433643:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2de09 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23498752->23502847] (req 23498752->23502847) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.433652:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2de09 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23498752->23502847] (req 23498752->23502847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.433660:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760728.433664:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2de09 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23498752->23502847] (req 23498752->23502847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.433673:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2de09 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23498752->23502847] (req 23498752->23502847) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.433693:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2de09 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23498752->23502847] (req 23498752->23502847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.433716:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2de09 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23498752->23502847] (req 23498752->23502847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760728.433887:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac405ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512593792:12345-192.168.203.56@tcp:4:dd.0 Request processed in 49543us (49937us total) trans 4294973056 rc 0/0 00000100:00100000:1.0:1789760728.433901:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5767 00000100:00100000:1.0:1789760728.433911:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 707906735176 time 707948625773 state 0x2 ref 0 00010000:00010000:0.0:1789760728.434855:0:7647:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789760728.437931:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760728.437978:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512593920 00000100:00100000:1.0:1789760728.437987:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512593920 00000100:00100000:1.0:1789760728.438051:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 707906735176 time 707948625773 state 0x0 ref 1 00000100:00100000:1.0:1789760728.438068:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 707948886189 time 707948625773 state 0x1 ref 1 00000100:00100000:1.0:1789760728.438099:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 707948886189 time 708002398475 state 0x0 ref 1 00000100:00100000:1.0:1789760728.438120:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4058e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512593920:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760728.438370:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23502848, mtime: 0x6aad94d7, atime: 0x0, ctime: 0x6aad94d7, blocks: 0xb807 00010000:00010000:1.0:1789760728.438380:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2de10 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.438395:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=708s v=(0 0) 00010000:00010000:1.0:1789760728.438399:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760728.438405:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2de10 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23502848->23506943] (req 23502848->23506943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.438421:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2de10 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23502848->23506943] (req 23502848->23506943) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760728.484731:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973057 is committed 00002000:00010000:1.0:1789760728.484868:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23502848 -> 23506944 00010000:00010000:1.0:1789760728.484884:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2de10 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23502848->23506943] (req 23502848->23506943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.484898:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2de10 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23502848->23506943] (req 23502848->23506943) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.484907:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2de10 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23502848->23506943] (req 23502848->23506943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.484914:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760728.484918:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2de10 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23502848->23506943] (req 23502848->23506943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.484927:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2de10 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23502848->23506943] (req 23502848->23506943) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.484948:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2de10 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23502848->23506943] (req 23502848->23506943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.484957:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2de10 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23502848->23506943] (req 23502848->23506943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760728.485060:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4058e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512593920:12345-192.168.203.56@tcp:4:dd.0 Request processed in 46938us (47132us total) trans 4294973057 rc 0/0 00000100:00100000:1.0:1789760728.485080:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5768 00000100:00100000:1.0:1789760728.485090:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 707948886189 time 708002398475 state 0x2 ref 0 00000100:00100000:3.0:1789760728.490341:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760728.490495:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512594048 00000100:00100000:0.0:1789760728.490505:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512594048 00000100:00100000:0.0:1789760728.490570:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 707948886189 time 708002398475 state 0x0 ref 1 00000100:00100000:0.0:1789760728.490587:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 708002658891 time 708002398475 state 0x1 ref 1 00000100:00100000:0.0:1789760728.490617:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 708002658891 time 708054916895 state 0x0 ref 1 00000100:00100000:0.0:1789760728.490630:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac437e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512594048:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760728.490678:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23506944, mtime: 0x6aad94d7, atime: 0x0, ctime: 0x6aad94d7, blocks: 0xb807 00010000:00010000:0.0:1789760728.490687:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2de17 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.490737:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=708s v=(0 0) 00010000:00010000:0.0:1789760728.490745:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760728.490752:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2de17 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23506944->23511039] (req 23506944->23511039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.490766:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2de17 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23506944->23511039] (req 23506944->23511039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760728.524655:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973058 is committed 00002000:00010000:1.0:1789760728.524760:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23506944 -> 23511040 00002000:00010000:1.0:1789760728.524765:0:13062:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760727 -> 1789760728 00002000:00010000:1.0:1789760728.524768:0:13062:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760727 -> 1789760728 00010000:00010000:1.0:1789760728.524782:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2de17 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23506944->23511039] (req 23506944->23511039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.524794:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2de17 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23506944->23511039] (req 23506944->23511039) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.524802:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2de17 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23506944->23511039] (req 23506944->23511039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.524809:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760728.524815:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2de17 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23506944->23511039] (req 23506944->23511039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.524823:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2de17 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23506944->23511039] (req 23506944->23511039) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.524843:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2de17 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23506944->23511039] (req 23506944->23511039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.524852:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2de17 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23506944->23511039] (req 23506944->23511039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760728.524980:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac437e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512594048:12345-192.168.203.56@tcp:4:dd.0 Request processed in 34347us (34643us total) trans 4294973058 rc 0/0 00000100:00100000:1.0:1789760728.524989:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5769 00000100:00100000:1.0:1789760728.525000:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 708002658891 time 708054916895 state 0x2 ref 0 00000100:00100000:2.0:1789760728.529563:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760728.529823:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512594176 00000100:00100000:1.0:1789760728.529833:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512594176 00000100:00100000:1.0:1789760728.530265:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 708002658891 time 708054916895 state 0x0 ref 1 00000100:00100000:1.0:1789760728.530282:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 708055177311 time 708054916895 state 0x1 ref 1 00000100:00100000:1.0:1789760728.530602:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 708055177311 time 708094899489 state 0x0 ref 1 00000100:00100000:1.0:1789760728.530617:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febfb89ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512594176:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760728.530666:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23511040, mtime: 0x6aad94d8, atime: 0x0, ctime: 0x6aad94d8, blocks: 0xb807 00010000:00010000:1.0:1789760728.530675:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2de1e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.530691:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=708s v=(0 0) 00010000:00010000:1.0:1789760728.530731:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760728.530738:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2de1e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23511040->23515135] (req 23511040->23515135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.530752:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2de1e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23511040->23515135] (req 23511040->23515135) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760728.564554:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973059 is committed 00002000:00010000:3.0:1789760728.564719:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23511040 -> 23515136 00010000:00010000:3.0:1789760728.564734:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2de1e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23511040->23515135] (req 23511040->23515135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.564748:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2de1e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23511040->23515135] (req 23511040->23515135) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.564757:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2de1e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23511040->23515135] (req 23511040->23515135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.564765:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760728.564771:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2de1e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23511040->23515135] (req 23511040->23515135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.564780:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2de1e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23511040->23515135] (req 23511040->23515135) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.564798:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2de1e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23511040->23515135] (req 23511040->23515135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.564806:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2de1e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23511040->23515135] (req 23511040->23515135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760728.564893:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febfb89ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512594176:12345-192.168.203.56@tcp:4:dd.0 Request processed in 34276us (35336us total) trans 4294973059 rc 0/0 00000100:00100000:3.0:1789760728.564904:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5770 00000100:00100000:3.0:1789760728.564913:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 708055177311 time 708094899489 state 0x2 ref 0 00000100:00100000:2.0:1789760728.570355:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760728.570585:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512594304 00000100:00100000:1.0:1789760728.570595:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512594304 00000100:00100000:1.0:1789760728.570649:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 708055177311 time 708094899489 state 0x0 ref 1 00000100:00100000:1.0:1789760728.570663:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 708095159905 time 708094899489 state 0x1 ref 1 00000100:00100000:1.0:1789760728.570767:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 708095159905 time 708135064988 state 0x0 ref 1 00000100:00100000:1.0:1789760728.570781:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febfb89fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512594304:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760728.570829:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23515136, mtime: 0x6aad94d8, atime: 0x0, ctime: 0x6aad94d8, blocks: 0xb807 00010000:00010000:1.0:1789760728.570838:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2de25 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.570852:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=708s v=(0 0) 00010000:00010000:1.0:1789760728.570857:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760728.570863:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2de25 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23515136->23519231] (req 23515136->23519231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.570877:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2de25 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23515136->23519231] (req 23515136->23519231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760728.622282:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973060 is committed 00002000:00010000:0.0:1789760728.622887:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23515136 -> 23519232 00010000:00010000:0.0:1789760728.622904:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2de25 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23515136->23519231] (req 23515136->23519231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.622918:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2de25 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23515136->23519231] (req 23515136->23519231) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.622927:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2de25 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23515136->23519231] (req 23515136->23519231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.622935:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760728.623038:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2de25 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23515136->23519231] (req 23515136->23519231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.623049:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2de25 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23515136->23519231] (req 23515136->23519231) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.623075:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2de25 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23515136->23519231] (req 23515136->23519231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.623083:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2de25 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23515136->23519231] (req 23515136->23519231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760728.623352:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febfb89fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512594304:12345-192.168.203.56@tcp:4:dd.0 Request processed in 52569us (53001us total) trans 4294973060 rc 0/0 00000100:00100000:0.0:1789760728.623364:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5771 00000100:00100000:0.0:1789760728.623373:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 708095159905 time 708135064988 state 0x2 ref 0 00000100:00100000:0.0:1789760728.626349:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760728.626501:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512594432 00000100:00100000:1.0:1789760728.626512:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512594432 00000100:00100000:1.0:1789760728.626574:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 708095159905 time 708135064988 state 0x0 ref 1 00000100:00100000:1.0:1789760728.626592:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 708135325404 time 708135064988 state 0x1 ref 1 00000100:00100000:1.0:1789760728.626653:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 708135325404 time 708190951338 state 0x0 ref 1 00000100:00100000:1.0:1789760728.626666:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5306a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512594432:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760728.626746:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23519232, mtime: 0x6aad94d8, atime: 0x0, ctime: 0x6aad94d8, blocks: 0xb807 00010000:00010000:1.0:1789760728.626756:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2de2c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.626772:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=708s v=(0 0) 00010000:00010000:1.0:1789760728.626776:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760728.626782:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2de2c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23519232->23523327] (req 23519232->23523327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.626796:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2de2c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23519232->23523327] (req 23519232->23523327) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760728.663321:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973061 is committed 00002000:00010000:0.0:1789760728.665412:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23519232 -> 23523328 00010000:00010000:0.0:1789760728.665442:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2de2c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23519232->23523327] (req 23519232->23523327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.665456:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2de2c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23519232->23523327] (req 23519232->23523327) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.665466:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2de2c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23519232->23523327] (req 23519232->23523327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.665475:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760728.665481:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2de2c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23519232->23523327] (req 23519232->23523327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.665497:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2de2c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23519232->23523327] (req 23519232->23523327) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.665517:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2de2c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23519232->23523327] (req 23519232->23523327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.665524:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2de2c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23519232->23523327] (req 23519232->23523327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760728.665626:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5306a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512594432:12345-192.168.203.56@tcp:4:dd.0 Request processed in 38957us (39283us total) trans 4294973061 rc 0/0 00000100:00100000:0.0:1789760728.665638:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5772 00000100:00100000:0.0:1789760728.665648:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 708135325404 time 708190951338 state 0x2 ref 0 00000100:00100000:2.0:1789760728.670908:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760728.671139:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512594560 00000100:00100000:1.0:1789760728.671150:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512594560 00000100:00100000:1.0:1789760728.671200:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 708135325404 time 708190951338 state 0x0 ref 1 00000100:00100000:1.0:1789760728.671216:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 708191211754 time 708190951338 state 0x1 ref 1 00000100:00100000:1.0:1789760728.671248:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 708191211754 time 708235547682 state 0x0 ref 1 00000100:00100000:1.0:1789760728.671260:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febffb9f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512594560:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760728.671314:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23523328, mtime: 0x6aad94d8, atime: 0x0, ctime: 0x6aad94d8, blocks: 0xb807 00010000:00010000:1.0:1789760728.671324:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2de33 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.671341:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=708s v=(0 0) 00010000:00010000:1.0:1789760728.671345:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760728.671351:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2de33 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23523328->23527423] (req 23523328->23527423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.671365:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2de33 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23523328->23527423] (req 23523328->23527423) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760728.714577:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973062 is committed 00002000:00010000:1.0:1789760728.714653:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23523328 -> 23527424 00010000:00010000:1.0:1789760728.714670:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2de33 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23523328->23527423] (req 23523328->23527423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.714683:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2de33 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23523328->23527423] (req 23523328->23527423) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.714692:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2de33 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23523328->23527423] (req 23523328->23527423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.714762:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760728.714767:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2de33 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23523328->23527423] (req 23523328->23527423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.714777:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2de33 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23523328->23527423] (req 23523328->23527423) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.714797:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2de33 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23523328->23527423] (req 23523328->23527423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.714804:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2de33 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23523328->23527423] (req 23523328->23527423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760728.714929:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febffb9f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512594560:12345-192.168.203.56@tcp:4:dd.0 Request processed in 43667us (44025us total) trans 4294973062 rc 0/0 00000100:00100000:1.0:1789760728.714940:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5773 00000100:00100000:1.0:1789760728.714949:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 708191211754 time 708235547682 state 0x2 ref 0 00000100:00100000:2.0:1789760728.717569:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760728.717664:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512594688 00000100:00100000:0.0:1789760728.717674:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512594688 00000100:00100000:0.0:1789760728.717812:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 708191211754 time 708235547682 state 0x0 ref 1 00000100:00100000:0.0:1789760728.717919:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 708235808098 time 708235547682 state 0x1 ref 1 00000100:00100000:0.0:1789760728.717982:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 708235808098 time 708282280041 state 0x0 ref 1 00000100:00100000:0.0:1789760728.717997:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febffb9ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512594688:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760728.718046:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23527424, mtime: 0x6aad94d8, atime: 0x0, ctime: 0x6aad94d8, blocks: 0xb807 00010000:00010000:0.0:1789760728.718056:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2de3a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.718071:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=708s v=(0 0) 00010000:00010000:0.0:1789760728.718075:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760728.718082:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2de3a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23527424->23531519] (req 23527424->23531519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.718096:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2de3a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23527424->23531519] (req 23527424->23531519) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760728.753596:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973063 is committed 00002000:00010000:0.0:1789760728.753613:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23527424 -> 23531520 00010000:00010000:0.0:1789760728.753633:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2de3a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23527424->23531519] (req 23527424->23531519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.753647:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2de3a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23527424->23531519] (req 23527424->23531519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.753653:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2de3a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23527424->23531519] (req 23527424->23531519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.753658:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760728.753662:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2de3a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23527424->23531519] (req 23527424->23531519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.753668:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2de3a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23527424->23531519] (req 23527424->23531519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.753687:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2de3a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23527424->23531519] (req 23527424->23531519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.753692:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2de3a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23527424->23531519] (req 23527424->23531519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760728.753807:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febffb9ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512594688:12345-192.168.203.56@tcp:4:dd.0 Request processed in 35808us (36242us total) trans 4294973063 rc 0/0 00000100:00100000:0.0:1789760728.753817:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5774 00000100:00100000:0.0:1789760728.753906:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 708235808098 time 708282280041 state 0x2 ref 0 00000100:00100000:0.0:1789760728.757186:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760728.757521:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512594816 00000100:00100000:1.0:1789760728.757531:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512594816 00000100:00100000:1.0:1789760728.757586:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 708235808098 time 708282280041 state 0x0 ref 1 00000100:00100000:1.0:1789760728.757603:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 708282540457 time 708282280041 state 0x1 ref 1 00000100:00100000:1.0:1789760728.757632:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 708282540457 time 708321932963 state 0x0 ref 1 00000100:00100000:1.0:1789760728.757645:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febfbddc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512594816:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760728.757735:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23531520, mtime: 0x6aad94d8, atime: 0x0, ctime: 0x6aad94d8, blocks: 0xb807 00010000:00010000:1.0:1789760728.757745:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2de41 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.757761:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=708s v=(0 0) 00010000:00010000:1.0:1789760728.757766:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760728.757772:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2de41 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23531520->23535615] (req 23531520->23535615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.757785:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2de41 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23531520->23535615] (req 23531520->23535615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760728.799678:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973064 is committed 00002000:00010000:0.0:1789760728.799812:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23531520 -> 23535616 00010000:00010000:0.0:1789760728.799828:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2de41 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23531520->23535615] (req 23531520->23535615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.799841:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2de41 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23531520->23535615] (req 23531520->23535615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.799852:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2de41 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23531520->23535615] (req 23531520->23535615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.799860:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760728.799865:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2de41 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23531520->23535615] (req 23531520->23535615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.799881:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2de41 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23531520->23535615] (req 23531520->23535615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.799901:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2de41 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23531520->23535615] (req 23531520->23535615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.799920:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2de41 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23531520->23535615] (req 23531520->23535615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760728.800016:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febfbddc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512594816:12345-192.168.203.56@tcp:4:dd.0 Request processed in 42369us (42835us total) trans 4294973064 rc 0/0 00000100:00100000:0.0:1789760728.800027:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5775 00000100:00100000:0.0:1789760728.800035:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 708282540457 time 708321932963 state 0x2 ref 0 00000100:00100000:2.0:1789760728.806508:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760728.806611:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512594944 00000100:00100000:1.0:1789760728.806621:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512594944 00000100:00100000:1.0:1789760728.806676:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 708282540457 time 708321932963 state 0x0 ref 1 00000100:00100000:1.0:1789760728.806692:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 708322193379 time 708321932963 state 0x1 ref 1 00000100:00100000:1.0:1789760728.806765:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 708322193379 time 708371064774 state 0x0 ref 1 00000100:00100000:1.0:1789760728.806778:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febffb9c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512594944:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760728.806839:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23535616, mtime: 0x6aad94d8, atime: 0x0, ctime: 0x6aad94d8, blocks: 0xb807 00010000:00010000:1.0:1789760728.806847:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2de48 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.806863:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=708s v=(0 0) 00010000:00010000:1.0:1789760728.806866:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760728.806872:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2de48 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23535616->23539711] (req 23535616->23539711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.806886:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2de48 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23535616->23539711] (req 23535616->23539711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760728.850492:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23535616 -> 23539712 00010000:00010000:1.0:1789760728.850513:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2de48 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23535616->23539711] (req 23535616->23539711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.850532:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2de48 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23535616->23539711] (req 23535616->23539711) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.850542:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2de48 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23535616->23539711] (req 23535616->23539711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.850549:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760728.850554:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2de48 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23535616->23539711] (req 23535616->23539711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.850563:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2de48 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23535616->23539711] (req 23535616->23539711) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.850585:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2de48 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23535616->23539711] (req 23535616->23539711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.850593:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2de48 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23535616->23539711] (req 23535616->23539711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760728.850685:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febffb9c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2360:x1876699512594944:12345-192.168.203.56@tcp:4:dd.0 Request processed in 43903us (44182us total) trans 4294973065 rc 0/0 00000100:00100000:1.0:1789760728.850729:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5776 00000100:00100000:1.0:1789760728.850739:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 708322193379 time 708371064774 state 0x2 ref 0 00000001:00080000:0.0:1789760728.851928:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973065 is committed 00000100:00100000:3.0:1789760728.852980:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760728.853069:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512595072 00000100:00100000:0.0:1789760728.853078:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512595072 00000100:00100000:0.0:1789760728.853242:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 708322193379 time 708371064774 state 0x0 ref 1 00000100:00100000:0.0:1789760728.853257:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 708371325190 time 708371064774 state 0x1 ref 1 00000100:00100000:0.0:1789760728.853320:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 708371325190 time 708417619413 state 0x0 ref 1 00000100:00100000:0.0:1789760728.853334:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac437ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512595072:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760728.853379:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23539712, mtime: 0x6aad94d8, atime: 0x0, ctime: 0x6aad94d8, blocks: 0xb807 00010000:00010000:0.0:1789760728.853389:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2de4f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.853404:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=708s v=(0 0) 00010000:00010000:0.0:1789760728.853408:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760728.853415:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2de4f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23539712->23543807] (req 23539712->23543807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.853441:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2de4f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23539712->23543807] (req 23539712->23543807) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760728.884206:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973066 is committed 00002000:00010000:0.0:1789760728.884296:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23539712 -> 23543808 00010000:00010000:0.0:1789760728.884311:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2de4f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23539712->23543807] (req 23539712->23543807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.884325:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2de4f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23539712->23543807] (req 23539712->23543807) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.884334:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2de4f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23539712->23543807] (req 23539712->23543807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.884342:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760728.884347:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2de4f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23539712->23543807] (req 23539712->23543807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.884357:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2de4f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23539712->23543807] (req 23539712->23543807) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.884382:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2de4f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23539712->23543807] (req 23539712->23543807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.884390:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2de4f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23539712->23543807] (req 23539712->23543807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760728.884562:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac437ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512595072:12345-192.168.203.56@tcp:4:dd.0 Request processed in 31160us (31519us total) trans 4294973066 rc 0/0 00000100:00100000:0.0:1789760728.884574:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5777 00000100:00100000:0.0:1789760728.884584:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 708371325190 time 708417619413 state 0x2 ref 0 00000100:00100000:3.0:1789760728.887387:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760728.888102:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512595200 00000100:00100000:0.0:1789760728.888114:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512595200 00000100:00100000:0.0:1789760728.888170:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 708371325190 time 708417619413 state 0x0 ref 1 00000100:00100000:0.0:1789760728.888186:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 708417879829 time 708417619413 state 0x1 ref 1 00000100:00100000:0.0:1789760728.888217:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 708417879829 time 708452516618 state 0x0 ref 1 00000100:00100000:0.0:1789760728.888229:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac437df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512595200:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760728.888278:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23543808, mtime: 0x6aad94d8, atime: 0x0, ctime: 0x6aad94d8, blocks: 0xb807 00010000:00010000:0.0:1789760728.888287:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2de56 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.888304:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=708s v=(0 0) 00010000:00010000:0.0:1789760728.888309:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760728.888315:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2de56 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23543808->23547903] (req 23543808->23547903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.888329:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2de56 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23543808->23547903] (req 23543808->23547903) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760728.914410:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973067 is committed 00002000:00010000:0.0:1789760728.914566:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23543808 -> 23547904 00010000:00010000:0.0:1789760728.914581:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2de56 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23543808->23547903] (req 23543808->23547903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.914594:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2de56 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23543808->23547903] (req 23543808->23547903) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.914604:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2de56 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23543808->23547903] (req 23543808->23547903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.914611:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760728.914616:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2de56 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23543808->23547903] (req 23543808->23547903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.914625:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2de56 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23543808->23547903] (req 23543808->23547903) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.914644:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2de56 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23543808->23547903] (req 23543808->23547903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.914658:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2de56 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23543808->23547903] (req 23543808->23547903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760728.914841:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac437df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512595200:12345-192.168.203.56@tcp:4:dd.0 Request processed in 26609us (27470us total) trans 4294973067 rc 0/0 00000100:00100000:0.0:1789760728.914851:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5778 00000100:00100000:0.0:1789760728.914861:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 708417879829 time 708452516618 state 0x2 ref 0 00000100:00100000:3.0:1789760728.917665:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760728.917992:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512595328 00000100:00100000:1.0:1789760728.918002:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512595328 00000100:00100000:1.0:1789760728.918051:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 708417879829 time 708452516618 state 0x0 ref 1 00000100:00100000:1.0:1789760728.918065:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 708452777034 time 708452516618 state 0x1 ref 1 00000100:00100000:1.0:1789760728.918094:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 708452777034 time 708482394287 state 0x0 ref 1 00000100:00100000:1.0:1789760728.918115:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac437fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512595328:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760728.918159:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23547904, mtime: 0x6aad94d8, atime: 0x0, ctime: 0x6aad94d8, blocks: 0xb807 00010000:00010000:1.0:1789760728.918169:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2de5d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.918185:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=708s v=(0 0) 00010000:00010000:1.0:1789760728.918189:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760728.918195:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2de5d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23547904->23551999] (req 23547904->23551999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.918209:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2de5d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23547904->23551999] (req 23547904->23551999) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760728.957076:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973068 is committed 00002000:00010000:0.0:1789760728.957342:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23547904 -> 23552000 00010000:00010000:0.0:1789760728.957358:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2de5d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23547904->23551999] (req 23547904->23551999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.957371:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2de5d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23547904->23551999] (req 23547904->23551999) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.957381:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2de5d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23547904->23551999] (req 23547904->23551999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.957389:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760728.957394:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2de5d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23547904->23551999] (req 23547904->23551999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.957403:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2de5d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23547904->23551999] (req 23547904->23551999) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.957426:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2de5d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23547904->23551999] (req 23547904->23551999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.957444:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2de5d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23547904->23551999] (req 23547904->23551999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760728.957569:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac437fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512595328:12345-192.168.203.56@tcp:4:dd.0 Request processed in 39459us (39907us total) trans 4294973068 rc 0/0 00000100:00100000:0.0:1789760728.957580:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5779 00000100:00100000:0.0:1789760728.957590:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 708452777034 time 708482394287 state 0x2 ref 0 00000100:00100000:3.0:1789760728.960601:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760728.961815:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512595456 00000100:00100000:0.0:1789760728.961938:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512595456 00000100:00100000:0.0:1789760728.962073:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 708452777034 time 708482394287 state 0x0 ref 1 00000100:00100000:0.0:1789760728.962089:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 708482654703 time 708482394287 state 0x1 ref 1 00000100:00100000:0.0:1789760728.962123:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 708482654703 time 708526421939 state 0x0 ref 1 00000100:00100000:0.0:1789760728.962137:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac437f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512595456:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760728.962184:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23552000, mtime: 0x6aad94d8, atime: 0x0, ctime: 0x6aad94d8, blocks: 0xb807 00010000:00010000:0.0:1789760728.962193:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2de64 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.962209:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=708s v=(0 0) 00010000:00010000:0.0:1789760728.962214:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760728.962220:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2de64 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23552000->23556095] (req 23552000->23556095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.962234:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2de64 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23552000->23556095] (req 23552000->23556095) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760728.992467:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973069 is committed 00002000:00010000:0.0:1789760728.992586:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23552000 -> 23556096 00010000:00010000:0.0:1789760728.992602:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2de64 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23552000->23556095] (req 23552000->23556095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.992615:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2de64 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23552000->23556095] (req 23552000->23556095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.992624:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2de64 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23552000->23556095] (req 23552000->23556095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.992632:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760728.992637:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2de64 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23552000->23556095] (req 23552000->23556095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.992646:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2de64 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23552000->23556095] (req 23552000->23556095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.992667:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2de64 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23552000->23556095] (req 23552000->23556095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.992676:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2de64 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23552000->23556095] (req 23552000->23556095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760728.992956:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac437f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512595456:12345-192.168.203.56@tcp:4:dd.0 Request processed in 30816us (32357us total) trans 4294973069 rc 0/0 00000100:00100000:0.0:1789760728.992967:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5780 00000100:00100000:0.0:1789760728.992976:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 708482654703 time 708526421939 state 0x2 ref 0 00000100:00100000:2.0:1789760728.997370:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760728.997874:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512595584 00000100:00100000:0.0:1789760728.997886:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512595584 00000100:00100000:2.0:1789760728.998576:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 708482654703 time 708526421939 state 0x0 ref 1 00000100:00100000:2.0:1789760728.998592:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 708526682355 time 708526421939 state 0x1 ref 1 00000100:00100000:2.0:1789760728.998988:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 708526682355 time 708563285961 state 0x0 ref 1 00000100:00100000:2.0:1789760728.999004:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febffb9c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512595584:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760728.999054:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23556096, mtime: 0x6aad94d8, atime: 0x0, ctime: 0x6aad94d8, blocks: 0xb807 00010000:00010000:2.0:1789760728.999063:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2de6b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.999079:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=708s v=(0 0) 00010000:00010000:2.0:1789760728.999084:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760728.999090:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2de6b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23556096->23560191] (req 23556096->23560191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.999109:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2de6b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23556096->23560191] (req 23556096->23560191) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760729.042974:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973070 is committed 00002000:00010000:3.0:1789760729.043185:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23556096 -> 23560192 00010000:00010000:3.0:1789760729.043202:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2de6b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23556096->23560191] (req 23556096->23560191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.043215:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2de6b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23556096->23560191] (req 23556096->23560191) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.043300:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2de6b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23556096->23560191] (req 23556096->23560191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.043311:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760729.043316:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2de6b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23556096->23560191] (req 23556096->23560191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.043326:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2de6b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23556096->23560191] (req 23556096->23560191) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.043348:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2de6b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23556096->23560191] (req 23556096->23560191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.043355:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2de6b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23556096->23560191] (req 23556096->23560191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760729.043495:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febffb9c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512595584:12345-192.168.203.56@tcp:4:dd.0 Request processed in 44489us (46131us total) trans 4294973070 rc 0/0 00000100:00100000:3.0:1789760729.043507:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5781 00000100:00100000:3.0:1789760729.043517:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 708526682355 time 708563285961 state 0x2 ref 0 00000100:00100000:3.0:1789760729.049390:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760729.049518:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512595712 00000100:00100000:0.0:1789760729.049528:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512595712 00000100:00100000:0.0:1789760729.049608:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 708526682355 time 708563285961 state 0x0 ref 1 00000100:00100000:0.0:1789760729.049624:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 708563546377 time 708563285961 state 0x1 ref 1 00000100:00100000:0.0:1789760729.049664:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 708563546377 time 708613963682 state 0x0 ref 1 00000100:00100000:0.0:1789760729.049680:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac437d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512595712:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760729.049772:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23560192, mtime: 0x6aad94d8, atime: 0x0, ctime: 0x6aad94d8, blocks: 0xb807 00010000:00010000:0.0:1789760729.049782:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2de72 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.049799:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=708s v=(0 0) 00010000:00010000:0.0:1789760729.049804:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760729.049810:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2de72 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23560192->23564287] (req 23560192->23564287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.049829:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2de72 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23560192->23564287] (req 23560192->23564287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:1.0:1789760729.050280:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:3.0:1789760729.095027:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973071 is committed 00002000:00010000:1.0:1789760729.095229:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23560192 -> 23564288 00010000:00010000:1.0:1789760729.095244:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2de72 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23560192->23564287] (req 23560192->23564287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.095258:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2de72 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23560192->23564287] (req 23560192->23564287) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.095267:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2de72 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23560192->23564287] (req 23560192->23564287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.095274:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760729.095280:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2de72 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23560192->23564287] (req 23560192->23564287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.095290:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2de72 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23560192->23564287] (req 23560192->23564287) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.095307:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2de72 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23560192->23564287] (req 23560192->23564287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.095314:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2de72 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23560192->23564287] (req 23560192->23564287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760729.095437:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac437d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512595712:12345-192.168.203.56@tcp:4:dd.0 Request processed in 45748us (46060us total) trans 4294973071 rc 0/0 00000100:00100000:1.0:1789760729.095448:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5782 00000100:00100000:1.0:1789760729.095458:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 708563546377 time 708613963682 state 0x2 ref 0 00000100:00100000:2.0:1789760729.098954:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760729.101216:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512595840 00000100:00100000:0.0:1789760729.101229:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512595840 00000100:00100000:0.0:1789760729.101289:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 708563546377 time 708613963682 state 0x0 ref 1 00000100:00100000:0.0:1789760729.101304:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 708614224098 time 708613963682 state 0x1 ref 1 00000100:00100000:0.0:1789760729.101336:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 708614224098 time 708665635607 state 0x0 ref 1 00000100:00100000:0.0:1789760729.101348:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febea4b8450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512595840:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760729.101396:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23564288, mtime: 0x6aad94d8, atime: 0x0, ctime: 0x6aad94d8, blocks: 0xb807 00010000:00010000:0.0:1789760729.101404:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2de79 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.101420:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=708s v=(0 0) 00010000:00010000:0.0:1789760729.101424:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760729.101442:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2de79 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23564288->23568383] (req 23564288->23568383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.101456:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2de79 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23564288->23568383] (req 23564288->23568383) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760729.136233:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23564288 -> 23568384 00010000:00010000:1.0:1789760729.136254:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2de79 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23564288->23568383] (req 23564288->23568383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.136268:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2de79 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23564288->23568383] (req 23564288->23568383) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.136277:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2de79 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23564288->23568383] (req 23564288->23568383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.136285:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760729.136290:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2de79 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23564288->23568383] (req 23564288->23568383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.136299:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2de79 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23564288->23568383] (req 23564288->23568383) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.136317:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2de79 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23564288->23568383] (req 23564288->23568383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.136325:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2de79 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23564288->23568383] (req 23564288->23568383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760729.136413:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febea4b8450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2359:x1876699512595840:12345-192.168.203.56@tcp:4:dd.0 Request processed in 35063us (37464us total) trans 4294973072 rc 0/0 00000100:00100000:1.0:1789760729.136423:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5783 00000100:00100000:1.0:1789760729.136442:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 708614224098 time 708665635607 state 0x2 ref 0 00000001:00080000:3.0:1789760729.137183:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973072 is committed 00000100:00100000:2.0:1789760729.145349:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760729.145451:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512595968 00000100:00100000:3.0:1789760729.145460:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512595968 00000100:00100000:3.0:1789760729.145514:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 708614224098 time 708665635607 state 0x0 ref 1 00000100:00100000:3.0:1789760729.145529:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 708665896023 time 708665635607 state 0x1 ref 1 00000100:00100000:3.0:1789760729.145565:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 708665896023 time 708709863854 state 0x0 ref 1 00000100:00100000:3.0:1789760729.145578:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febffb9d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512595968:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760729.145629:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23568384, mtime: 0x6aad94d8, atime: 0x0, ctime: 0x6aad94d8, blocks: 0xb807 00010000:00010000:3.0:1789760729.145638:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2de80 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.145654:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=708s v=(0 0) 00010000:00010000:3.0:1789760729.145658:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760729.145664:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2de80 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23568384->23572479] (req 23568384->23572479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.145678:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2de80 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23568384->23572479] (req 23568384->23572479) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760729.192836:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973073 is committed 00002000:00010000:1.0:1789760729.193158:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23568384 -> 23572480 00010000:00010000:1.0:1789760729.193174:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2de80 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23568384->23572479] (req 23568384->23572479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.193187:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2de80 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23568384->23572479] (req 23568384->23572479) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.193196:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2de80 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23568384->23572479] (req 23568384->23572479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.193204:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760729.193209:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2de80 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23568384->23572479] (req 23568384->23572479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.193218:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2de80 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23568384->23572479] (req 23568384->23572479) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.193237:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2de80 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23568384->23572479] (req 23568384->23572479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.193245:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2de80 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23568384->23572479] (req 23568384->23572479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760729.193343:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febffb9d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512595968:12345-192.168.203.56@tcp:4:dd.0 Request processed in 47761us (47997us total) trans 4294973073 rc 0/0 00000100:00100000:1.0:1789760729.193354:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5784 00000100:00100000:1.0:1789760729.193364:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 708665896023 time 708709863854 state 0x2 ref 0 00000100:00100000:0.0:1789760729.199422:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760729.199535:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512596096 00000100:00100000:1.0:1789760729.199546:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512596096 00000100:00100000:1.0:1789760729.199604:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 708665896023 time 708709863854 state 0x0 ref 1 00000100:00100000:1.0:1789760729.199620:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 708710124270 time 708709863854 state 0x1 ref 1 00000100:00100000:1.0:1789760729.199652:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 708710124270 time 708763950978 state 0x0 ref 1 00000100:00100000:1.0:1789760729.199665:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febfbddc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512596096:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760729.199743:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23572480, mtime: 0x6aad94d8, atime: 0x0, ctime: 0x6aad94d8, blocks: 0xb807 00010000:00010000:1.0:1789760729.199754:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2de87 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.199772:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=708s v=(0 0) 00010000:00010000:1.0:1789760729.199776:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760729.199784:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2de87 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23572480->23576575] (req 23572480->23576575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.199799:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2de87 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23572480->23576575] (req 23572480->23576575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760729.233371:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973074 is committed 00002000:00010000:1.0:1789760729.233590:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23572480 -> 23576576 00010000:00010000:1.0:1789760729.233605:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2de87 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23572480->23576575] (req 23572480->23576575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.233624:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2de87 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23572480->23576575] (req 23572480->23576575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.233632:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2de87 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23572480->23576575] (req 23572480->23576575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.233639:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760729.233644:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2de87 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23572480->23576575] (req 23572480->23576575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.233652:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2de87 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23572480->23576575] (req 23572480->23576575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.233671:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2de87 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23572480->23576575] (req 23572480->23576575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.233679:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2de87 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23572480->23576575] (req 23572480->23576575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760729.233839:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febfbddc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512596096:12345-192.168.203.56@tcp:4:dd.0 Request processed in 34171us (34420us total) trans 4294973074 rc 0/0 00000100:00100000:1.0:1789760729.233849:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5785 00000100:00100000:1.0:1789760729.233858:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 708710124270 time 708763950978 state 0x2 ref 0 00000100:00100000:2.0:1789760729.241302:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760729.241373:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512596224 00000100:00100000:1.0:1789760729.241383:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512596224 00000100:00100000:1.0:1789760729.241449:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 708710124270 time 708763950978 state 0x0 ref 1 00000100:00100000:1.0:1789760729.241465:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 708764211394 time 708763950978 state 0x1 ref 1 00000100:00100000:1.0:1789760729.241549:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 708764211394 time 708805847503 state 0x0 ref 1 00000100:00100000:1.0:1789760729.241562:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febffb9f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512596224:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760729.241611:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23576576, mtime: 0x6aad94d8, atime: 0x0, ctime: 0x6aad94d8, blocks: 0xb807 00010000:00010000:1.0:1789760729.241622:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2de8e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.241637:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=708s v=(0 0) 00010000:00010000:1.0:1789760729.241642:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760729.241648:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2de8e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23576576->23580671] (req 23576576->23580671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.241664:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2de8e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23576576->23580671] (req 23576576->23580671) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760729.278015:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973075 is committed 00002000:00010000:1.0:1789760729.278275:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23576576 -> 23580672 00010000:00010000:1.0:1789760729.278293:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2de8e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23576576->23580671] (req 23576576->23580671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.278306:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2de8e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23576576->23580671] (req 23576576->23580671) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.278320:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2de8e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23576576->23580671] (req 23576576->23580671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.278327:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760729.278333:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2de8e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23576576->23580671] (req 23576576->23580671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.278341:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2de8e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23576576->23580671] (req 23576576->23580671) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.278359:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2de8e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23576576->23580671] (req 23576576->23580671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.278365:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2de8e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23576576->23580671] (req 23576576->23580671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760729.278471:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febffb9f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512596224:12345-192.168.203.56@tcp:4:dd.0 Request processed in 36904us (37172us total) trans 4294973075 rc 0/0 00000100:00100000:1.0:1789760729.278480:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5786 00000100:00100000:1.0:1789760729.278489:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 708764211394 time 708805847503 state 0x2 ref 0 00000100:00100000:3.0:1789760729.287303:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760729.287440:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512596352 00000100:00100000:1.0:1789760729.287451:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512596352 00000100:00100000:1.0:1789760729.287504:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 708764211394 time 708805847503 state 0x0 ref 1 00000100:00100000:1.0:1789760729.287519:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 708806107919 time 708805847503 state 0x1 ref 1 00000100:00100000:1.0:1789760729.287583:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 708806107919 time 708851882942 state 0x0 ref 1 00000100:00100000:1.0:1789760729.287606:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac437e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512596352:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760729.287655:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23580672, mtime: 0x6aad94d8, atime: 0x0, ctime: 0x6aad94d8, blocks: 0xb807 00010000:00010000:1.0:1789760729.287664:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2de95 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.287679:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=708s v=(0 0) 00010000:00010000:1.0:1789760729.287684:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760729.287690:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2de95 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23580672->23584767] (req 23580672->23584767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.287750:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2de95 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23580672->23584767] (req 23580672->23584767) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760729.335738:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23580672 -> 23584768 00010000:00010000:1.0:1789760729.335760:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2de95 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23580672->23584767] (req 23580672->23584767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.335775:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2de95 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23580672->23584767] (req 23580672->23584767) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.335784:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2de95 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23580672->23584767] (req 23580672->23584767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.335792:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760729.335798:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2de95 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23580672->23584767] (req 23580672->23584767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.335807:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2de95 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23580672->23584767] (req 23580672->23584767) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.335826:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2de95 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23580672->23584767] (req 23580672->23584767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.335834:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2de95 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23580672->23584767] (req 23580672->23584767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760729.335926:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac437e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2361:x1876699512596352:12345-192.168.203.56@tcp:4:dd.0 Request processed in 48327us (48628us total) trans 4294973076 rc 0/0 00000100:00100000:1.0:1789760729.335937:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5787 00000100:00100000:1.0:1789760729.335946:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 708806107919 time 708851882942 state 0x2 ref 0 00000001:00080000:0.0:1789760729.336018:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973076 is committed 00000100:00100000:2.0:1789760729.339930:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760729.340039:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512596480 00000100:00100000:1.0:1789760729.340049:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512596480 00000100:00100000:1.0:1789760729.340101:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 708806107919 time 708851882942 state 0x0 ref 1 00000100:00100000:1.0:1789760729.340116:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 708852143358 time 708851882942 state 0x1 ref 1 00000100:00100000:1.0:1789760729.340149:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 708852143358 time 708904448299 state 0x0 ref 1 00000100:00100000:1.0:1789760729.340161:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febffb9f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512596480:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760729.340208:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23584768, mtime: 0x6aad94d8, atime: 0x0, ctime: 0x6aad94d8, blocks: 0xb807 00010000:00010000:1.0:1789760729.340217:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2de9c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.340238:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=708s v=(0 0) 00010000:00010000:1.0:1789760729.340242:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760729.340248:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2de9c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23584768->23588863] (req 23584768->23588863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.340262:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2de9c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23584768->23588863] (req 23584768->23588863) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760729.347940:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760729.347982:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760729.348116:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760729.348162:0:5842:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512596608 00000100:00100000:0.0:1789760729.348173:0:5842:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512596608 00000100:00100000:0.0:1789760729.348229:0:5842:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 392 00000100:00100000:0.0:1789760729.348239:0:5842:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febe0500050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:e6cf7267-9b19-462a-a263-14115df15dab+13:2360:x1876699512596608:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:0.0:1789760729.348316:0:5842:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febe0500050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:e6cf7267-9b19-462a-a263-14115df15dab+13:2360:x1876699512596608:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 78us (385us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760729.348324:0:5842:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 392 00000100:00100000:2.0:1789760729.350362:0:5848:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512596864 00000100:00100000:2.0:1789760729.350375:0:5848:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512596864 00000100:00100000:2.0:1789760729.350440:0:5848:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512596736 00000100:00100000:2.0:1789760729.350444:0:5848:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512596736 00000100:00100000:2.0:1789760729.350475:0:5848:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 459 00000100:00100000:2.0:1789760729.350485:0:5848:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac437e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:d2c40150-e97b-40da-952b-cb9f18bf7e96+5:2360:x1876699512596864:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:2.0:1789760729.350953:0:5848:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac437e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:d2c40150-e97b-40da-952b-cb9f18bf7e96+5:2360:x1876699512596864:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 468us (2968us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760729.350963:0:5848:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 459 00000100:00100000:2.0:1789760729.350980:0:5848:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 460 00000100:00100000:2.0:1789760729.350987:0:5848:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac437ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+8:2360:x1876699512596736:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:2.0:1789760729.351017:0:5848:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac437ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+8:2360:x1876699512596736:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 30us (2901us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760729.351023:0:5848:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 460 00000100:00080000:2.0:1789760729.351039:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000001:00080000:0.0:1789760729.375597:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973077 is committed 00002000:00010000:3.0:1789760729.375748:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23584768 -> 23588864 00010000:00010000:3.0:1789760729.375763:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2de9c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23584768->23588863] (req 23584768->23588863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.375777:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2de9c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23584768->23588863] (req 23584768->23588863) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.375787:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2de9c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23584768->23588863] (req 23584768->23588863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.375795:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760729.375800:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2de9c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23584768->23588863] (req 23584768->23588863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.375809:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2de9c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23584768->23588863] (req 23584768->23588863) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.375827:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2de9c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23584768->23588863] (req 23584768->23588863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.375841:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2de9c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23584768->23588863] (req 23584768->23588863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760729.375929:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febffb9f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512596480:12345-192.168.203.56@tcp:4:dd.0 Request processed in 35766us (36005us total) trans 4294973077 rc 0/0 00000100:00100000:3.0:1789760729.375939:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5788 00000100:00100000:3.0:1789760729.375949:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 708852143358 time 708904448299 state 0x2 ref 0 00000100:00100000:3.0:1789760729.380912:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760729.381798:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512596992 00000100:00100000:1.0:1789760729.381811:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512596992 00000100:00100000:1.0:1789760729.381868:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 708852143358 time 708904448299 state 0x0 ref 1 00000100:00100000:1.0:1789760729.381884:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 708904708715 time 708904448299 state 0x1 ref 1 00000100:00100000:1.0:1789760729.381910:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 708904708715 time 708946210218 state 0x0 ref 1 00000100:00100000:1.0:1789760729.381923:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac437df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512596992:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760729.382334:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23588864, mtime: 0x6aad94d8, atime: 0x0, ctime: 0x6aad94d8, blocks: 0xb807 00010000:00010000:1.0:1789760729.382345:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2dea3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.382361:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=708s v=(0 0) 00010000:00010000:1.0:1789760729.382365:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760729.382372:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2dea3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23588864->23592959] (req 23588864->23592959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.382386:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2dea3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23588864->23592959] (req 23588864->23592959) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760729.413205:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973078 is committed 00002000:00010000:1.0:1789760729.413336:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23588864 -> 23592960 00010000:00010000:1.0:1789760729.413354:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2dea3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23588864->23592959] (req 23588864->23592959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.413366:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2dea3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23588864->23592959] (req 23588864->23592959) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.413375:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2dea3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23588864->23592959] (req 23588864->23592959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.413383:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760729.413388:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2dea3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23588864->23592959] (req 23588864->23592959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.413396:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2dea3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23588864->23592959] (req 23588864->23592959) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.413415:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2dea3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23588864->23592959] (req 23588864->23592959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.413423:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2dea3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23588864->23592959] (req 23588864->23592959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760729.413531:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac437df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512596992:12345-192.168.203.56@tcp:4:dd.0 Request processed in 31605us (32625us total) trans 4294973078 rc 0/0 00000100:00100000:1.0:1789760729.413541:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5789 00000100:00100000:1.0:1789760729.413552:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 708904708715 time 708946210218 state 0x2 ref 0 00000100:00100000:0.0:1789760729.418443:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760729.418542:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512597120 00000100:00100000:1.0:1789760729.418553:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512597120 00000100:00100000:1.0:1789760729.418609:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 708904708715 time 708946210218 state 0x0 ref 1 00000100:00100000:1.0:1789760729.418625:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 708946470634 time 708946210218 state 0x1 ref 1 00000100:00100000:1.0:1789760729.418690:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 708946470634 time 708982989456 state 0x0 ref 1 00000100:00100000:1.0:1789760729.418785:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac28f0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512597120:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760729.418965:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23592960, mtime: 0x6aad94d8, atime: 0x0, ctime: 0x6aad94d8, blocks: 0xb807 00010000:00010000:1.0:1789760729.418974:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2deaa lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.418991:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=708s v=(0 0) 00010000:00010000:1.0:1789760729.418995:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760729.419002:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2deaa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23592960->23597055] (req 23592960->23597055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.419015:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2deaa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23592960->23597055] (req 23592960->23597055) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.458919:0:5836:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:3.0:1789760729.461731:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973079 is committed 00002000:00010000:2.0:1789760729.461850:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23592960 -> 23597056 00010000:00010000:2.0:1789760729.461867:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2deaa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23592960->23597055] (req 23592960->23597055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.461881:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2deaa lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23592960->23597055] (req 23592960->23597055) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.461891:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2deaa lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23592960->23597055] (req 23592960->23597055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.461899:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760729.461905:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2deaa lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23592960->23597055] (req 23592960->23597055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.461914:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2deaa lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23592960->23597055] (req 23592960->23597055) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.461935:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2deaa lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23592960->23597055] (req 23592960->23597055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.461943:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2deaa lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23592960->23597055] (req 23592960->23597055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760729.462031:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac28f0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512597120:12345-192.168.203.56@tcp:4:dd.0 Request processed in 43246us (43601us total) trans 4294973079 rc 0/0 00000100:00100000:2.0:1789760729.462040:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5790 00000100:00100000:2.0:1789760729.462050:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 708946470634 time 708982989456 state 0x2 ref 0 00000100:00100000:3.0:1789760729.465004:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760729.465175:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512597248 00000100:00100000:1.0:1789760729.465185:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512597248 00000100:00100000:1.0:1789760729.465238:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 708946470634 time 708982989456 state 0x0 ref 1 00000100:00100000:1.0:1789760729.465253:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 708983249872 time 708982989456 state 0x1 ref 1 00000100:00100000:1.0:1789760729.465283:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 708983249872 time 709029582436 state 0x0 ref 1 00000100:00100000:1.0:1789760729.465295:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac437fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512597248:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760729.465340:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23597056, mtime: 0x6aad94d8, atime: 0x0, ctime: 0x6aad94d8, blocks: 0xb807 00010000:00010000:1.0:1789760729.465348:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2deb1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.465363:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=709s v=(0 0) 00010000:00010000:1.0:1789760729.465368:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760729.465374:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2deb1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23597056->23601151] (req 23597056->23601151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.465386:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2deb1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23597056->23601151] (req 23597056->23601151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760729.503161:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973080 is committed 00002000:00010000:0.0:1789760729.503933:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23597056 -> 23601152 00010000:00010000:0.0:1789760729.503949:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2deb1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23597056->23601151] (req 23597056->23601151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.503962:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2deb1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23597056->23601151] (req 23597056->23601151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.503971:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2deb1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23597056->23601151] (req 23597056->23601151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.503978:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760729.504052:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2deb1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23597056->23601151] (req 23597056->23601151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.504128:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2deb1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23597056->23601151] (req 23597056->23601151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.504156:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2deb1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23597056->23601151] (req 23597056->23601151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.504164:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2deb1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23597056->23601151] (req 23597056->23601151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760729.504319:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac437fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512597248:12345-192.168.203.56@tcp:4:dd.0 Request processed in 39020us (39319us total) trans 4294973080 rc 0/0 00000100:00100000:0.0:1789760729.504330:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5791 00000100:00100000:0.0:1789760729.504341:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 708983249872 time 709029582436 state 0x2 ref 0 00000100:00100000:3.0:1789760729.509892:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760729.510071:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512597376 00000100:00100000:1.0:1789760729.510081:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512597376 00000100:00100000:1.0:1789760729.510206:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 708983249872 time 709029582436 state 0x0 ref 1 00000100:00100000:1.0:1789760729.510222:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 709029842852 time 709029582436 state 0x1 ref 1 00000100:00100000:1.0:1789760729.510246:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 709029842852 time 709074545912 state 0x0 ref 1 00000100:00100000:1.0:1789760729.510256:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac437f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512597376:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760729.510306:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23601152, mtime: 0x6aad94d8, atime: 0x0, ctime: 0x6aad94d8, blocks: 0xb807 00010000:00010000:1.0:1789760729.510316:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2deb8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.510333:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=709s v=(0 0) 00010000:00010000:1.0:1789760729.510338:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760729.510344:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2deb8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23601152->23605247] (req 23601152->23605247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.510359:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2deb8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23601152->23605247] (req 23601152->23605247) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760729.545665:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973081 is committed 00002000:00010000:3.0:1789760729.545812:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23601152 -> 23605248 00002000:00010000:3.0:1789760729.545817:0:6710:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760728 -> 1789760729 00002000:00010000:3.0:1789760729.545820:0:6710:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760728 -> 1789760729 00010000:00010000:3.0:1789760729.545835:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2deb8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23601152->23605247] (req 23601152->23605247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.545849:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2deb8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23601152->23605247] (req 23601152->23605247) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.545859:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2deb8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23601152->23605247] (req 23601152->23605247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.545873:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760729.545880:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2deb8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23601152->23605247] (req 23601152->23605247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.545891:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2deb8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23601152->23605247] (req 23601152->23605247) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.545912:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2deb8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23601152->23605247] (req 23601152->23605247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.545921:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2deb8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23601152->23605247] (req 23601152->23605247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760729.546111:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac437f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512597376:12345-192.168.203.56@tcp:4:dd.0 Request processed in 35849us (36225us total) trans 4294973081 rc 0/0 00000100:00100000:3.0:1789760729.546123:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5792 00000100:00100000:3.0:1789760729.546133:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 709029842852 time 709074545912 state 0x2 ref 0 00000100:00100000:3.0:1789760729.550526:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760729.550653:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512597504 00000100:00100000:2.0:1789760729.550664:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512597504 00000100:00100000:2.0:1789760729.550932:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 709029842852 time 709074545912 state 0x0 ref 1 00000100:00100000:2.0:1789760729.550948:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 709074806328 time 709074545912 state 0x1 ref 1 00000100:00100000:2.0:1789760729.551044:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 709074806328 time 709115342264 state 0x0 ref 1 00000100:00100000:2.0:1789760729.551058:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac437d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512597504:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760729.551120:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23605248, mtime: 0x6aad94d9, atime: 0x0, ctime: 0x6aad94d9, blocks: 0xb807 00010000:00010000:2.0:1789760729.551130:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2debf lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.551146:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=709s v=(0 0) 00010000:00010000:2.0:1789760729.551150:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760729.551156:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2debf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23605248->23609343] (req 23605248->23609343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.551170:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2debf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23605248->23609343] (req 23605248->23609343) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760729.586538:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23605248 -> 23609344 00010000:00010000:2.0:1789760729.586559:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2debf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23605248->23609343] (req 23605248->23609343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.586574:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2debf lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23605248->23609343] (req 23605248->23609343) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.586591:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2debf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23605248->23609343] (req 23605248->23609343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.586599:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760729.586604:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2debf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23605248->23609343] (req 23605248->23609343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.586614:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2debf lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23605248->23609343] (req 23605248->23609343) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.586633:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2debf lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23605248->23609343] (req 23605248->23609343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.586641:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2debf lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23605248->23609343] (req 23605248->23609343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760729.586771:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac437d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2359:x1876699512597504:12345-192.168.203.56@tcp:4:dd.0 Request processed in 35710us (36249us total) trans 4294973082 rc 0/0 00000100:00100000:2.0:1789760729.586782:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5793 00000100:00100000:2.0:1789760729.586793:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 709074806328 time 709115342264 state 0x2 ref 0 00000001:00080000:1.0:1789760729.589243:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973082 is committed 00000100:00100000:2.0:1789760729.590978:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760729.591467:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512597632 00000100:00100000:0.0:1789760729.591477:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512597632 00000100:00100000:0.0:1789760729.591534:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 709074806328 time 709115342264 state 0x0 ref 1 00000100:00100000:0.0:1789760729.591550:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 709115602680 time 709115342264 state 0x1 ref 1 00000100:00100000:0.0:1789760729.591589:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 709115602680 time 709155887750 state 0x0 ref 1 00000100:00100000:0.0:1789760729.591602:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febffb9f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512597632:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760729.591657:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23609344, mtime: 0x6aad94d9, atime: 0x0, ctime: 0x6aad94d9, blocks: 0xb807 00010000:00010000:0.0:1789760729.591666:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2dec6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.591691:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=709s v=(0 0) 00010000:00010000:0.0:1789760729.591728:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760729.591736:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2dec6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23609344->23613439] (req 23609344->23613439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.591751:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2dec6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23609344->23613439] (req 23609344->23613439) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760729.625386:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23609344 -> 23613440 00010000:00010000:1.0:1789760729.625409:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2dec6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23609344->23613439] (req 23609344->23613439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.625436:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2dec6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23609344->23613439] (req 23609344->23613439) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.625446:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2dec6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23609344->23613439] (req 23609344->23613439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.625455:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760729.625460:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2dec6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23609344->23613439] (req 23609344->23613439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.625469:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2dec6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23609344->23613439] (req 23609344->23613439) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.625489:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2dec6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23609344->23613439] (req 23609344->23613439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.625498:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2dec6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23609344->23613439] (req 23609344->23613439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760729.625602:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febffb9f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2359:x1876699512597632:12345-192.168.203.56@tcp:4:dd.0 Request processed in 33997us (34627us total) trans 4294973083 rc 0/0 00000100:00100000:1.0:1789760729.625613:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5794 00000100:00100000:1.0:1789760729.625624:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 709115602680 time 709155887750 state 0x2 ref 0 00000001:00080000:2.0:1789760729.627413:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973083 is committed 00000100:00100000:3.0:1789760729.629375:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760729.629541:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512597760 00000100:00100000:1.0:1789760729.629552:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512597760 00000100:00100000:1.0:1789760729.629608:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 709115602680 time 709155887750 state 0x0 ref 1 00000100:00100000:1.0:1789760729.629624:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 709156148166 time 709155887750 state 0x1 ref 1 00000100:00100000:1.0:1789760729.629656:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 709156148166 time 709193954852 state 0x0 ref 1 00000100:00100000:1.0:1789760729.629672:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4074700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512597760:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760729.629758:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23613440, mtime: 0x6aad94d9, atime: 0x0, ctime: 0x6aad94d9, blocks: 0xb807 00010000:00010000:1.0:1789760729.629767:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2decd lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.629784:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=709s v=(0 0) 00010000:00010000:1.0:1789760729.629789:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760729.629796:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2decd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23613440->23617535] (req 23613440->23617535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.629811:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2decd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23613440->23617535] (req 23613440->23617535) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760729.658161:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973084 is committed 00002000:00010000:1.0:1789760729.658305:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23613440 -> 23617536 00010000:00010000:1.0:1789760729.658320:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2decd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23613440->23617535] (req 23613440->23617535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.658334:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2decd lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23613440->23617535] (req 23613440->23617535) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.658343:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2decd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23613440->23617535] (req 23613440->23617535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.658350:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760729.658355:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2decd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23613440->23617535] (req 23613440->23617535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.658365:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2decd lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23613440->23617535] (req 23613440->23617535) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.658383:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2decd lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23613440->23617535] (req 23613440->23617535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.658390:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2decd lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23613440->23617535] (req 23613440->23617535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760729.658511:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4074700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512597760:12345-192.168.203.56@tcp:4:dd.0 Request processed in 28837us (29141us total) trans 4294973084 rc 0/0 00000100:00100000:1.0:1789760729.658522:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5795 00000100:00100000:1.0:1789760729.658531:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 709156148166 time 709193954852 state 0x2 ref 0 00000100:00100000:3.0:1789760729.661370:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760729.662103:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512597888 00000100:00100000:2.0:1789760729.662115:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512597888 00000100:00100000:2.0:1789760729.662169:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 709156148166 time 709193954852 state 0x0 ref 1 00000100:00100000:2.0:1789760729.662185:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 709194215268 time 709193954852 state 0x1 ref 1 00000100:00100000:2.0:1789760729.662215:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 709194215268 time 709226514809 state 0x0 ref 1 00000100:00100000:2.0:1789760729.662229:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4074a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512597888:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760729.662277:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23617536, mtime: 0x6aad94d9, atime: 0x0, ctime: 0x6aad94d9, blocks: 0xb807 00010000:00010000:2.0:1789760729.662287:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2ded4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.662304:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=709s v=(0 0) 00010000:00010000:2.0:1789760729.662309:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760729.662316:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2ded4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23617536->23621631] (req 23617536->23621631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.662330:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2ded4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23617536->23621631] (req 23617536->23621631) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760729.701722:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973085 is committed 00002000:00010000:3.0:1789760729.701857:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23617536 -> 23621632 00010000:00010000:3.0:1789760729.701873:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2ded4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23617536->23621631] (req 23617536->23621631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.701892:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2ded4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23617536->23621631] (req 23617536->23621631) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.701901:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2ded4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23617536->23621631] (req 23617536->23621631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.701909:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760729.701914:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2ded4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23617536->23621631] (req 23617536->23621631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.701924:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2ded4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23617536->23621631] (req 23617536->23621631) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.701942:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2ded4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23617536->23621631] (req 23617536->23621631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.701950:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2ded4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23617536->23621631] (req 23617536->23621631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760729.702241:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4074a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512597888:12345-192.168.203.56@tcp:4:dd.0 Request processed in 40009us (40887us total) trans 4294973085 rc 0/0 00000100:00100000:3.0:1789760729.702251:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5796 00000100:00100000:3.0:1789760729.702260:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 709194215268 time 709226514809 state 0x2 ref 0 00000100:00100000:2.0:1789760729.708141:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760729.708351:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512598016 00000100:00100000:1.0:1789760729.708363:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512598016 00000100:00100000:1.0:1789760729.708420:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 709194215268 time 709226514809 state 0x0 ref 1 00000100:00100000:1.0:1789760729.708446:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 709226775225 time 709226514809 state 0x1 ref 1 00000100:00100000:1.0:1789760729.708477:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 709226775225 time 709272776379 state 0x0 ref 1 00000100:00100000:1.0:1789760729.708490:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febffb9ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512598016:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760729.708539:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23621632, mtime: 0x6aad94d9, atime: 0x0, ctime: 0x6aad94d9, blocks: 0xb807 00010000:00010000:1.0:1789760729.708549:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2dedb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.708566:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=709s v=(0 0) 00010000:00010000:1.0:1789760729.708571:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760729.708578:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2dedb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23621632->23625727] (req 23621632->23625727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.708593:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2dedb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23621632->23625727] (req 23621632->23625727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760729.759794:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973086 is committed 00002000:00010000:2.0:1789760729.760693:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23621632 -> 23625728 00010000:00010000:2.0:1789760729.760791:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2dedb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23621632->23625727] (req 23621632->23625727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.760805:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2dedb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23621632->23625727] (req 23621632->23625727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.762174:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2dedb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23621632->23625727] (req 23621632->23625727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.762182:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760729.762188:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2dedb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23621632->23625727] (req 23621632->23625727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.762197:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2dedb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23621632->23625727] (req 23621632->23625727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.762217:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2dedb lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23621632->23625727] (req 23621632->23625727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.762225:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2dedb lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23621632->23625727] (req 23621632->23625727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760729.762342:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febffb9ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512598016:12345-192.168.203.56@tcp:4:dd.0 Request processed in 53850us (54208us total) trans 4294973086 rc 0/0 00000100:00100000:2.0:1789760729.762352:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5797 00000100:00100000:2.0:1789760729.762361:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 709226775225 time 709272776379 state 0x2 ref 0 00000100:00100000:2.0:1789760729.767488:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760729.767590:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512598144 00000100:00100000:1.0:1789760729.767601:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512598144 00000100:00100000:1.0:1789760729.767655:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 709226775225 time 709272776379 state 0x0 ref 1 00000100:00100000:1.0:1789760729.767671:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 709273036795 time 709272776379 state 0x1 ref 1 00000100:00100000:1.0:1789760729.767732:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 709273036795 time 709332031253 state 0x0 ref 1 00000100:00100000:1.0:1789760729.767745:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febffb9c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512598144:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760729.767796:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23625728, mtime: 0x6aad94d9, atime: 0x0, ctime: 0x6aad94d9, blocks: 0xb807 00010000:00010000:1.0:1789760729.767805:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2dee2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.767956:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=709s v=(0 0) 00010000:00010000:1.0:1789760729.767961:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760729.767968:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2dee2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23625728->23629823] (req 23625728->23629823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.767988:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2dee2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23625728->23629823] (req 23625728->23629823) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760729.833357:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973087 is committed 00002000:00010000:2.0:1789760729.834784:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23625728 -> 23629824 00010000:00010000:2.0:1789760729.834802:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2dee2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23625728->23629823] (req 23625728->23629823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.834819:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2dee2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23625728->23629823] (req 23625728->23629823) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.834827:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2dee2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23625728->23629823] (req 23625728->23629823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.834834:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760729.834838:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2dee2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23625728->23629823] (req 23625728->23629823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.834847:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2dee2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23625728->23629823] (req 23625728->23629823) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.834865:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2dee2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23625728->23629823] (req 23625728->23629823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.834872:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2dee2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23625728->23629823] (req 23625728->23629823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760729.834979:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febffb9c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512598144:12345-192.168.203.56@tcp:4:dd.0 Request processed in 67231us (67496us total) trans 4294973087 rc 0/0 00000100:00100000:2.0:1789760729.835002:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5798 00000100:00100000:2.0:1789760729.835011:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 709273036795 time 709332031253 state 0x2 ref 0 00000100:00100000:3.0:1789760729.837877:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760729.838005:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512598272 00000100:00100000:2.0:1789760729.838016:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512598272 00000100:00100000:2.0:1789760729.838074:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 709273036795 time 709332031253 state 0x0 ref 1 00000100:00100000:2.0:1789760729.838091:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 709332291669 time 709332031253 state 0x1 ref 1 00000100:00100000:2.0:1789760729.838121:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 709332291669 time 709402421180 state 0x0 ref 1 00000100:00100000:2.0:1789760729.838135:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febea651500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512598272:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760729.838186:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23629824, mtime: 0x6aad94d9, atime: 0x0, ctime: 0x6aad94d9, blocks: 0xb807 00010000:00010000:2.0:1789760729.838195:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2dee9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.838210:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=709s v=(0 0) 00010000:00010000:2.0:1789760729.838214:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760729.838221:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2dee9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23629824->23633919] (req 23629824->23633919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.838235:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2dee9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23629824->23633919] (req 23629824->23633919) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760729.874614:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973088 is committed 00002000:00010000:2.0:1789760729.874633:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23629824 -> 23633920 00010000:00010000:2.0:1789760729.874801:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2dee9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23629824->23633919] (req 23629824->23633919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.874820:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2dee9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23629824->23633919] (req 23629824->23633919) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.874835:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2dee9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23629824->23633919] (req 23629824->23633919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.874842:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760729.874847:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2dee9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23629824->23633919] (req 23629824->23633919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.874857:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2dee9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23629824->23633919] (req 23629824->23633919) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.874877:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2dee9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23629824->23633919] (req 23629824->23633919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.874885:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2dee9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23629824->23633919] (req 23629824->23633919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760729.875081:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febea651500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512598272:12345-192.168.203.56@tcp:4:dd.0 Request processed in 36942us (37210us total) trans 4294973088 rc 0/0 00000100:00100000:3.0:1789760729.875093:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5799 00000100:00100000:3.0:1789760729.875103:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 709332291669 time 709402421180 state 0x2 ref 0 00000100:00100000:3.0:1789760729.877679:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760729.878350:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512598400 00000100:00100000:2.0:1789760729.878361:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512598400 00000100:00100000:2.0:1789760729.878415:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 709332291669 time 709402421180 state 0x0 ref 1 00000100:00100000:2.0:1789760729.878443:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 709402681596 time 709402421180 state 0x1 ref 1 00000100:00100000:2.0:1789760729.878757:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 709402681596 time 709443054868 state 0x0 ref 1 00000100:00100000:2.0:1789760729.879200:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febea650380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512598400:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760729.879263:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23633920, mtime: 0x6aad94d9, atime: 0x0, ctime: 0x6aad94d9, blocks: 0xb807 00010000:00010000:2.0:1789760729.879271:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2def0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.879289:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=709s v=(0 0) 00010000:00010000:2.0:1789760729.879293:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760729.879300:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2def0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23633920->23638015] (req 23633920->23638015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.879316:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2def0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23633920->23638015] (req 23633920->23638015) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760729.911648:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973089 is committed 00002000:00010000:2.0:1789760729.911845:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23633920 -> 23638016 00010000:00010000:2.0:1789760729.911862:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2def0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23633920->23638015] (req 23633920->23638015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.911875:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2def0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23633920->23638015] (req 23633920->23638015) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.911885:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2def0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23633920->23638015] (req 23633920->23638015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.911892:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760729.911897:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2def0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23633920->23638015] (req 23633920->23638015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.911906:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2def0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23633920->23638015] (req 23633920->23638015) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.911924:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2def0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23633920->23638015] (req 23633920->23638015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.911933:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2def0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23633920->23638015] (req 23633920->23638015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760729.912172:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febea650380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512598400:12345-192.168.203.56@tcp:4:dd.0 Request processed in 33399us (34498us total) trans 4294973089 rc 0/0 00000100:00100000:2.0:1789760729.912183:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5800 00000100:00100000:2.0:1789760729.912192:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 709402681596 time 709443054868 state 0x2 ref 0 00000100:00100000:2.0:1789760729.914569:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760729.914680:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512598528 00000100:00100000:1.0:1789760729.914690:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512598528 00000100:00100000:1.0:1789760729.914844:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 709402681596 time 709443054868 state 0x0 ref 1 00000100:00100000:1.0:1789760729.914861:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 709443315284 time 709443054868 state 0x1 ref 1 00000100:00100000:1.0:1789760729.914927:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 709443315284 time 709479224933 state 0x0 ref 1 00000100:00100000:1.0:1789760729.914940:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febffb9c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512598528:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760729.914987:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23638016, mtime: 0x6aad94d9, atime: 0x0, ctime: 0x6aad94d9, blocks: 0xb807 00010000:00010000:1.0:1789760729.915009:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2def7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.915026:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=709s v=(0 0) 00010000:00010000:1.0:1789760729.915031:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760729.915037:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2def7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23638016->23642111] (req 23638016->23642111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.915051:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2def7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23638016->23642111] (req 23638016->23642111) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760729.945541:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973090 is committed 00002000:00010000:3.0:1789760729.945626:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23638016 -> 23642112 00010000:00010000:3.0:1789760729.945643:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2def7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23638016->23642111] (req 23638016->23642111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.945656:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2def7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23638016->23642111] (req 23638016->23642111) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.945666:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2def7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23638016->23642111] (req 23638016->23642111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.945673:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760729.945679:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2def7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23638016->23642111] (req 23638016->23642111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.945689:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2def7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23638016->23642111] (req 23638016->23642111) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.945788:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2def7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23638016->23642111] (req 23638016->23642111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.945802:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2def7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23638016->23642111] (req 23638016->23642111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760729.945933:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febffb9c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512598528:12345-192.168.203.56@tcp:4:dd.0 Request processed in 30991us (31368us total) trans 4294973090 rc 0/0 00000100:00100000:3.0:1789760729.945943:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5801 00000100:00100000:3.0:1789760729.945953:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 709443315284 time 709479224933 state 0x2 ref 0 00000100:00100000:2.0:1789760729.950080:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760729.950223:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512598656 00000100:00100000:3.0:1789760729.950233:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512598656 00000100:00100000:3.0:1789760729.950286:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 709443315284 time 709479224933 state 0x0 ref 1 00000100:00100000:3.0:1789760729.950301:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 709479485349 time 709479224933 state 0x1 ref 1 00000100:00100000:3.0:1789760729.950370:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 709479485349 time 709514669169 state 0x0 ref 1 00000100:00100000:3.0:1789760729.950383:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac38b6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512598656:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760729.950437:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23642112, mtime: 0x6aad94d9, atime: 0x0, ctime: 0x6aad94d9, blocks: 0xb807 00010000:00010000:3.0:1789760729.950454:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2defe lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.950471:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=709s v=(0 0) 00010000:00010000:3.0:1789760729.950475:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760729.950481:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2defe lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23642112->23646207] (req 23642112->23646207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.950500:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2defe lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23642112->23646207] (req 23642112->23646207) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760729.982798:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23642112 -> 23646208 00010000:00010000:3.0:1789760729.982893:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2defe lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23642112->23646207] (req 23642112->23646207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.982907:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2defe lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23642112->23646207] (req 23642112->23646207) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.982916:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2defe lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23642112->23646207] (req 23642112->23646207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.982925:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760729.982931:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2defe lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23642112->23646207] (req 23642112->23646207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.982940:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2defe lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23642112->23646207] (req 23642112->23646207) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.982959:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2defe lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23642112->23646207] (req 23642112->23646207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.982968:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2defe lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23642112->23646207] (req 23642112->23646207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760729.983070:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac38b6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2360:x1876699512598656:12345-192.168.203.56@tcp:4:dd.0 Request processed in 32684us (32993us total) trans 4294973091 rc 0/0 00000100:00100000:3.0:1789760729.983080:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5802 00000100:00100000:3.0:1789760729.983101:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 709479485349 time 709514669169 state 0x2 ref 0 00000001:00080000:1.0:1789760729.983238:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973091 is committed 00000100:00100000:3.0:1789760729.986013:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760729.986144:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512598784 00000100:00100000:2.0:1789760729.986154:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512598784 00000100:00100000:2.0:1789760729.986208:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 709479485349 time 709514669169 state 0x0 ref 1 00000100:00100000:2.0:1789760729.986224:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 709514929585 time 709514669169 state 0x1 ref 1 00000100:00100000:2.0:1789760729.986254:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 709514929585 time 709550553974 state 0x0 ref 1 00000100:00100000:2.0:1789760729.986267:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac3352300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512598784:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760729.986313:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23646208, mtime: 0x6aad94d9, atime: 0x0, ctime: 0x6aad94d9, blocks: 0xb807 00010000:00010000:2.0:1789760729.986322:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2df05 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.986343:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=709s v=(0 0) 00010000:00010000:2.0:1789760729.986347:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760729.986353:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2df05 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23646208->23650303] (req 23646208->23650303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.986368:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2df05 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23646208->23650303] (req 23646208->23650303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760730.019556:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23646208 -> 23650304 00010000:00010000:2.0:1789760730.019578:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2df05 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23646208->23650303] (req 23646208->23650303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.019592:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2df05 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23646208->23650303] (req 23646208->23650303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.019601:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2df05 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23646208->23650303] (req 23646208->23650303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.019610:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760730.019615:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2df05 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23646208->23650303] (req 23646208->23650303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.019624:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2df05 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23646208->23650303] (req 23646208->23650303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.019642:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2df05 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23646208->23650303] (req 23646208->23650303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.019650:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2df05 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23646208->23650303] (req 23646208->23650303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760730.019770:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac3352300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2359:x1876699512598784:12345-192.168.203.56@tcp:4:dd.0 Request processed in 33500us (33760us total) trans 4294973092 rc 0/0 00000100:00100000:2.0:1789760730.019780:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5803 00000100:00100000:2.0:1789760730.019790:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 709514929585 time 709550553974 state 0x2 ref 0 00000001:00080000:1.0:1789760730.022128:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973092 is committed 00000100:00100000:2.0:1789760730.022757:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760730.022822:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512598912 00000100:00100000:3.0:1789760730.022830:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512598912 00000100:00100000:3.0:1789760730.022876:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 709514929585 time 709550553974 state 0x0 ref 1 00000100:00100000:3.0:1789760730.022889:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 709550814390 time 709550553974 state 0x1 ref 1 00000100:00100000:3.0:1789760730.022910:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 709550814390 time 709587210200 state 0x0 ref 1 00000100:00100000:3.0:1789760730.022920:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2fb9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512598912:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760730.022960:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23650304, mtime: 0x6aad94d9, atime: 0x0, ctime: 0x6aad94d9, blocks: 0xb807 00010000:00010000:3.0:1789760730.022968:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2df0c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.022982:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=709s v=(0 0) 00010000:00010000:3.0:1789760730.022986:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760730.022992:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2df0c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23650304->23654399] (req 23650304->23654399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.023004:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2df0c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23650304->23654399] (req 23650304->23654399) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760730.056345:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973093 is committed 00002000:00010000:3.0:1789760730.056503:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23650304 -> 23654400 00010000:00010000:3.0:1789760730.056518:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2df0c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23650304->23654399] (req 23650304->23654399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.056530:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2df0c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23650304->23654399] (req 23650304->23654399) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.056540:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2df0c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23650304->23654399] (req 23650304->23654399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.056548:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760730.056553:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2df0c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23650304->23654399] (req 23650304->23654399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.056561:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2df0c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23650304->23654399] (req 23650304->23654399) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.056581:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2df0c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23650304->23654399] (req 23650304->23654399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.056589:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2df0c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23650304->23654399] (req 23650304->23654399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760730.056681:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2fb9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512598912:12345-192.168.203.56@tcp:4:dd.0 Request processed in 33757us (33923us total) trans 4294973093 rc 0/0 00000100:00100000:3.0:1789760730.056802:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5804 00000100:00100000:3.0:1789760730.056821:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 709550814390 time 709587210200 state 0x2 ref 0 00000100:00100000:3.0:1789760730.059122:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760730.059238:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512599040 00000100:00100000:0.0:1789760730.059247:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512599040 00000100:00100000:0.0:1789760730.059293:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 709550814390 time 709587210200 state 0x0 ref 1 00000100:00100000:0.0:1789760730.059307:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 709587470616 time 709587210200 state 0x1 ref 1 00000100:00100000:0.0:1789760730.059333:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 709587470616 time 709623633413 state 0x0 ref 1 00000100:00100000:0.0:1789760730.059344:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac3353800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512599040:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760730.059384:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23654400, mtime: 0x6aad94d9, atime: 0x0, ctime: 0x6aad94d9, blocks: 0xb807 00010000:00010000:0.0:1789760730.059392:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2df13 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.059411:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=709s v=(0 0) 00010000:00010000:0.0:1789760730.059415:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760730.059431:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2df13 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23654400->23658495] (req 23654400->23658495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.059445:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2df13 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23654400->23658495] (req 23654400->23658495) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760730.091619:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973094 is committed 00002000:00010000:0.0:1789760730.092306:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23654400 -> 23658496 00010000:00010000:0.0:1789760730.092322:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2df13 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23654400->23658495] (req 23654400->23658495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.092334:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2df13 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23654400->23658495] (req 23654400->23658495) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.092343:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2df13 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23654400->23658495] (req 23654400->23658495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.092350:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760730.092354:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2df13 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23654400->23658495] (req 23654400->23658495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.092364:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2df13 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23654400->23658495] (req 23654400->23658495) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.092383:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2df13 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23654400->23658495] (req 23654400->23658495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.092391:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2df13 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23654400->23658495] (req 23654400->23658495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760730.092525:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac3353800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512599040:12345-192.168.203.56@tcp:4:dd.0 Request processed in 33178us (33406us total) trans 4294973094 rc 0/0 00000100:00100000:0.0:1789760730.092536:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5805 00000100:00100000:0.0:1789760730.092545:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 709587470616 time 709623633413 state 0x2 ref 0 00000100:00100000:0.0:1789760730.097068:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760730.097235:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512599168 00000100:00100000:2.0:1789760730.097245:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512599168 00000100:00100000:2.0:1789760730.097301:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 709587470616 time 709623633413 state 0x0 ref 1 00000100:00100000:2.0:1789760730.097315:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 709623893829 time 709623633413 state 0x1 ref 1 00000100:00100000:2.0:1789760730.097387:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 709623893829 time 709661685071 state 0x0 ref 1 00000100:00100000:2.0:1789760730.097400:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac28f2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512599168:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760730.097456:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23658496, mtime: 0x6aad94d9, atime: 0x0, ctime: 0x6aad94d9, blocks: 0xb807 00010000:00010000:2.0:1789760730.097466:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2df1a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.097481:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=709s v=(0 0) 00010000:00010000:2.0:1789760730.097485:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760730.097491:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2df1a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23658496->23662591] (req 23658496->23662591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.097505:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2df1a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23658496->23662591] (req 23658496->23662591) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760730.132106:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23658496 -> 23662592 00010000:00010000:3.0:1789760730.132126:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2df1a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23658496->23662591] (req 23658496->23662591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.132139:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2df1a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23658496->23662591] (req 23658496->23662591) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.132148:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2df1a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23658496->23662591] (req 23658496->23662591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.132156:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760730.132162:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2df1a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23658496->23662591] (req 23658496->23662591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.132171:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2df1a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23658496->23662591] (req 23658496->23662591) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.132189:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2df1a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23658496->23662591] (req 23658496->23662591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.132197:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2df1a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23658496->23662591] (req 23658496->23662591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760730.132284:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac28f2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2361:x1876699512599168:12345-192.168.203.56@tcp:4:dd.0 Request processed in 34882us (35223us total) trans 4294973095 rc 0/0 00000100:00100000:3.0:1789760730.132294:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5806 00000100:00100000:3.0:1789760730.132303:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 709623893829 time 709661685071 state 0x2 ref 0 00000100:00100000:0.0:1789760730.134849:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760730.134966:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512599296 00000100:00100000:3.0:1789760730.134976:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512599296 00000100:00100000:3.0:1789760730.135027:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 709623893829 time 709661685071 state 0x0 ref 1 00000100:00100000:3.0:1789760730.135041:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 709661945487 time 709661685071 state 0x1 ref 1 00000100:00100000:3.0:1789760730.135072:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 709661945487 time 709699371856 state 0x0 ref 1 00000100:00100000:3.0:1789760730.135084:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac28f1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2359:x1876699512599296:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760730.135141:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23662592, mtime: 0x6aad94d9, atime: 0x0, ctime: 0x6aad94d9, blocks: 0xb807 00010000:00010000:3.0:1789760730.135150:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2df21 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.135166:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=709s v=(0 0) 00010000:00010000:3.0:1789760730.135171:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760730.135177:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2df21 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23662592->23666687] (req 23662592->23666687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.135191:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2df21 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23662592->23666687] (req 23662592->23666687) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760730.135731:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973095 is committed 00002000:00010000:3.0:1789760730.162458:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23662592 -> 23666688 00010000:00010000:3.0:1789760730.162478:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2df21 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23662592->23666687] (req 23662592->23666687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.162492:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2df21 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23662592->23666687] (req 23662592->23666687) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.162501:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2df21 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23662592->23666687] (req 23662592->23666687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.162508:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760730.162513:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2df21 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23662592->23666687] (req 23662592->23666687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.162521:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2df21 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23662592->23666687] (req 23662592->23666687) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.162539:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2df21 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23662592->23666687] (req 23662592->23666687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.162546:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2df21 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23662592->23666687] (req 23662592->23666687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760730.162637:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac28f1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2359:x1876699512599296:12345-192.168.203.56@tcp:4:dd.0 Request processed in 27550us (27792us total) trans 4294973096 rc 0/0 00000100:00100000:3.0:1789760730.162652:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5807 00000100:00100000:3.0:1789760730.162662:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 709661945487 time 709699371856 state 0x2 ref 0 00000001:00080000:2.0:1789760730.162799:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973096 is committed 00000100:00100000:0.0:1789760730.166542:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760730.166642:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512599424 00000100:00100000:2.0:1789760730.166652:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512599424 00000100:00100000:2.0:1789760730.166752:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 709661945487 time 709699371856 state 0x0 ref 1 00000100:00100000:2.0:1789760730.166767:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 709699632272 time 709699371856 state 0x1 ref 1 00000100:00100000:2.0:1789760730.166789:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 709699632272 time 709731088760 state 0x0 ref 1 00000100:00100000:2.0:1789760730.166801:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac28f0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512599424:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760730.167007:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23666688, mtime: 0x6aad94d9, atime: 0x0, ctime: 0x6aad94d9, blocks: 0xb807 00010000:00010000:2.0:1789760730.167018:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2df28 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.167035:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=709s v=(0 0) 00010000:00010000:2.0:1789760730.167040:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760730.167046:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2df28 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23666688->23670783] (req 23666688->23670783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.167061:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2df28 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23666688->23670783] (req 23666688->23670783) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760730.196044:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973097 is committed 00002000:00010000:0.0:1789760730.196209:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23666688 -> 23670784 00010000:00010000:0.0:1789760730.196225:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2df28 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23666688->23670783] (req 23666688->23670783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.196238:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2df28 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23666688->23670783] (req 23666688->23670783) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.196247:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2df28 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23666688->23670783] (req 23666688->23670783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.196254:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760730.196259:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2df28 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23666688->23670783] (req 23666688->23670783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.196274:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2df28 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23666688->23670783] (req 23666688->23670783) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.196295:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2df28 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23666688->23670783] (req 23666688->23670783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.196302:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2df28 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23666688->23670783] (req 23666688->23670783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760730.196441:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac28f0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512599424:12345-192.168.203.56@tcp:4:dd.0 Request processed in 29637us (29903us total) trans 4294973097 rc 0/0 00000100:00100000:0.0:1789760730.196452:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5808 00000100:00100000:0.0:1789760730.196463:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 709699632272 time 709731088760 state 0x2 ref 0 00000100:00100000:0.0:1789760730.199094:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760730.199230:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512599552 00000100:00100000:3.0:1789760730.199240:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512599552 00000100:00100000:3.0:1789760730.199292:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 709699632272 time 709731088760 state 0x0 ref 1 00000100:00100000:3.0:1789760730.199308:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 709731349176 time 709731088760 state 0x1 ref 1 00000100:00100000:3.0:1789760730.199367:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 709731349176 time 709763666389 state 0x0 ref 1 00000100:00100000:3.0:1789760730.199380:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febfbeae300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512599552:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760730.199434:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23670784, mtime: 0x6aad94d9, atime: 0x0, ctime: 0x6aad94d9, blocks: 0xb807 00010000:00010000:3.0:1789760730.199443:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2df2f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.199459:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=709s v=(0 0) 00010000:00010000:3.0:1789760730.199464:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760730.199470:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2df2f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23670784->23674879] (req 23670784->23674879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.199484:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2df2f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23670784->23674879] (req 23670784->23674879) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760730.230329:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973098 is committed 00002000:00010000:2.0:1789760730.230586:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23670784 -> 23674880 00010000:00010000:2.0:1789760730.230603:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2df2f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23670784->23674879] (req 23670784->23674879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.230616:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2df2f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23670784->23674879] (req 23670784->23674879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.230625:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2df2f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23670784->23674879] (req 23670784->23674879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.230632:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760730.230638:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2df2f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23670784->23674879] (req 23670784->23674879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.230647:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2df2f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23670784->23674879] (req 23670784->23674879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.230668:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2df2f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23670784->23674879] (req 23670784->23674879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.230677:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2df2f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23670784->23674879] (req 23670784->23674879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760730.230951:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febfbeae300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512599552:12345-192.168.203.56@tcp:4:dd.0 Request processed in 31567us (31859us total) trans 4294973098 rc 0/0 00000100:00100000:0.0:1789760730.230962:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5809 00000100:00100000:0.0:1789760730.230972:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 709731349176 time 709763666389 state 0x2 ref 0 00000100:00100000:2.0:1789760730.233830:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760730.233874:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512599680 00000100:00100000:3.0:1789760730.233884:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512599680 00000100:00100000:3.0:1789760730.233937:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 709731349176 time 709763666389 state 0x0 ref 1 00000100:00100000:3.0:1789760730.233953:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 709763926805 time 709763666389 state 0x1 ref 1 00000100:00100000:3.0:1789760730.233986:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 709763926805 time 709798285843 state 0x0 ref 1 00000100:00100000:3.0:1789760730.233999:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2fb8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512599680:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760730.234044:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23674880, mtime: 0x6aad94d9, atime: 0x0, ctime: 0x6aad94d9, blocks: 0xb807 00010000:00010000:3.0:1789760730.234052:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2df36 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.234068:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=709s v=(0 0) 00010000:00010000:3.0:1789760730.234072:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760730.234078:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2df36 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23674880->23678975] (req 23674880->23678975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.234104:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2df36 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23674880->23678975] (req 23674880->23678975) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760730.262468:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973099 is committed 00002000:00010000:3.0:1789760730.262566:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23674880 -> 23678976 00010000:00010000:3.0:1789760730.262581:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2df36 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23674880->23678975] (req 23674880->23678975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.262595:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2df36 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23674880->23678975] (req 23674880->23678975) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.262610:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2df36 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23674880->23678975] (req 23674880->23678975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.262618:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760730.262623:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2df36 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23674880->23678975] (req 23674880->23678975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.262633:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2df36 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23674880->23678975] (req 23674880->23678975) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.262652:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2df36 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23674880->23678975] (req 23674880->23678975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.262659:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2df36 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23674880->23678975] (req 23674880->23678975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760730.262804:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2fb8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512599680:12345-192.168.203.56@tcp:4:dd.0 Request processed in 28802us (28979us total) trans 4294973099 rc 0/0 00000100:00100000:3.0:1789760730.262825:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5810 00000100:00100000:3.0:1789760730.262835:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 709763926805 time 709798285843 state 0x2 ref 0 00000100:00100000:1.0:1789760730.266581:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760730.268057:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512599808 00000100:00100000:0.0:1789760730.268072:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512599808 00000100:00100000:0.0:1789760730.268137:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 709763926805 time 709798285843 state 0x0 ref 1 00000100:00100000:0.0:1789760730.268156:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 709798546259 time 709798285843 state 0x1 ref 1 00000100:00100000:0.0:1789760730.268391:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 709798546259 time 709832689319 state 0x0 ref 1 00000100:00100000:0.0:1789760730.268404:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4130000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512599808:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760730.268463:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23678976, mtime: 0x6aad94d9, atime: 0x0, ctime: 0x6aad94d9, blocks: 0xb807 00010000:00010000:0.0:1789760730.268473:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2df3d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.268489:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=709s v=(0 0) 00010000:00010000:0.0:1789760730.268494:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760730.268500:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2df3d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23678976->23683071] (req 23678976->23683071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.268514:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2df3d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23678976->23683071] (req 23678976->23683071) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760730.307801:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973100 is committed 00002000:00010000:0.0:1789760730.307879:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23678976 -> 23683072 00010000:00010000:0.0:1789760730.307894:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2df3d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23678976->23683071] (req 23678976->23683071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.307907:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2df3d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23678976->23683071] (req 23678976->23683071) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.307916:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2df3d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23678976->23683071] (req 23678976->23683071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.307924:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760730.307930:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2df3d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23678976->23683071] (req 23678976->23683071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.307939:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2df3d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23678976->23683071] (req 23678976->23683071) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.307958:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2df3d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23678976->23683071] (req 23678976->23683071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.307966:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2df3d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23678976->23683071] (req 23678976->23683071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760730.308050:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4130000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512599808:12345-192.168.203.56@tcp:4:dd.0 Request processed in 39644us (41475us total) trans 4294973100 rc 0/0 00000100:00100000:0.0:1789760730.308060:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5811 00000100:00100000:0.0:1789760730.308069:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 709798546259 time 709832689319 state 0x2 ref 0 00000100:00100000:2.0:1789760730.317767:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760730.320284:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512599936 00000100:00100000:0.0:1789760730.320300:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512599936 00000100:00100000:0.0:1789760730.320362:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 709798546259 time 709832689319 state 0x0 ref 1 00000100:00100000:0.0:1789760730.320377:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 709832949735 time 709832689319 state 0x1 ref 1 00000100:00100000:0.0:1789760730.320457:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 709832949735 time 709884756201 state 0x0 ref 1 00000100:00100000:0.0:1789760730.320475:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2fba680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512599936:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760730.320525:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23683072, mtime: 0x6aad94d9, atime: 0x0, ctime: 0x6aad94d9, blocks: 0xb807 00010000:00010000:0.0:1789760730.320533:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2df44 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.320549:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=709s v=(0 0) 00010000:00010000:0.0:1789760730.320553:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760730.320559:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2df44 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23683072->23687167] (req 23683072->23687167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.320572:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2df44 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23683072->23687167] (req 23683072->23687167) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760730.349657:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973101 is committed 00002000:00010000:2.0:1789760730.349863:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23683072 -> 23687168 00010000:00010000:2.0:1789760730.349877:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2df44 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23683072->23687167] (req 23683072->23687167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.349891:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2df44 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23683072->23687167] (req 23683072->23687167) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.349900:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2df44 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23683072->23687167] (req 23683072->23687167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.349908:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760730.349912:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2df44 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23683072->23687167] (req 23683072->23687167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.349921:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2df44 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23683072->23687167] (req 23683072->23687167) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.349939:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2df44 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23683072->23687167] (req 23683072->23687167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.349947:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2df44 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23683072->23687167] (req 23683072->23687167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760730.350031:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2fba680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512599936:12345-192.168.203.56@tcp:4:dd.0 Request processed in 29556us (32270us total) trans 4294973101 rc 0/0 00000100:00100000:2.0:1789760730.350041:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5812 00000100:00100000:2.0:1789760730.350050:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 709832949735 time 709884756201 state 0x2 ref 0 00000100:00100000:2.0:1789760730.352199:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760730.352280:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512600064 00000100:00100000:3.0:1789760730.352290:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512600064 00000100:00100000:3.0:1789760730.352338:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 709832949735 time 709884756201 state 0x0 ref 1 00000100:00100000:3.0:1789760730.352355:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 709885016617 time 709884756201 state 0x1 ref 1 00000100:00100000:3.0:1789760730.352384:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 709885016617 time 709916683697 state 0x0 ref 1 00000100:00100000:3.0:1789760730.352396:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2fb8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512600064:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760730.352448:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23687168, mtime: 0x6aad94d9, atime: 0x0, ctime: 0x6aad94d9, blocks: 0xb807 00010000:00010000:3.0:1789760730.352457:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2df4b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.352472:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=709s v=(0 0) 00010000:00010000:3.0:1789760730.352476:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760730.352482:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2df4b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23687168->23691263] (req 23687168->23691263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.352494:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2df4b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23687168->23691263] (req 23687168->23691263) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760730.388540:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973102 is committed 00002000:00010000:3.0:1789760730.388630:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23687168 -> 23691264 00010000:00010000:3.0:1789760730.388647:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2df4b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23687168->23691263] (req 23687168->23691263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.388660:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2df4b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23687168->23691263] (req 23687168->23691263) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.388670:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2df4b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23687168->23691263] (req 23687168->23691263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.388682:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760730.388687:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2df4b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23687168->23691263] (req 23687168->23691263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.388716:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2df4b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23687168->23691263] (req 23687168->23691263) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.388736:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2df4b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23687168->23691263] (req 23687168->23691263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.388744:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2df4b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23687168->23691263] (req 23687168->23691263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760730.388873:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2fb8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512600064:12345-192.168.203.56@tcp:4:dd.0 Request processed in 36475us (36679us total) trans 4294973102 rc 0/0 00000100:00100000:3.0:1789760730.388884:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5813 00000100:00100000:3.0:1789760730.388893:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 709885016617 time 709916683697 state 0x2 ref 0 00000100:00100000:0.0:1789760730.394229:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760730.396493:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512600192 00000100:00100000:2.0:1789760730.396510:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512600192 00000100:00100000:2.0:1789760730.396568:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 709885016617 time 709916683697 state 0x0 ref 1 00000100:00100000:2.0:1789760730.396584:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 709916944113 time 709916683697 state 0x1 ref 1 00000100:00100000:2.0:1789760730.396615:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 709916944113 time 709960915296 state 0x0 ref 1 00000100:00100000:2.0:1789760730.396627:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2875c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512600192:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760730.396671:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23691264, mtime: 0x6aad94d9, atime: 0x0, ctime: 0x6aad94d9, blocks: 0xb807 00010000:00010000:2.0:1789760730.396679:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2df52 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.396789:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=709s v=(0 0) 00010000:00010000:2.0:1789760730.396794:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760730.396800:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2df52 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23691264->23695359] (req 23691264->23695359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.396815:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2df52 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23691264->23695359] (req 23691264->23695359) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760730.448487:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973103 is committed 00002000:00010000:2.0:1789760730.448564:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23691264 -> 23695360 00010000:00010000:2.0:1789760730.448579:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2df52 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23691264->23695359] (req 23691264->23695359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.448600:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2df52 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23691264->23695359] (req 23691264->23695359) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.448610:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2df52 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23691264->23695359] (req 23691264->23695359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.448618:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760730.448623:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2df52 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23691264->23695359] (req 23691264->23695359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.448632:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2df52 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23691264->23695359] (req 23691264->23695359) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.448653:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2df52 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23691264->23695359] (req 23691264->23695359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.448661:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2df52 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23691264->23695359] (req 23691264->23695359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760730.448771:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2875c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512600192:12345-192.168.203.56@tcp:4:dd.0 Request processed in 52142us (54546us total) trans 4294973103 rc 0/0 00000100:00100000:2.0:1789760730.448781:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5814 00000100:00100000:2.0:1789760730.448791:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 709916944113 time 709960915296 state 0x2 ref 0 00000100:00100000:2.0:1789760730.456498:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760730.456610:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512600320 00000100:00100000:0.0:1789760730.456619:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512600320 00000100:00100000:0.0:1789760730.456675:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 709916944113 time 709960915296 state 0x0 ref 1 00000100:00100000:0.0:1789760730.456689:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 709961175712 time 709960915296 state 0x1 ref 1 00000100:00100000:2.0:1789760730.456966:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 709961175712 time 710021263028 state 0x0 ref 1 00000100:00100000:2.0:1789760730.456980:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2fbb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512600320:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760730.457026:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23695360, mtime: 0x6aad94d9, atime: 0x0, ctime: 0x6aad94d9, blocks: 0xb807 00010000:00010000:2.0:1789760730.457035:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2df59 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.457051:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=710s v=(0 0) 00010000:00010000:2.0:1789760730.457055:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760730.457062:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2df59 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23695360->23699455] (req 23695360->23699455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.457075:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2df59 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23695360->23699455] (req 23695360->23699455) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.482767:0:5835:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:2.0:1789760730.493401:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23695360 -> 23699456 00010000:00010000:2.0:1789760730.493429:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2df59 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23695360->23699455] (req 23695360->23699455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.493441:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2df59 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23695360->23699455] (req 23695360->23699455) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.493449:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2df59 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23695360->23699455] (req 23695360->23699455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.493455:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760730.493460:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2df59 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23695360->23699455] (req 23695360->23699455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.493469:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2df59 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23695360->23699455] (req 23695360->23699455) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.493486:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2df59 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23695360->23699455] (req 23695360->23699455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.493493:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2df59 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23695360->23699455] (req 23695360->23699455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760730.493578:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2fbb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2360:x1876699512600320:12345-192.168.203.56@tcp:4:dd.0 Request processed in 36596us (37083us total) trans 4294973104 rc 0/0 00000100:00100000:2.0:1789760730.493588:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5815 00000100:00100000:2.0:1789760730.493597:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 709961175712 time 710021263028 state 0x2 ref 0 00000001:00080000:3.0:1789760730.494444:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973104 is committed 00000100:00100000:3.0:1789760730.499182:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760730.499634:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512600448 00000100:00100000:0.0:1789760730.499644:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512600448 00000100:00100000:0.0:1789760730.500549:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 709961175712 time 710021263028 state 0x0 ref 1 00000100:00100000:0.0:1789760730.500576:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 710021523444 time 710021263028 state 0x1 ref 1 00000100:00100000:1.0:1789760730.500723:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 710021523444 time 710064992759 state 0x0 ref 1 00000100:00100000:1.0:1789760730.500742:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac3352a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512600448:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760730.501021:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23699456, mtime: 0x6aad94d9, atime: 0x0, ctime: 0x6aad94d9, blocks: 0xb807 00010000:00010000:1.0:1789760730.501031:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2df60 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.501046:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=710s v=(0 0) 00010000:00010000:1.0:1789760730.501050:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760730.501057:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2df60 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23699456->23703551] (req 23699456->23703551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.501071:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2df60 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23699456->23703551] (req 23699456->23703551) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760730.536412:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973105 is committed 00002000:00010000:0.0:1789760730.536518:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23699456 -> 23703552 00002000:00010000:0.0:1789760730.536523:0:6709:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760729 -> 1789760730 00002000:00010000:0.0:1789760730.536525:0:6709:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760729 -> 1789760730 00010000:00010000:0.0:1789760730.536537:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2df60 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23699456->23703551] (req 23699456->23703551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.536550:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2df60 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23699456->23703551] (req 23699456->23703551) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.536559:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2df60 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23699456->23703551] (req 23699456->23703551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.536566:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760730.536570:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2df60 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23699456->23703551] (req 23699456->23703551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.536579:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2df60 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23699456->23703551] (req 23699456->23703551) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.536597:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2df60 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23699456->23703551] (req 23699456->23703551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.536605:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2df60 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23699456->23703551] (req 23699456->23703551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760730.536693:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac3352a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512600448:12345-192.168.203.56@tcp:4:dd.0 Request processed in 35954us (37517us total) trans 4294973105 rc 0/0 00000100:00100000:0.0:1789760730.536744:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5816 00000100:00100000:0.0:1789760730.536753:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 710021523444 time 710064992759 state 0x2 ref 0 00000100:00100000:0.0:1789760730.541564:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760730.541669:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512600576 00000100:00100000:2.0:1789760730.541679:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512600576 00000100:00100000:2.0:1789760730.542051:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 710021523444 time 710064992759 state 0x0 ref 1 00000100:00100000:2.0:1789760730.542069:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 710065253175 time 710064992759 state 0x1 ref 1 00000100:00100000:2.0:1789760730.542100:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 710065253175 time 710106400206 state 0x0 ref 1 00000100:00100000:2.0:1789760730.542113:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2877100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512600576:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760730.542161:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23703552, mtime: 0x6aad94da, atime: 0x0, ctime: 0x6aad94da, blocks: 0xb807 00010000:00010000:2.0:1789760730.542170:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2df67 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.542299:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=710s v=(0 0) 00010000:00010000:2.0:1789760730.542304:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760730.542381:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2df67 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23703552->23707647] (req 23703552->23707647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.542398:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2df67 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23703552->23707647] (req 23703552->23707647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760730.595976:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973106 is committed 00002000:00010000:1.0:1789760730.596116:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23703552 -> 23707648 00010000:00010000:1.0:1789760730.596131:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2df67 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23703552->23707647] (req 23703552->23707647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.596142:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2df67 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23703552->23707647] (req 23703552->23707647) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.596155:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2df67 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23703552->23707647] (req 23703552->23707647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.596161:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760730.596166:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2df67 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23703552->23707647] (req 23703552->23707647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.596390:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2df67 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23703552->23707647] (req 23703552->23707647) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.596411:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2df67 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23703552->23707647] (req 23703552->23707647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.596429:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2df67 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23703552->23707647] (req 23703552->23707647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760730.596564:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2877100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512600576:12345-192.168.203.56@tcp:4:dd.0 Request processed in 54449us (55003us total) trans 4294973106 rc 0/0 00000100:00100000:1.0:1789760730.596575:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5817 00000100:00100000:1.0:1789760730.596584:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 710065253175 time 710106400206 state 0x2 ref 0 00000100:00100000:2.0:1789760730.600862:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760730.601464:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512600704 00000100:00100000:0.0:1789760730.601474:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512600704 00000100:00100000:0.0:1789760730.601528:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 710065253175 time 710106400206 state 0x0 ref 1 00000100:00100000:0.0:1789760730.601544:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 710106660622 time 710106400206 state 0x1 ref 1 00000100:00100000:0.0:1789760730.601577:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 710106660622 time 710165876431 state 0x0 ref 1 00000100:00100000:0.0:1789760730.601591:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2fb8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512600704:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760730.601641:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23707648, mtime: 0x6aad94da, atime: 0x0, ctime: 0x6aad94da, blocks: 0xb807 00010000:00010000:0.0:1789760730.601650:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2df6e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.601667:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=710s v=(0 0) 00010000:00010000:0.0:1789760730.601673:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760730.601679:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2df6e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23707648->23711743] (req 23707648->23711743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.601693:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2df6e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23707648->23711743] (req 23707648->23711743) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760730.646182:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973107 is committed 00002000:00010000:1.0:1789760730.646347:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23707648 -> 23711744 00010000:00010000:1.0:1789760730.646362:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2df6e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23707648->23711743] (req 23707648->23711743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.646395:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2df6e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23707648->23711743] (req 23707648->23711743) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.646404:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2df6e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23707648->23711743] (req 23707648->23711743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.646412:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760730.646427:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2df6e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23707648->23711743] (req 23707648->23711743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.646437:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2df6e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23707648->23711743] (req 23707648->23711743) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.646457:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2df6e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23707648->23711743] (req 23707648->23711743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.646464:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2df6e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23707648->23711743] (req 23707648->23711743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760730.646554:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2fb8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512600704:12345-192.168.203.56@tcp:4:dd.0 Request processed in 44962us (45697us total) trans 4294973107 rc 0/0 00000100:00100000:1.0:1789760730.646564:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5818 00000100:00100000:1.0:1789760730.646574:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 710106660622 time 710165876431 state 0x2 ref 0 00000100:00100000:3.0:1789760730.650671:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760730.650965:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512600832 00000100:00100000:0.0:1789760730.650974:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512600832 00000100:00100000:0.0:1789760730.651028:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 710106660622 time 710165876431 state 0x0 ref 1 00000100:00100000:0.0:1789760730.651044:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 710166136847 time 710165876431 state 0x1 ref 1 00000100:00100000:0.0:1789760730.651103:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 710166136847 time 710215402607 state 0x0 ref 1 00000100:00100000:0.0:1789760730.651115:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac3353480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512600832:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760730.651164:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23711744, mtime: 0x6aad94da, atime: 0x0, ctime: 0x6aad94da, blocks: 0xb807 00010000:00010000:0.0:1789760730.651173:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2df75 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.651189:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=710s v=(0 0) 00010000:00010000:0.0:1789760730.651193:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760730.651200:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2df75 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23711744->23715839] (req 23711744->23715839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.651213:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2df75 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23711744->23715839] (req 23711744->23715839) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760730.693134:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973108 is committed 00002000:00010000:0.0:1789760730.693198:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23711744 -> 23715840 00010000:00010000:0.0:1789760730.693214:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2df75 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23711744->23715839] (req 23711744->23715839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.693228:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2df75 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23711744->23715839] (req 23711744->23715839) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.693238:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2df75 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23711744->23715839] (req 23711744->23715839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.693246:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760730.693251:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2df75 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23711744->23715839] (req 23711744->23715839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.693260:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2df75 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23711744->23715839] (req 23711744->23715839) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.693278:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2df75 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23711744->23715839] (req 23711744->23715839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.693297:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2df75 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23711744->23715839] (req 23711744->23715839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760730.693397:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac3353480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512600832:12345-192.168.203.56@tcp:4:dd.0 Request processed in 42278us (42730us total) trans 4294973108 rc 0/0 00000100:00100000:0.0:1789760730.693407:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5819 00000100:00100000:0.0:1789760730.693428:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 710166136847 time 710215402607 state 0x2 ref 0 00000100:00100000:0.0:1789760730.702328:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760730.702452:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512600960 00000100:00100000:1.0:1789760730.702462:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512600960 00000100:00100000:1.0:1789760730.702518:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 710166136847 time 710215402607 state 0x0 ref 1 00000100:00100000:1.0:1789760730.702534:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 710215663023 time 710215402607 state 0x1 ref 1 00000100:00100000:1.0:1789760730.702568:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 710215663023 time 710266867144 state 0x0 ref 1 00000100:00100000:1.0:1789760730.702581:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2877b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512600960:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760730.702628:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23715840, mtime: 0x6aad94da, atime: 0x0, ctime: 0x6aad94da, blocks: 0xb807 00010000:00010000:1.0:1789760730.702637:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2df7c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.702651:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=710s v=(0 0) 00010000:00010000:1.0:1789760730.702656:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760730.702662:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2df7c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23715840->23719935] (req 23715840->23719935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.702675:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2df7c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23715840->23719935] (req 23715840->23719935) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760730.751395:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973109 is committed 00002000:00010000:1.0:1789760730.751502:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23715840 -> 23719936 00010000:00010000:1.0:1789760730.751517:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2df7c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23715840->23719935] (req 23715840->23719935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.751531:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2df7c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23715840->23719935] (req 23715840->23719935) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.751545:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2df7c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23715840->23719935] (req 23715840->23719935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.751552:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760730.751556:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2df7c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23715840->23719935] (req 23715840->23719935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.751565:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2df7c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23715840->23719935] (req 23715840->23719935) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.751584:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2df7c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23715840->23719935] (req 23715840->23719935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.751593:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2df7c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23715840->23719935] (req 23715840->23719935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760730.751684:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2877b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512600960:12345-192.168.203.56@tcp:4:dd.0 Request processed in 49101us (49361us total) trans 4294973109 rc 0/0 00000100:00100000:1.0:1789760730.752191:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5820 00000100:00100000:1.0:1789760730.752201:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 710215663023 time 710266867144 state 0x2 ref 0 00000100:00100000:3.0:1789760730.754223:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760730.754344:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512601088 00000100:00100000:1.0:1789760730.754353:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512601088 00000100:00100000:1.0:1789760730.754404:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 710215663023 time 710266867144 state 0x0 ref 1 00000100:00100000:1.0:1789760730.754431:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 710267127560 time 710266867144 state 0x1 ref 1 00000100:00100000:1.0:1789760730.754465:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 710267127560 time 710318764272 state 0x0 ref 1 00000100:00100000:1.0:1789760730.754477:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febfb894700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512601088:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760730.754522:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23719936, mtime: 0x6aad94da, atime: 0x0, ctime: 0x6aad94da, blocks: 0xb807 00010000:00010000:1.0:1789760730.754530:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2df83 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.754544:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=710s v=(0 0) 00010000:00010000:1.0:1789760730.754549:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760730.754555:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2df83 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23719936->23724031] (req 23719936->23724031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.754567:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2df83 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23719936->23724031] (req 23719936->23724031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760730.790237:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973110 is committed 00002000:00010000:3.0:1789760730.790465:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23719936 -> 23724032 00010000:00010000:3.0:1789760730.790480:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2df83 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23719936->23724031] (req 23719936->23724031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.790499:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2df83 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23719936->23724031] (req 23719936->23724031) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.790508:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2df83 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23719936->23724031] (req 23719936->23724031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.790515:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760730.790520:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2df83 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23719936->23724031] (req 23719936->23724031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.790529:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2df83 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23719936->23724031] (req 23719936->23724031) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.790552:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2df83 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23719936->23724031] (req 23719936->23724031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.790560:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2df83 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23719936->23724031] (req 23719936->23724031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760730.790805:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febfb894700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512601088:12345-192.168.203.56@tcp:4:dd.0 Request processed in 36325us (36586us total) trans 4294973110 rc 0/0 00000100:00100000:3.0:1789760730.790815:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5821 00000100:00100000:3.0:1789760730.790825:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 710267127560 time 710318764272 state 0x2 ref 0 00000100:00100000:3.0:1789760730.795624:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760730.795786:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512601216 00000100:00100000:0.0:1789760730.795796:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512601216 00000100:00100000:0.0:1789760730.795858:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 710267127560 time 710318764272 state 0x0 ref 1 00000100:00100000:0.0:1789760730.795873:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 710319024688 time 710318764272 state 0x1 ref 1 00000100:00100000:0.0:1789760730.795902:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 710319024688 time 710360202457 state 0x0 ref 1 00000100:00100000:0.0:1789760730.795914:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febfb897100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512601216:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760730.795960:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23724032, mtime: 0x6aad94da, atime: 0x0, ctime: 0x6aad94da, blocks: 0xb807 00010000:00010000:0.0:1789760730.795968:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2df8a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.795983:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=710s v=(0 0) 00010000:00010000:0.0:1789760730.795988:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760730.795994:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2df8a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23724032->23728127] (req 23724032->23728127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.796006:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2df8a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23724032->23728127] (req 23724032->23728127) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760730.834391:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23724032 -> 23728128 00010000:00010000:0.0:1789760730.834410:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2df8a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23724032->23728127] (req 23724032->23728127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.834440:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2df8a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23724032->23728127] (req 23724032->23728127) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.834449:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2df8a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23724032->23728127] (req 23724032->23728127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.834456:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760730.834462:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2df8a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23724032->23728127] (req 23724032->23728127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.834471:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2df8a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23724032->23728127] (req 23724032->23728127) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.834491:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2df8a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23724032->23728127] (req 23724032->23728127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.834499:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2df8a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23724032->23728127] (req 23724032->23728127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760730.834600:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febfb897100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2359:x1876699512601216:12345-192.168.203.56@tcp:4:dd.0 Request processed in 38681us (38978us total) trans 4294973111 rc 0/0 00000100:00100000:0.0:1789760730.834610:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5822 00000100:00100000:0.0:1789760730.834620:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 710319024688 time 710360202457 state 0x2 ref 0 00000001:00080000:0.0:1789760730.834659:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973111 is committed 00000100:00100000:0.0:1789760730.839148:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760730.839218:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512601344 00000100:00100000:1.0:1789760730.839228:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512601344 00000100:00100000:1.0:1789760730.839279:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 710319024688 time 710360202457 state 0x0 ref 1 00000100:00100000:1.0:1789760730.839295:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 710360462873 time 710360202457 state 0x1 ref 1 00000100:00100000:1.0:1789760730.839331:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 710360462873 time 710403630225 state 0x0 ref 1 00000100:00100000:1.0:1789760730.839343:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2874700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512601344:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760730.839393:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23728128, mtime: 0x6aad94da, atime: 0x0, ctime: 0x6aad94da, blocks: 0xb807 00010000:00010000:1.0:1789760730.839403:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2df91 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.839428:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=710s v=(0 0) 00010000:00010000:1.0:1789760730.839433:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760730.839441:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2df91 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23728128->23732223] (req 23728128->23732223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.839455:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2df91 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23728128->23732223] (req 23728128->23732223) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760730.927297:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23728128 -> 23732224 00010000:00010000:2.0:1789760730.927319:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2df91 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23728128->23732223] (req 23728128->23732223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.927331:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2df91 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23728128->23732223] (req 23728128->23732223) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.927340:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2df91 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23728128->23732223] (req 23728128->23732223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.927347:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760730.927352:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2df91 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23728128->23732223] (req 23728128->23732223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.927361:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2df91 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23728128->23732223] (req 23728128->23732223) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.927380:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2df91 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23728128->23732223] (req 23728128->23732223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.927386:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2df91 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23728128->23732223] (req 23728128->23732223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760730.927485:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2874700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2359:x1876699512601344:12345-192.168.203.56@tcp:4:dd.0 Request processed in 88139us (88341us total) trans 4294973112 rc 0/0 00000100:00100000:2.0:1789760730.927495:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5823 00000100:00100000:2.0:1789760730.927505:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 710360462873 time 710403630225 state 0x2 ref 0 00000001:00080000:0.0:1789760730.928346:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973112 is committed 00000100:00100000:3.0:1789760730.931820:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760730.932119:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512601472 00000100:00100000:1.0:1789760730.932141:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512601472 00000100:00100000:1.0:1789760730.932207:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 710360462873 time 710403630225 state 0x0 ref 1 00000100:00100000:1.0:1789760730.932225:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 710403890641 time 710403630225 state 0x1 ref 1 00000100:00100000:1.0:1789760730.932256:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 710403890641 time 710496555892 state 0x0 ref 1 00000100:00100000:1.0:1789760730.932270:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febfb895880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512601472:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760730.932318:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23732224, mtime: 0x6aad94da, atime: 0x0, ctime: 0x6aad94da, blocks: 0xb807 00010000:00010000:1.0:1789760730.932327:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2df98 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.932345:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=710s v=(0 0) 00010000:00010000:1.0:1789760730.932350:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760730.932357:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2df98 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23732224->23736319] (req 23732224->23736319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.932373:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2df98 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23732224->23736319] (req 23732224->23736319) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760730.997321:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973113 is committed 00002000:00010000:2.0:1789760730.997410:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23732224 -> 23736320 00010000:00010000:2.0:1789760730.997436:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2df98 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23732224->23736319] (req 23732224->23736319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.997448:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2df98 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23732224->23736319] (req 23732224->23736319) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.997457:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2df98 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23732224->23736319] (req 23732224->23736319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.997465:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760730.997470:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2df98 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23732224->23736319] (req 23732224->23736319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.997485:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2df98 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23732224->23736319] (req 23732224->23736319) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.997506:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2df98 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23732224->23736319] (req 23732224->23736319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.997513:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2df98 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23732224->23736319] (req 23732224->23736319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760730.997644:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febfb895880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512601472:12345-192.168.203.56@tcp:4:dd.0 Request processed in 65372us (65841us total) trans 4294973113 rc 0/0 00000100:00100000:2.0:1789760730.997654:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5824 00000100:00100000:2.0:1789760730.997663:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 710403890641 time 710496555892 state 0x2 ref 0 00000100:00100000:3.0:1789760731.005050:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760731.005177:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512601600 00000100:00100000:0.0:1789760731.005187:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512601600 00000100:00100000:0.0:1789760731.005319:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 710403890641 time 710496555892 state 0x0 ref 1 00000100:00100000:0.0:1789760731.005334:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 710496816308 time 710496555892 state 0x1 ref 1 00000100:00100000:0.0:1789760731.005378:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 710496816308 time 710569677432 state 0x0 ref 1 00000100:00100000:0.0:1789760731.005391:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febfc0a3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512601600:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760731.005454:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23736320, mtime: 0x6aad94da, atime: 0x0, ctime: 0x6aad94da, blocks: 0xb807 00010000:00010000:0.0:1789760731.005463:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2df9f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.005479:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=710s v=(0 0) 00010000:00010000:0.0:1789760731.005483:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760731.005490:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2df9f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23736320->23740415] (req 23736320->23740415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.005503:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2df9f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23736320->23740415] (req 23736320->23740415) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1789760731.007833:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00010000:1.0:1789760731.044018:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23736320 -> 23740416 00010000:00010000:1.0:1789760731.044036:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2df9f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23736320->23740415] (req 23736320->23740415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.044048:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2df9f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23736320->23740415] (req 23736320->23740415) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.044056:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2df9f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23736320->23740415] (req 23736320->23740415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.044063:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760731.044067:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2df9f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23736320->23740415] (req 23736320->23740415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.044076:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2df9f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23736320->23740415] (req 23736320->23740415) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.044113:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2df9f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23736320->23740415] (req 23736320->23740415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.044121:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2df9f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23736320->23740415] (req 23736320->23740415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760731.044410:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febfc0a3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2358:x1876699512601600:12345-192.168.203.56@tcp:4:dd.0 Request processed in 39015us (39364us total) trans 4294973114 rc 0/0 00000100:00100000:1.0:1789760731.044430:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5825 00000100:00100000:1.0:1789760731.044440:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 710496816308 time 710569677432 state 0x2 ref 0 00000001:00080000:2.0:1789760731.045729:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973114 is committed 00000100:00100000:1.0:1789760731.048133:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760731.048259:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512601728 00000100:00100000:2.0:1789760731.048268:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512601728 00000100:00100000:2.0:1789760731.048318:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 710496816308 time 710569677432 state 0x0 ref 1 00000100:00100000:2.0:1789760731.048333:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 710569937848 time 710569677432 state 0x1 ref 1 00000100:00100000:2.0:1789760731.048365:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 710569937848 time 710612665057 state 0x0 ref 1 00000100:00100000:2.0:1789760731.048377:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4133480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512601728:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760731.048439:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23740416, mtime: 0x6aad94da, atime: 0x0, ctime: 0x6aad94da, blocks: 0xb807 00010000:00010000:2.0:1789760731.048449:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2dfa6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.048464:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=710s v=(0 0) 00010000:00010000:2.0:1789760731.048469:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760731.048475:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2dfa6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23740416->23744511] (req 23740416->23744511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.048489:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2dfa6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23740416->23744511] (req 23740416->23744511) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760731.058891:0:3310:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8febfc0a3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:6df98d54-eb52-4c5c-8fc9-ee9d74f90f98:3310:1876699540221568:0@lo:400:kworker.0 00000100:00100000:3.0:1789760731.058961:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760731.059007:0:3310:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8feac28a2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3310:1876699540222080:0@lo:400:kworker.0 00000100:00100000:3.0:1789760731.059036:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760731.059063:0:3310:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1789760731.059071:0:3310:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8feac28a1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3310:1876699540221952:0@lo:400:kworker.0 00000100:00100000:3.0:1789760731.059118:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760731.059151:0:3311:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8febc7b33480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3311:1876699540221696:0@lo:400:kworker.0 00000100:00100000:3.0:1789760731.059177:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760731.059195:0:3311:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8feac28a3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3311:1876699540222208:0@lo:400:kworker.0 00000100:00100000:3.0:1789760731.059215:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760731.059232:0:3311:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1789760731.059237:0:3311:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8febc7b32680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3311:1876699540221824:0@lo:400:kworker.0 00000100:00100000:3.0:1789760731.059256:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760731.059286:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540221568 00000100:00100000:3.0:1789760731.059293:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540221568 00000100:00100000:3.0:1789760731.059336:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 393 00000100:00100000:3.0:1789760731.059344:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febe0502450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:6df98d54-eb52-4c5c-8fc9-ee9d74f90f98+19:3310:x1876699540221568:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760731.059392:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febe0502450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:6df98d54-eb52-4c5c-8fc9-ee9d74f90f98+19:3310:x1876699540221568:12345-0@lo:400:kworker.0 Request processed in 47us (430us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760731.059399:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 393 00000100:00100000:3.0:1789760731.059442:0:5848:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540222080 00000100:00100000:3.0:1789760731.059447:0:5848:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540222080 00000100:00100000:3.0:1789760731.059479:0:5848:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540221696 00000100:00100000:3.0:1789760731.059483:0:5848:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540221696 00000100:00100000:3.0:1789760731.059496:0:5848:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540221824 00000100:00100000:3.0:1789760731.059498:0:5848:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540221824 00000100:00100000:3.0:1789760731.059515:0:5848:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 461 00000100:00100000:3.0:1789760731.059520:0:5848:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac28a0e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3310:x1876699540222080:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760731.059551:0:5848:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac28a0e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3310:x1876699540222080:12345-0@lo:400:kworker.0 Request processed in 31us (515us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760731.059557:0:5848:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 461 00000100:00100000:3.0:1789760731.059569:0:5848:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 462 00000100:00100000:3.0:1789760731.059573:0:5848:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac531c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3311:x1876699540221696:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760731.059603:0:5848:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac531c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3311:x1876699540221696:12345-0@lo:400:kworker.0 Request processed in 29us (424us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760731.059608:0:5848:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 462 00000100:00100000:3.0:1789760731.059619:0:5848:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 463 00000100:00100000:3.0:1789760731.059625:0:5848:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac531d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3311:x1876699540221824:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760731.059649:0:5848:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac531d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3311:x1876699540221824:12345-0@lo:400:kworker.0 Request processed in 25us (392us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760731.059654:0:5848:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 463 00000100:00100000:3.0:1789760731.059671:0:13064:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540221952 00000100:00100000:3.0:1789760731.059675:0:13064:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540221952 00000100:00100000:3.0:1789760731.059730:0:13064:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540222208 00000100:00100000:3.0:1789760731.059733:0:13064:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540222208 00000100:00100000:3.0:1789760731.059755:0:13064:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 177 00000100:00100000:3.0:1789760731.059762:0:13064:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac531f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3310:x1876699540221952:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760731.059792:0:13064:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac531f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3310:x1876699540221952:12345-0@lo:400:kworker.0 Request processed in 31us (673us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760731.059799:0:13064:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 177 00000100:00100000:3.0:1789760731.059852:0:13064:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 178 00000100:00100000:3.0:1789760731.059859:0:13064:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac531ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3311:x1876699540222208:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760731.059887:0:13064:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac531ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3311:x1876699540222208:12345-0@lo:400:kworker.0 Request processed in 29us (670us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760731.059892:0:13064:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 178 00000100:00080000:3.0:1789760731.059920:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:3.0:1789760731.059924:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:3.0:1789760731.059926:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789760731.059962:0:3311:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8febc7b32680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3311:1876699540221824:0@lo:400:kworker.0 00000100:00100000:3.0:1789760731.059972:0:3311:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8febc7b33480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3311:1876699540221696:0@lo:400:kworker.0 00000100:00100000:3.0:1789760731.059981:0:3311:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8feac28a3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3311:1876699540222208:0@lo:400:kworker.0 00000100:00100000:3.0:1789760731.060009:0:3310:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8feac28a1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3310:1876699540221952:0@lo:400:kworker.0 00000100:00100000:3.0:1789760731.060018:0:3310:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8febfc0a3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:6df98d54-eb52-4c5c-8fc9-ee9d74f90f98:3310:1876699540221568:0@lo:400:kworker.0 00000100:00100000:3.0:1789760731.060025:0:3310:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8feac28a2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3310:1876699540222080:0@lo:400:kworker.0 00000100:00100000:3.0:1789760731.061017:0:3311:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1789760731.061032:0:3311:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8feac531d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3311:1876699540222336:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1789760731.061075:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760731.061850:0:6706:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540222336 00000100:00100000:2.0:1789760731.061859:0:6706:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540222336 00000100:00100000:2.0:1789760731.061908:0:6706:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 192 00000100:00100000:2.0:1789760731.061917:0:6706:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac531d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3311:x1876699540222336:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1789760731.061938:0:6706:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 527630336 pending 1703936 free 3834642432 avail 3821010944 00000100:00100000:2.0:1789760731.061990:0:6706:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac531d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3311:x1876699540222336:12345-0@lo:13:osp-pre-0-0.0 Request processed in 74us (914us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760731.061997:0:6706:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 192 00000100:00100000:2.0:1789760731.062043:0:3311:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8feac531d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3311:1876699540222336:0@lo:13:osp-pre-0-0.0 00002000:00010000:2.0:1789760731.080441:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23740416 -> 23744512 00010000:00010000:2.0:1789760731.080462:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2dfa6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23740416->23744511] (req 23740416->23744511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.080477:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2dfa6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23740416->23744511] (req 23740416->23744511) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.080487:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2dfa6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23740416->23744511] (req 23740416->23744511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.080495:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760731.080500:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2dfa6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23740416->23744511] (req 23740416->23744511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.080510:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2dfa6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23740416->23744511] (req 23740416->23744511) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.080531:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2dfa6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23740416->23744511] (req 23740416->23744511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.080539:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2dfa6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23740416->23744511] (req 23740416->23744511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760731.080635:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4133480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2359:x1876699512601728:12345-192.168.203.56@tcp:4:dd.0 Request processed in 32255us (32504us total) trans 4294973115 rc 0/0 00000100:00100000:2.0:1789760731.080646:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5826 00000100:00100000:2.0:1789760731.080656:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 710569937848 time 710612665057 state 0x2 ref 0 00000001:00080000:2.0:1789760731.080729:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973115 is committed 00000100:00100000:0.0:1789760731.084410:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760731.084502:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512601856 00000100:00100000:2.0:1789760731.084512:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512601856 00000100:00100000:2.0:1789760731.084568:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 710569937848 time 710612665057 state 0x0 ref 1 00000100:00100000:2.0:1789760731.084586:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 710612925473 time 710612665057 state 0x1 ref 1 00000100:00100000:2.0:1789760731.084618:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 710612925473 time 710648918949 state 0x0 ref 1 00000100:00100000:2.0:1789760731.084631:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2876a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512601856:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760731.084680:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23744512, mtime: 0x6aad94da, atime: 0x0, ctime: 0x6aad94da, blocks: 0xb807 00010000:00010000:2.0:1789760731.084689:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2dfad lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.084741:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=710s v=(0 0) 00010000:00010000:2.0:1789760731.084745:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760731.084751:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2dfad lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23744512->23748607] (req 23744512->23748607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.084763:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2dfad lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23744512->23748607] (req 23744512->23748607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760731.113722:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23744512 -> 23748608 00010000:00010000:2.0:1789760731.113744:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2dfad lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23744512->23748607] (req 23744512->23748607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.113759:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2dfad lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23744512->23748607] (req 23744512->23748607) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.113770:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2dfad lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23744512->23748607] (req 23744512->23748607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.113783:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760731.113789:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2dfad lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23744512->23748607] (req 23744512->23748607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.113799:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2dfad lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23744512->23748607] (req 23744512->23748607) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.113819:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2dfad lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23744512->23748607] (req 23744512->23748607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.113826:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2dfad lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23744512->23748607] (req 23744512->23748607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760731.113925:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2876a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2358:x1876699512601856:12345-192.168.203.56@tcp:4:dd.0 Request processed in 29290us (29518us total) trans 4294973116 rc 0/0 00000100:00100000:2.0:1789760731.113935:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5827 00000100:00100000:2.0:1789760731.113945:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 710612925473 time 710648918949 state 0x2 ref 0 00000001:00080000:0.0:1789760731.114240:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973116 is committed 00000100:00100000:0.0:1789760731.123367:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760731.123472:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512601984 00000100:00100000:1.0:1789760731.123482:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512601984 00000100:00100000:1.0:1789760731.123533:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 710612925473 time 710648918949 state 0x0 ref 1 00000100:00100000:1.0:1789760731.123549:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 710649179365 time 710648918949 state 0x1 ref 1 00000100:00100000:1.0:1789760731.123580:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 710649179365 time 710687879108 state 0x0 ref 1 00000100:00100000:1.0:1789760731.123593:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2875880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512601984:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760731.123645:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23748608, mtime: 0x6aad94da, atime: 0x0, ctime: 0x6aad94da, blocks: 0xb807 00010000:00010000:1.0:1789760731.123654:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2dfb4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.123669:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=710s v=(0 0) 00010000:00010000:1.0:1789760731.123674:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760731.123680:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2dfb4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23748608->23752703] (req 23748608->23752703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.123693:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2dfb4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23748608->23752703] (req 23748608->23752703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760731.166941:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973117 is committed 00002000:00010000:1.0:1789760731.167800:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23748608 -> 23752704 00010000:00010000:1.0:1789760731.167817:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2dfb4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23748608->23752703] (req 23748608->23752703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.167830:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2dfb4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23748608->23752703] (req 23748608->23752703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.167839:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2dfb4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23748608->23752703] (req 23748608->23752703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.167847:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760731.167853:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2dfb4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23748608->23752703] (req 23748608->23752703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.167862:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2dfb4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23748608->23752703] (req 23748608->23752703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.167881:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2dfb4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23748608->23752703] (req 23748608->23752703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.167889:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2dfb4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23748608->23752703] (req 23748608->23752703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760731.168049:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2875880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512601984:12345-192.168.203.56@tcp:4:dd.0 Request processed in 44452us (44764us total) trans 4294973117 rc 0/0 00000100:00100000:1.0:1789760731.168063:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5828 00000100:00100000:1.0:1789760731.168073:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 710649179365 time 710687879108 state 0x2 ref 0 00000100:00100000:1.0:1789760731.175499:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760731.177285:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512602112 00000100:00100000:2.0:1789760731.177301:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512602112 00000100:00100000:2.0:1789760731.177448:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 710649179365 time 710687879108 state 0x0 ref 1 00000100:00100000:2.0:1789760731.177465:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 710688139524 time 710687879108 state 0x1 ref 1 00000100:00100000:2.0:1789760731.177559:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 710688139524 time 710741857776 state 0x0 ref 1 00000100:00100000:2.0:1789760731.177576:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4130700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512602112:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760731.177627:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23752704, mtime: 0x6aad94da, atime: 0x0, ctime: 0x6aad94da, blocks: 0xb807 00010000:00010000:2.0:1789760731.177637:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2dfbb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.177652:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=710s v=(0 0) 00010000:00010000:2.0:1789760731.177656:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760731.177663:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2dfbb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23752704->23756799] (req 23752704->23756799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.177677:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2dfbb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23752704->23756799] (req 23752704->23756799) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760731.237479:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23752704 -> 23756800 00010000:00010000:2.0:1789760731.237500:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2dfbb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23752704->23756799] (req 23752704->23756799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.237515:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2dfbb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23752704->23756799] (req 23752704->23756799) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.237524:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2dfbb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23752704->23756799] (req 23752704->23756799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.237536:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760731.237541:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2dfbb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23752704->23756799] (req 23752704->23756799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.237550:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2dfbb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23752704->23756799] (req 23752704->23756799) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760731.237568:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973118 is committed 00010000:00010000:2.0:1789760731.237569:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2dfbb lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23752704->23756799] (req 23752704->23756799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.237576:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2dfbb lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23752704->23756799] (req 23752704->23756799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760731.237665:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4130700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512602112:12345-192.168.203.56@tcp:4:dd.0 Request processed in 60087us (62172us total) trans 4294973118 rc 0/0 00000100:00100000:2.0:1789760731.237675:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5829 00000100:00100000:2.0:1789760731.237685:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 710688139524 time 710741857776 state 0x2 ref 0 00000100:00100000:3.0:1789760731.239923:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760731.240157:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512602240 00000100:00100000:2.0:1789760731.240168:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512602240 00000100:00100000:2.0:1789760731.240217:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 710688139524 time 710741857776 state 0x0 ref 1 00000100:00100000:2.0:1789760731.240232:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 710742118192 time 710741857776 state 0x1 ref 1 00000100:00100000:2.0:1789760731.240260:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 710742118192 time 710804559622 state 0x0 ref 1 00000100:00100000:2.0:1789760731.240271:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac531ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512602240:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760731.240488:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23756800, mtime: 0x6aad94da, atime: 0x0, ctime: 0x6aad94da, blocks: 0xb807 00010000:00010000:0.0:1789760731.240499:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2dfc2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.240519:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=710s v=(0 0) 00010000:00010000:0.0:1789760731.240524:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760731.240530:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2dfc2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23756800->23760895] (req 23756800->23760895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.240545:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2dfc2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23756800->23760895] (req 23756800->23760895) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760731.285668:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973119 is committed 00002000:00010000:3.0:1789760731.285949:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23756800 -> 23760896 00010000:00010000:3.0:1789760731.285964:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2dfc2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23756800->23760895] (req 23756800->23760895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.285978:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2dfc2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23756800->23760895] (req 23756800->23760895) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.285988:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2dfc2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23756800->23760895] (req 23756800->23760895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.285995:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760731.286000:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2dfc2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23756800->23760895] (req 23756800->23760895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.286009:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2dfc2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23756800->23760895] (req 23756800->23760895) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.286028:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2dfc2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23756800->23760895] (req 23756800->23760895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.286036:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2dfc2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23756800->23760895] (req 23756800->23760895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760731.286142:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac531ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512602240:12345-192.168.203.56@tcp:4:dd.0 Request processed in 45867us (46222us total) trans 4294973119 rc 0/0 00000100:00100000:3.0:1789760731.286152:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5830 00000100:00100000:3.0:1789760731.286163:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 710742118192 time 710804559622 state 0x2 ref 0 00000100:00100000:0.0:1789760731.289084:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760731.290459:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512602368 00000100:00100000:1.0:1789760731.290474:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512602368 00000100:00100000:1.0:1789760731.290534:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 710742118192 time 710804559622 state 0x0 ref 1 00000100:00100000:1.0:1789760731.290550:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 710804820038 time 710804559622 state 0x1 ref 1 00000100:00100000:1.0:1789760731.290589:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 710804820038 time 710854889523 state 0x0 ref 1 00000100:00100000:1.0:1789760731.290602:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2fb6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512602368:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760731.290650:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23760896, mtime: 0x6aad94da, atime: 0x0, ctime: 0x6aad94da, blocks: 0xb807 00010000:00010000:1.0:1789760731.290659:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2dfc9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.290676:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=710s v=(0 0) 00010000:00010000:1.0:1789760731.290680:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760731.290685:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2dfc9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23760896->23764991] (req 23760896->23764991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.290736:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2dfc9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23760896->23764991] (req 23760896->23764991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760731.349592:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973120 is committed 00002000:00010000:2.0:1789760731.349953:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23760896 -> 23764992 00010000:00010000:2.0:1789760731.349969:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2dfc9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23760896->23764991] (req 23760896->23764991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.349982:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2dfc9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23760896->23764991] (req 23760896->23764991) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.349991:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2dfc9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23760896->23764991] (req 23760896->23764991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.349999:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760731.350003:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2dfc9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23760896->23764991] (req 23760896->23764991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.350012:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2dfc9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23760896->23764991] (req 23760896->23764991) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.350032:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2dfc9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23760896->23764991] (req 23760896->23764991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.350041:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2dfc9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23760896->23764991] (req 23760896->23764991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760731.350233:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2fb6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512602368:12345-192.168.203.56@tcp:4:dd.0 Request processed in 59628us (61153us total) trans 4294973120 rc 0/0 00000100:00100000:2.0:1789760731.350244:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5831 00000100:00100000:2.0:1789760731.350255:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 710804820038 time 710854889523 state 0x2 ref 0 00000100:00100000:0.0:1789760731.355753:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760731.355893:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512602496 00000100:00100000:2.0:1789760731.355903:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512602496 00000100:00100000:2.0:1789760731.355955:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 710804820038 time 710854889523 state 0x0 ref 1 00000100:00100000:2.0:1789760731.355969:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 710855149939 time 710854889523 state 0x1 ref 1 00000100:00100000:2.0:1789760731.355999:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 710855149939 time 710920298735 state 0x0 ref 1 00000100:00100000:2.0:1789760731.356011:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febe7907100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512602496:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760731.356055:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23764992, mtime: 0x6aad94da, atime: 0x0, ctime: 0x6aad94da, blocks: 0xb807 00010000:00010000:2.0:1789760731.356065:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2dfd0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.356091:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=710s v=(0 0) 00010000:00010000:2.0:1789760731.356096:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760731.356103:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2dfd0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23764992->23769087] (req 23764992->23769087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.356117:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2dfd0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23764992->23769087] (req 23764992->23769087) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760731.395736:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973121 is committed 00002000:00010000:2.0:1789760731.396583:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23764992 -> 23769088 00010000:00010000:2.0:1789760731.396599:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2dfd0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23764992->23769087] (req 23764992->23769087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.396612:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2dfd0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23764992->23769087] (req 23764992->23769087) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.396626:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2dfd0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23764992->23769087] (req 23764992->23769087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.396633:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760731.396638:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2dfd0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23764992->23769087] (req 23764992->23769087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.396647:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2dfd0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23764992->23769087] (req 23764992->23769087) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.396664:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2dfd0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23764992->23769087] (req 23764992->23769087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.396672:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2dfd0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23764992->23769087] (req 23764992->23769087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760731.396794:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febe7907100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512602496:12345-192.168.203.56@tcp:4:dd.0 Request processed in 40780us (41047us total) trans 4294973121 rc 0/0 00000100:00100000:2.0:1789760731.396810:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5832 00000100:00100000:2.0:1789760731.396820:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 710855149939 time 710920298735 state 0x2 ref 0 00000100:00100000:0.0:1789760731.399577:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760731.399659:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512602624 00000100:00100000:3.0:1789760731.399669:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512602624 00000100:00100000:3.0:1789760731.399811:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 710855149939 time 710920298735 state 0x0 ref 1 00000100:00100000:3.0:1789760731.399830:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 710920559151 time 710920298735 state 0x1 ref 1 00000100:00100000:3.0:1789760731.399933:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 710920559151 time 710964232513 state 0x0 ref 1 00000100:00100000:3.0:1789760731.399948:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2879f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512602624:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760731.399997:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23769088, mtime: 0x6aad94da, atime: 0x0, ctime: 0x6aad94da, blocks: 0xb807 00010000:00010000:3.0:1789760731.400007:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2dfd7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.400024:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=710s v=(0 0) 00010000:00010000:3.0:1789760731.400029:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760731.400036:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2dfd7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23769088->23773183] (req 23769088->23773183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.400050:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2dfd7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23769088->23773183] (req 23769088->23773183) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760731.429681:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973122 is committed 00002000:00010000:3.0:1789760731.429790:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23769088 -> 23773184 00010000:00010000:3.0:1789760731.429805:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2dfd7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23769088->23773183] (req 23769088->23773183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.429819:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2dfd7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23769088->23773183] (req 23769088->23773183) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.429827:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2dfd7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23769088->23773183] (req 23769088->23773183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.429906:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760731.429912:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2dfd7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23769088->23773183] (req 23769088->23773183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.429922:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2dfd7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23769088->23773183] (req 23769088->23773183) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.429942:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2dfd7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23769088->23773183] (req 23769088->23773183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.429949:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2dfd7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23769088->23773183] (req 23769088->23773183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760731.430075:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2879f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512602624:12345-192.168.203.56@tcp:4:dd.0 Request processed in 30125us (30505us total) trans 4294973122 rc 0/0 00000100:00100000:3.0:1789760731.430149:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5833 00000100:00100000:3.0:1789760731.430159:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 710920559151 time 710964232513 state 0x2 ref 0 00000100:00100000:0.0:1789760731.435308:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760731.435401:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512602752 00000100:00100000:2.0:1789760731.435421:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512602752 00000100:00100000:2.0:1789760731.435472:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 710920559151 time 710964232513 state 0x0 ref 1 00000100:00100000:2.0:1789760731.435487:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 710964492929 time 710964232513 state 0x1 ref 1 00000100:00100000:2.0:1789760731.435550:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 710964492929 time 710999848995 state 0x0 ref 1 00000100:00100000:2.0:1789760731.435563:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2878380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512602752:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760731.435612:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23773184, mtime: 0x6aad94da, atime: 0x0, ctime: 0x6aad94da, blocks: 0xb807 00010000:00010000:2.0:1789760731.435621:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2dfde lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.435636:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=710s v=(0 0) 00010000:00010000:2.0:1789760731.435641:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760731.435648:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2dfde lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23773184->23777279] (req 23773184->23777279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.435661:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2dfde lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23773184->23777279] (req 23773184->23777279) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760731.472157:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973123 is committed 00002000:00010000:0.0:1789760731.472337:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23773184 -> 23777280 00010000:00010000:0.0:1789760731.472351:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2dfde lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23773184->23777279] (req 23773184->23777279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.472364:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2dfde lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23773184->23777279] (req 23773184->23777279) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.472373:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2dfde lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23773184->23777279] (req 23773184->23777279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.472381:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760731.472386:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2dfde lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23773184->23777279] (req 23773184->23777279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.472395:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2dfde lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23773184->23777279] (req 23773184->23777279) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.472426:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2dfde lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23773184->23777279] (req 23773184->23777279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.472435:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2dfde lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23773184->23777279] (req 23773184->23777279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760731.472523:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2878380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512602752:12345-192.168.203.56@tcp:4:dd.0 Request processed in 36958us (37220us total) trans 4294973123 rc 0/0 00000100:00100000:0.0:1789760731.472533:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5834 00000100:00100000:0.0:1789760731.472543:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 710964492929 time 710999848995 state 0x2 ref 0 00000100:00100000:2.0:1789760731.482351:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760731.482609:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512602880 00000100:00100000:1.0:1789760731.482620:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512602880 00000100:00100000:1.0:1789760731.482678:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 710964492929 time 710999848995 state 0x0 ref 1 00000100:00100000:1.0:1789760731.482719:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 711000109411 time 710999848995 state 0x1 ref 1 00000100:00100000:1.0:1789760731.482787:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 711000109411 time 711047086058 state 0x0 ref 1 00000100:00100000:1.0:1789760731.482801:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2fb9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512602880:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760731.482869:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23777280, mtime: 0x6aad94da, atime: 0x0, ctime: 0x6aad94da, blocks: 0xb807 00010000:00010000:1.0:1789760731.482878:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2dfe5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.482894:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=711s v=(0 0) 00010000:00010000:1.0:1789760731.482899:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760731.482906:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2dfe5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23777280->23781375] (req 23777280->23781375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.482930:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2dfe5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23777280->23781375] (req 23777280->23781375) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.506813:0:7647:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:1.0:1789760731.524754:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23777280 -> 23781376 00010000:00010000:1.0:1789760731.524775:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2dfe5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23777280->23781375] (req 23777280->23781375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.524789:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2dfe5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23777280->23781375] (req 23777280->23781375) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.524799:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2dfe5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23777280->23781375] (req 23777280->23781375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.524807:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760731.524812:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2dfe5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23777280->23781375] (req 23777280->23781375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.524821:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2dfe5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23777280->23781375] (req 23777280->23781375) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.524840:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2dfe5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23777280->23781375] (req 23777280->23781375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.524849:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2dfe5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23777280->23781375] (req 23777280->23781375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760731.524937:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2fb9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2360:x1876699512602880:12345-192.168.203.56@tcp:4:dd.0 Request processed in 42133us (42591us total) trans 4294973124 rc 0/0 00000100:00100000:1.0:1789760731.524947:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5835 00000100:00100000:1.0:1789760731.524958:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 711000109411 time 711047086058 state 0x2 ref 0 00000001:00080000:2.0:1789760731.525368:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973124 is committed 00000100:00100000:1.0:1789760731.527746:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760731.527823:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512603008 00000100:00100000:3.0:1789760731.527833:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512603008 00000100:00100000:3.0:1789760731.527885:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 711000109411 time 711047086058 state 0x0 ref 1 00000100:00100000:3.0:1789760731.527901:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 711047346474 time 711047086058 state 0x1 ref 1 00000100:00100000:3.0:1789760731.527984:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 711047346474 time 711092283205 state 0x0 ref 1 00000100:00100000:3.0:1789760731.528001:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4131180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512603008:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760731.528047:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23781376, mtime: 0x6aad94da, atime: 0x0, ctime: 0x6aad94da, blocks: 0xb807 00010000:00010000:3.0:1789760731.528057:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2dfec lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.528071:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=711s v=(0 0) 00010000:00010000:3.0:1789760731.528075:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760731.528091:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2dfec lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23781376->23785471] (req 23781376->23785471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.528106:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2dfec lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23781376->23785471] (req 23781376->23785471) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760731.560262:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973125 is committed 00002000:00010000:3.0:1789760731.560433:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23781376 -> 23785472 00002000:00010000:3.0:1789760731.560438:0:6710:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760730 -> 1789760731 00002000:00010000:3.0:1789760731.560441:0:6710:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760730 -> 1789760731 00010000:00010000:3.0:1789760731.560455:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2dfec lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23781376->23785471] (req 23781376->23785471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.560469:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2dfec lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23781376->23785471] (req 23781376->23785471) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.560479:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2dfec lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23781376->23785471] (req 23781376->23785471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.560487:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760731.560492:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2dfec lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23781376->23785471] (req 23781376->23785471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.560501:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2dfec lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23781376->23785471] (req 23781376->23785471) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.560520:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2dfec lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23781376->23785471] (req 23781376->23785471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.560528:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2dfec lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23781376->23785471] (req 23781376->23785471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760731.560617:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4131180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512603008:12345-192.168.203.56@tcp:4:dd.0 Request processed in 32614us (32874us total) trans 4294973125 rc 0/0 00000100:00100000:3.0:1789760731.560628:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5836 00000100:00100000:3.0:1789760731.560637:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 711047346474 time 711092283205 state 0x2 ref 0 00000100:00100000:0.0:1789760731.564060:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760731.564340:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512603136 00000100:00100000:2.0:1789760731.564350:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512603136 00000100:00100000:2.0:1789760731.564401:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 711047346474 time 711092283205 state 0x0 ref 1 00000100:00100000:2.0:1789760731.564428:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 711092543621 time 711092283205 state 0x1 ref 1 00000100:00100000:2.0:1789760731.564461:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 711092543621 time 711128760819 state 0x0 ref 1 00000100:00100000:2.0:1789760731.564474:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac287a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512603136:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760731.564522:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23785472, mtime: 0x6aad94db, atime: 0x0, ctime: 0x6aad94db, blocks: 0xb807 00010000:00010000:2.0:1789760731.564531:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2dff3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.564547:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=711s v=(0 0) 00010000:00010000:2.0:1789760731.564552:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760731.564558:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2dff3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23785472->23789567] (req 23785472->23789567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.564572:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2dff3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23785472->23789567] (req 23785472->23789567) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760731.572664:0:3309:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8feac4131500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3309:1876699540222464:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1789760731.572758:0:3309:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760731.572910:0:6706:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540222464 00000100:00100000:0.0:1789760731.572917:0:6706:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540222464 00000100:00100000:0.0:1789760731.572961:0:6706:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 193 00000100:00100000:0.0:1789760731.572969:0:6706:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac287bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3309:x1876699540222464:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1789760731.573004:0:6706:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3859808256 avail 3857711104 00000100:00100000:0.0:1789760731.573048:0:6706:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac287bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3309:x1876699540222464:12345-0@lo:13:osp-pre-1-0.0 Request processed in 79us (289us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760731.573055:0:6706:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 193 00000100:00100000:0.0:1789760731.574192:0:3309:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8feac4131500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3309:1876699540222464:0@lo:13:osp-pre-1-0.0 00000001:00080000:3.0:1789760731.602838:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973126 is committed 00002000:00010000:2.0:1789760731.603138:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23785472 -> 23789568 00010000:00010000:2.0:1789760731.603154:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2dff3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23785472->23789567] (req 23785472->23789567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.603168:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2dff3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23785472->23789567] (req 23785472->23789567) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.603178:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2dff3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23785472->23789567] (req 23785472->23789567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.603186:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760731.603191:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2dff3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23785472->23789567] (req 23785472->23789567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.603201:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2dff3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23785472->23789567] (req 23785472->23789567) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.603220:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2dff3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23785472->23789567] (req 23785472->23789567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.603227:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2dff3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23785472->23789567] (req 23785472->23789567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760731.603455:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac287a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512603136:12345-192.168.203.56@tcp:4:dd.0 Request processed in 38977us (39399us total) trans 4294973126 rc 0/0 00000100:00100000:2.0:1789760731.603465:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5837 00000100:00100000:2.0:1789760731.603474:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 711092543621 time 711128760819 state 0x2 ref 0 00000100:00100000:2.0:1789760731.606504:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760731.606599:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512603264 00000100:00100000:1.0:1789760731.606609:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512603264 00000100:00100000:1.0:1789760731.606659:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 711092543621 time 711128760819 state 0x0 ref 1 00000100:00100000:1.0:1789760731.606677:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 711129021235 time 711128760819 state 0x1 ref 1 00000100:00100000:1.0:1789760731.606733:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 711129021235 time 711171032294 state 0x0 ref 1 00000100:00100000:1.0:1789760731.606745:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2fbaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512603264:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760731.606790:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23789568, mtime: 0x6aad94db, atime: 0x0, ctime: 0x6aad94db, blocks: 0xb807 00010000:00010000:1.0:1789760731.606799:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2dffa lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.606815:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=711s v=(0 0) 00010000:00010000:1.0:1789760731.606820:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760731.606826:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2dffa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23789568->23793663] (req 23789568->23793663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.606839:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2dffa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23789568->23793663] (req 23789568->23793663) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760731.640734:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973127 is committed 00002000:00010000:1.0:1789760731.640899:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23789568 -> 23793664 00010000:00010000:1.0:1789760731.640914:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2dffa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23789568->23793663] (req 23789568->23793663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.640927:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2dffa lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23789568->23793663] (req 23789568->23793663) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.640936:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2dffa lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23789568->23793663] (req 23789568->23793663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.640949:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760731.640954:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2dffa lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23789568->23793663] (req 23789568->23793663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.640962:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2dffa lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23789568->23793663] (req 23789568->23793663) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.640983:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2dffa lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23789568->23793663] (req 23789568->23793663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.640990:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2dffa lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23789568->23793663] (req 23789568->23793663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760731.641146:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2fbaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512603264:12345-192.168.203.56@tcp:4:dd.0 Request processed in 34333us (34582us total) trans 4294973127 rc 0/0 00000100:00100000:1.0:1789760731.641157:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5838 00000100:00100000:1.0:1789760731.641166:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 711129021235 time 711171032294 state 0x2 ref 0 00000100:00100000:1.0:1789760731.646467:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760731.646597:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512603392 00000100:00100000:3.0:1789760731.646607:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512603392 00000100:00100000:3.0:1789760731.646665:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 711129021235 time 711171032294 state 0x0 ref 1 00000100:00100000:3.0:1789760731.646680:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 711171292710 time 711171032294 state 0x1 ref 1 00000100:00100000:3.0:1789760731.646794:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 711171292710 time 711211093214 state 0x0 ref 1 00000100:00100000:3.0:1789760731.646817:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4130a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512603392:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760731.646868:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23793664, mtime: 0x6aad94db, atime: 0x0, ctime: 0x6aad94db, blocks: 0xb807 00010000:00010000:3.0:1789760731.646877:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2e001 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.647201:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=711s v=(0 0) 00010000:00010000:3.0:1789760731.647206:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760731.647213:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2e001 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23793664->23797759] (req 23793664->23797759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.647239:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2e001 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23793664->23797759] (req 23793664->23797759) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760731.687587:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973128 is committed 00002000:00010000:0.0:1789760731.687690:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23793664 -> 23797760 00010000:00010000:0.0:1789760731.687725:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2e001 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23793664->23797759] (req 23793664->23797759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.687738:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2e001 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23793664->23797759] (req 23793664->23797759) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.687747:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2e001 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23793664->23797759] (req 23793664->23797759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.687754:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760731.687759:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2e001 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23793664->23797759] (req 23793664->23797759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.687769:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2e001 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23793664->23797759] (req 23793664->23797759) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.687789:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2e001 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23793664->23797759] (req 23793664->23797759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.687795:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2e001 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23793664->23797759] (req 23793664->23797759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760731.688067:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4130a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512603392:12345-192.168.203.56@tcp:4:dd.0 Request processed in 41247us (41604us total) trans 4294973128 rc 0/0 00000100:00100000:0.0:1789760731.688087:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5839 00000100:00100000:0.0:1789760731.688097:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 711171292710 time 711211093214 state 0x2 ref 0 00000100:00100000:0.0:1789760731.695899:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760731.696350:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512603520 00000100:00100000:2.0:1789760731.696360:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512603520 00000100:00100000:2.0:1789760731.696428:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 711171292710 time 711211093214 state 0x0 ref 1 00000100:00100000:2.0:1789760731.696443:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 711211353630 time 711211093214 state 0x1 ref 1 00000100:00100000:2.0:1789760731.696476:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 711211353630 time 711260775830 state 0x0 ref 1 00000100:00100000:2.0:1789760731.696489:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febea4b5c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512603520:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760731.696541:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23797760, mtime: 0x6aad94db, atime: 0x0, ctime: 0x6aad94db, blocks: 0xb807 00010000:00010000:2.0:1789760731.696550:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2e008 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.696567:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=711s v=(0 0) 00010000:00010000:2.0:1789760731.696571:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760731.696577:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2e008 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23797760->23801855] (req 23797760->23801855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.696590:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2e008 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23797760->23801855] (req 23797760->23801855) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760731.729765:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23797760 -> 23801856 00010000:00010000:2.0:1789760731.729788:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2e008 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23797760->23801855] (req 23797760->23801855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.729812:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2e008 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23797760->23801855] (req 23797760->23801855) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.729821:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2e008 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23797760->23801855] (req 23797760->23801855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.729828:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760731.729833:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2e008 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23797760->23801855] (req 23797760->23801855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.729842:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2e008 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23797760->23801855] (req 23797760->23801855) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.729861:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2e008 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23797760->23801855] (req 23797760->23801855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.729869:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2e008 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23797760->23801855] (req 23797760->23801855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760731.729959:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febea4b5c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2361:x1876699512603520:12345-192.168.203.56@tcp:4:dd.0 Request processed in 33468us (34077us total) trans 4294973129 rc 0/0 00000100:00100000:2.0:1789760731.729969:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5840 00000100:00100000:2.0:1789760731.729978:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 711211353630 time 711260775830 state 0x2 ref 0 00000001:00080000:1.0:1789760731.732453:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973129 is committed 00000100:00100000:1.0:1789760731.734049:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760731.734228:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512603648 00000100:00100000:3.0:1789760731.734238:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512603648 00000100:00100000:3.0:1789760731.734295:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 711211353630 time 711260775830 state 0x0 ref 1 00000100:00100000:3.0:1789760731.734311:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 711261036246 time 711260775830 state 0x1 ref 1 00000100:00100000:3.0:1789760731.734343:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 711261036246 time 711298642856 state 0x0 ref 1 00000100:00100000:3.0:1789760731.734355:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4132d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512603648:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760731.734402:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23801856, mtime: 0x6aad94db, atime: 0x0, ctime: 0x6aad94db, blocks: 0xb807 00010000:00010000:3.0:1789760731.734423:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2e00f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.734440:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=711s v=(0 0) 00010000:00010000:3.0:1789760731.734445:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760731.734451:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2e00f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23801856->23805951] (req 23801856->23805951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.734465:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2e00f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23801856->23805951] (req 23801856->23805951) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760731.774674:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973130 is committed 00002000:00010000:3.0:1789760731.776510:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23801856 -> 23805952 00010000:00010000:3.0:1789760731.776528:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2e00f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23801856->23805951] (req 23801856->23805951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.776603:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2e00f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23801856->23805951] (req 23801856->23805951) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.776612:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2e00f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23801856->23805951] (req 23801856->23805951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.776620:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760731.776625:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2e00f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23801856->23805951] (req 23801856->23805951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.776633:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2e00f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23801856->23805951] (req 23801856->23805951) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.779629:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2e00f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23801856->23805951] (req 23801856->23805951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.779659:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2e00f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23801856->23805951] (req 23801856->23805951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760731.779873:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4132d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512603648:12345-192.168.203.56@tcp:4:dd.0 Request processed in 45514us (45829us total) trans 4294973130 rc 0/0 00000100:00100000:3.0:1789760731.779883:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5841 00000100:00100000:3.0:1789760731.779893:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 711261036246 time 711298642856 state 0x2 ref 0 00000100:00100000:2.0:1789760731.783050:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760731.783159:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512603776 00000100:00100000:0.0:1789760731.783169:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512603776 00000100:00100000:0.0:1789760731.783224:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 711261036246 time 711298642856 state 0x0 ref 1 00000100:00100000:0.0:1789760731.783252:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 711298903272 time 711298642856 state 0x1 ref 1 00000100:00100000:0.0:1789760731.783283:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 711298903272 time 711347583300 state 0x0 ref 1 00000100:00100000:0.0:1789760731.783295:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2fb9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512603776:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760731.783346:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23805952, mtime: 0x6aad94db, atime: 0x0, ctime: 0x6aad94db, blocks: 0xb807 00010000:00010000:0.0:1789760731.783354:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2e016 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.783370:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=711s v=(0 0) 00010000:00010000:0.0:1789760731.783375:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760731.783381:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2e016 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23805952->23810047] (req 23805952->23810047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.783396:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2e016 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23805952->23810047] (req 23805952->23810047) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760731.820557:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23805952 -> 23810048 00010000:00010000:0.0:1789760731.820581:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2e016 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23805952->23810047] (req 23805952->23810047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.820596:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2e016 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23805952->23810047] (req 23805952->23810047) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.820606:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2e016 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23805952->23810047] (req 23805952->23810047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.820614:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760731.820621:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2e016 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23805952->23810047] (req 23805952->23810047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.820632:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2e016 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23805952->23810047] (req 23805952->23810047) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.820655:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2e016 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23805952->23810047] (req 23805952->23810047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.820664:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2e016 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23805952->23810047] (req 23805952->23810047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760731.821786:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2fb9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2361:x1876699512603776:12345-192.168.203.56@tcp:4:dd.0 Request processed in 38484us (38736us total) trans 4294973131 rc 0/0 00000100:00100000:0.0:1789760731.821812:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5842 00000100:00100000:0.0:1789760731.821822:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 711298903272 time 711347583300 state 0x2 ref 0 00000001:00080000:3.0:1789760731.822144:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973131 is committed 00000100:00100000:0.0:1789760731.825349:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760731.825440:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512603904 00000100:00100000:2.0:1789760731.825451:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512603904 00000100:00100000:2.0:1789760731.825503:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 711298903272 time 711347583300 state 0x0 ref 1 00000100:00100000:2.0:1789760731.825519:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 711347843716 time 711347583300 state 0x1 ref 1 00000100:00100000:2.0:1789760731.825572:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 711347843716 time 711389870865 state 0x0 ref 1 00000100:00100000:2.0:1789760731.825584:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac287b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512603904:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760731.825632:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23810048, mtime: 0x6aad94db, atime: 0x0, ctime: 0x6aad94db, blocks: 0xb807 00010000:00010000:2.0:1789760731.825642:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2e01d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.825665:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=711s v=(0 0) 00010000:00010000:2.0:1789760731.825670:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760731.825676:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2e01d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23810048->23814143] (req 23810048->23814143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.825692:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2e01d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23810048->23814143] (req 23810048->23814143) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760731.857692:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973132 is committed 00002000:00010000:3.0:1789760731.857875:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23810048 -> 23814144 00010000:00010000:3.0:1789760731.857890:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2e01d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23810048->23814143] (req 23810048->23814143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.857904:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2e01d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23810048->23814143] (req 23810048->23814143) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.857914:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2e01d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23810048->23814143] (req 23810048->23814143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.857922:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760731.857928:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2e01d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23810048->23814143] (req 23810048->23814143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.857937:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2e01d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23810048->23814143] (req 23810048->23814143) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.857963:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2e01d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23810048->23814143] (req 23810048->23814143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.857971:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2e01d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23810048->23814143] (req 23810048->23814143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760731.858110:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac287b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512603904:12345-192.168.203.56@tcp:4:dd.0 Request processed in 32523us (32767us total) trans 4294973132 rc 0/0 00000100:00100000:3.0:1789760731.858122:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5843 00000100:00100000:3.0:1789760731.858132:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 711347843716 time 711389870865 state 0x2 ref 0 00000100:00100000:2.0:1789760731.862874:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760731.862952:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512604032 00000100:00100000:3.0:1789760731.862961:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512604032 00000100:00100000:3.0:1789760731.863014:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 711347843716 time 711389870865 state 0x0 ref 1 00000100:00100000:3.0:1789760731.863029:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 711390131281 time 711389870865 state 0x1 ref 1 00000100:00100000:3.0:1789760731.863060:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 711390131281 time 711427359974 state 0x0 ref 1 00000100:00100000:3.0:1789760731.863073:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2fb8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512604032:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760731.863122:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23814144, mtime: 0x6aad94db, atime: 0x0, ctime: 0x6aad94db, blocks: 0xb807 00010000:00010000:3.0:1789760731.863131:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2e024 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.863147:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=711s v=(0 0) 00010000:00010000:3.0:1789760731.863153:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760731.863159:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2e024 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23814144->23818239] (req 23814144->23818239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.863172:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2e024 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23814144->23818239] (req 23814144->23818239) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760731.936646:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973133 is committed 00002000:00010000:1.0:1789760731.936861:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23814144 -> 23818240 00010000:00010000:1.0:1789760731.936877:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2e024 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23814144->23818239] (req 23814144->23818239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.936896:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2e024 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23814144->23818239] (req 23814144->23818239) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.936905:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2e024 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23814144->23818239] (req 23814144->23818239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.936912:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760731.937145:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2e024 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23814144->23818239] (req 23814144->23818239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.937158:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2e024 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23814144->23818239] (req 23814144->23818239) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.937178:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2e024 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23814144->23818239] (req 23814144->23818239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.937185:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2e024 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23814144->23818239] (req 23814144->23818239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760731.937343:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2fb8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512604032:12345-192.168.203.56@tcp:4:dd.0 Request processed in 74267us (74474us total) trans 4294973133 rc 0/0 00000100:00100000:1.0:1789760731.937355:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5844 00000100:00100000:1.0:1789760731.937364:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 711390131281 time 711427359974 state 0x2 ref 0 00000100:00100000:2.0:1789760731.946153:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760731.946235:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512604160 00000100:00100000:0.0:1789760731.946247:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512604160 00000100:00100000:0.0:1789760731.946308:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 711390131281 time 711427359974 state 0x0 ref 1 00000100:00100000:0.0:1789760731.946323:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 711427620390 time 711427359974 state 0x1 ref 1 00000100:00100000:0.0:1789760731.946362:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 711427620390 time 711510661856 state 0x0 ref 1 00000100:00100000:0.0:1789760731.946375:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2fba680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512604160:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760731.946436:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23818240, mtime: 0x6aad94db, atime: 0x0, ctime: 0x6aad94db, blocks: 0xb807 00010000:00010000:0.0:1789760731.946549:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2e02b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.946568:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=711s v=(0 0) 00010000:00010000:0.0:1789760731.946573:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760731.946579:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2e02b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23818240->23822335] (req 23818240->23822335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.946593:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2e02b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23818240->23822335] (req 23818240->23822335) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760731.989003:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973134 is committed 00002000:00010000:3.0:1789760731.989154:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23818240 -> 23822336 00010000:00010000:3.0:1789760731.989171:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2e02b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23818240->23822335] (req 23818240->23822335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.989185:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2e02b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23818240->23822335] (req 23818240->23822335) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.989194:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2e02b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23818240->23822335] (req 23818240->23822335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.989202:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760731.989207:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2e02b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23818240->23822335] (req 23818240->23822335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.989216:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2e02b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23818240->23822335] (req 23818240->23822335) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.989239:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2e02b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23818240->23822335] (req 23818240->23822335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.989252:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2e02b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23818240->23822335] (req 23818240->23822335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760731.989381:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2fba680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512604160:12345-192.168.203.56@tcp:4:dd.0 Request processed in 43002us (43232us total) trans 4294973134 rc 0/0 00000100:00100000:3.0:1789760731.989392:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5845 00000100:00100000:3.0:1789760731.989402:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 711427620390 time 711510661856 state 0x2 ref 0 00000100:00100000:3.0:1789760731.994300:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760731.994416:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512604288 00000100:00100000:2.0:1789760731.994426:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512604288 00000100:00100000:2.0:1789760731.994478:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 711427620390 time 711510661856 state 0x0 ref 1 00000100:00100000:2.0:1789760731.994493:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 711510922272 time 711510661856 state 0x1 ref 1 00000100:00100000:2.0:1789760731.994582:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 711510922272 time 711558881281 state 0x0 ref 1 00000100:00100000:2.0:1789760731.994595:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac531ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512604288:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760731.994644:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23822336, mtime: 0x6aad94db, atime: 0x0, ctime: 0x6aad94db, blocks: 0xb807 00010000:00010000:2.0:1789760731.994653:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2e032 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.994669:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=711s v=(0 0) 00010000:00010000:2.0:1789760731.994673:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760731.994680:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2e032 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23822336->23826431] (req 23822336->23826431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.994694:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2e032 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23822336->23826431] (req 23822336->23826431) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760732.036476:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973135 is committed 00002000:00010000:3.0:1789760732.036640:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23822336 -> 23826432 00010000:00010000:3.0:1789760732.036656:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2e032 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23822336->23826431] (req 23822336->23826431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.036671:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2e032 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23822336->23826431] (req 23822336->23826431) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.036680:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2e032 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23822336->23826431] (req 23822336->23826431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.036688:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760732.036693:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2e032 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23822336->23826431] (req 23822336->23826431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.036929:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2e032 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23822336->23826431] (req 23822336->23826431) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.036957:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2e032 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23822336->23826431] (req 23822336->23826431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.036964:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2e032 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23822336->23826431] (req 23822336->23826431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760732.037060:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac531ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512604288:12345-192.168.203.56@tcp:4:dd.0 Request processed in 42463us (42766us total) trans 4294973135 rc 0/0 00000100:00100000:3.0:1789760732.037070:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5846 00000100:00100000:3.0:1789760732.037080:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 711510922272 time 711558881281 state 0x2 ref 0 00000100:00100000:2.0:1789760732.042292:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760732.042387:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512604416 00000100:00100000:1.0:1789760732.042397:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512604416 00000100:00100000:1.0:1789760732.042459:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 711510922272 time 711558881281 state 0x0 ref 1 00000100:00100000:1.0:1789760732.042967:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 711559141697 time 711558881281 state 0x1 ref 1 00000100:00100000:1.0:1789760732.043000:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 711559141697 time 711607299346 state 0x0 ref 1 00000100:00100000:1.0:1789760732.043015:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2fb8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512604416:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760732.043065:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23826432, mtime: 0x6aad94db, atime: 0x0, ctime: 0x6aad94db, blocks: 0xb807 00010000:00010000:1.0:1789760732.043075:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2e039 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.043091:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=711s v=(0 0) 00010000:00010000:1.0:1789760732.043096:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760732.043103:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2e039 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23826432->23830527] (req 23826432->23830527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.043117:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2e039 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23826432->23830527] (req 23826432->23830527) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760732.082056:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23826432 -> 23830528 00010000:00010000:2.0:1789760732.082076:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2e039 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23826432->23830527] (req 23826432->23830527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.082090:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2e039 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23826432->23830527] (req 23826432->23830527) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.082099:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2e039 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23826432->23830527] (req 23826432->23830527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.082106:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760732.082111:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2e039 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23826432->23830527] (req 23826432->23830527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.082119:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2e039 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23826432->23830527] (req 23826432->23830527) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.082139:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2e039 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23826432->23830527] (req 23826432->23830527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.082151:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2e039 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23826432->23830527] (req 23826432->23830527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760732.082246:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2fb8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2360:x1876699512604416:12345-192.168.203.56@tcp:4:dd.0 Request processed in 39230us (39959us total) trans 4294973136 rc 0/0 00000100:00100000:2.0:1789760732.082256:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5847 00000100:00100000:2.0:1789760732.082267:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 711559141697 time 711607299346 state 0x2 ref 0 00000001:00080000:0.0:1789760732.082633:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973136 is committed 00000100:00100000:1.0:1789760732.084922:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760732.085072:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512604544 00000100:00100000:3.0:1789760732.085081:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512604544 00000100:00100000:3.0:1789760732.085142:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 711559141697 time 711607299346 state 0x0 ref 1 00000100:00100000:3.0:1789760732.085158:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 711607559762 time 711607299346 state 0x1 ref 1 00000100:00100000:3.0:1789760732.085190:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 711607559762 time 711649490247 state 0x0 ref 1 00000100:00100000:3.0:1789760732.085203:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4132300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512604544:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760732.085251:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23830528, mtime: 0x6aad94db, atime: 0x0, ctime: 0x6aad94db, blocks: 0xb807 00010000:00010000:3.0:1789760732.085260:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2e040 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.085276:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=711s v=(0 0) 00010000:00010000:3.0:1789760732.085281:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760732.085288:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2e040 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23830528->23834623] (req 23830528->23834623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.085301:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2e040 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23830528->23834623] (req 23830528->23834623) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760732.139586:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973137 is committed 00002000:00010000:0.0:1789760732.139656:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23830528 -> 23834624 00010000:00010000:0.0:1789760732.139673:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2e040 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23830528->23834623] (req 23830528->23834623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.139686:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2e040 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23830528->23834623] (req 23830528->23834623) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.139713:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2e040 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23830528->23834623] (req 23830528->23834623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.139722:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760732.139728:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2e040 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23830528->23834623] (req 23830528->23834623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.139737:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2e040 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23830528->23834623] (req 23830528->23834623) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.139757:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2e040 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23830528->23834623] (req 23830528->23834623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.139766:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2e040 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23830528->23834623] (req 23830528->23834623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760732.139865:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4132300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512604544:12345-192.168.203.56@tcp:4:dd.0 Request processed in 54659us (54948us total) trans 4294973137 rc 0/0 00000100:00100000:0.0:1789760732.139881:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5848 00000100:00100000:0.0:1789760732.139891:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 711607559762 time 711649490247 state 0x2 ref 0 00000100:00100000:3.0:1789760732.144649:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760732.144784:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512604672 00000100:00100000:2.0:1789760732.144792:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512604672 00000100:00100000:2.0:1789760732.144842:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 711607559762 time 711649490247 state 0x0 ref 1 00000100:00100000:2.0:1789760732.144932:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 711649750663 time 711649490247 state 0x1 ref 1 00000100:00100000:2.0:1789760732.144965:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 711649750663 time 711709264684 state 0x0 ref 1 00000100:00100000:2.0:1789760732.144979:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac531e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512604672:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760732.145027:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23834624, mtime: 0x6aad94db, atime: 0x0, ctime: 0x6aad94db, blocks: 0xb807 00010000:00010000:2.0:1789760732.145036:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2e047 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.145052:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=711s v=(0 0) 00010000:00010000:2.0:1789760732.145056:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760732.145062:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2e047 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23834624->23838719] (req 23834624->23838719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.145080:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2e047 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23834624->23838719] (req 23834624->23838719) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760732.262348:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973138 is committed 00002000:00010000:0.0:1789760732.262499:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23834624 -> 23838720 00010000:00010000:0.0:1789760732.262514:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2e047 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23834624->23838719] (req 23834624->23838719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.262527:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2e047 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23834624->23838719] (req 23834624->23838719) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.262536:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2e047 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23834624->23838719] (req 23834624->23838719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.262543:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760732.262548:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2e047 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23834624->23838719] (req 23834624->23838719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.262557:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2e047 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23834624->23838719] (req 23834624->23838719) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.262577:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2e047 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23834624->23838719] (req 23834624->23838719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.262585:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2e047 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23834624->23838719] (req 23834624->23838719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760732.262685:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac531e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512604672:12345-192.168.203.56@tcp:4:dd.0 Request processed in 117703us (118040us total) trans 4294973138 rc 0/0 00000100:00100000:0.0:1789760732.262726:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5849 00000100:00100000:0.0:1789760732.262737:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 711649750663 time 711709264684 state 0x2 ref 0 00000100:00100000:2.0:1789760732.264895:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760732.265044:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512604800 00000100:00100000:1.0:1789760732.265055:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512604800 00000100:00100000:1.0:1789760732.265121:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 711649750663 time 711709264684 state 0x0 ref 1 00000100:00100000:1.0:1789760732.265138:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 711709525100 time 711709264684 state 0x1 ref 1 00000100:00100000:1.0:1789760732.265170:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 711709525100 time 711829469737 state 0x0 ref 1 00000100:00100000:1.0:1789760732.265183:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2fbb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512604800:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760732.265240:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23838720, mtime: 0x6aad94db, atime: 0x0, ctime: 0x6aad94db, blocks: 0xb807 00010000:00010000:1.0:1789760732.265251:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2e04e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.265267:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=711s v=(0 0) 00010000:00010000:1.0:1789760732.265272:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760732.265279:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2e04e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23838720->23842815] (req 23838720->23842815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.265292:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2e04e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23838720->23842815] (req 23838720->23842815) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760732.327734:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973139 is committed 00002000:00010000:0.0:1789760732.328001:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23838720 -> 23842816 00010000:00010000:0.0:1789760732.328018:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2e04e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23838720->23842815] (req 23838720->23842815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.328031:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2e04e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23838720->23842815] (req 23838720->23842815) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.328041:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2e04e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23838720->23842815] (req 23838720->23842815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.328048:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760732.328053:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2e04e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23838720->23842815] (req 23838720->23842815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.328062:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2e04e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23838720->23842815] (req 23838720->23842815) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.328089:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2e04e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23838720->23842815] (req 23838720->23842815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.328103:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2e04e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23838720->23842815] (req 23838720->23842815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760732.328198:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2fbb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512604800:12345-192.168.203.56@tcp:4:dd.0 Request processed in 63013us (63309us total) trans 4294973139 rc 0/0 00000100:00100000:0.0:1789760732.328208:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5850 00000100:00100000:0.0:1789760732.328218:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 711709525100 time 711829469737 state 0x2 ref 0 00000100:00100000:3.0:1789760732.331436:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760732.334528:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512604928 00000100:00100000:0.0:1789760732.334544:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512604928 00000100:00100000:0.0:1789760732.334607:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 711709525100 time 711829469737 state 0x0 ref 1 00000100:00100000:0.0:1789760732.334621:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 711829730153 time 711829469737 state 0x1 ref 1 00000100:00100000:0.0:1789760732.334657:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 711829730153 time 711898956363 state 0x0 ref 1 00000100:00100000:0.0:1789760732.334671:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac531dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512604928:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760732.334735:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23842816, mtime: 0x6aad94db, atime: 0x0, ctime: 0x6aad94db, blocks: 0xb807 00010000:00010000:0.0:1789760732.334744:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2e055 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.334759:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=711s v=(0 0) 00010000:00010000:0.0:1789760732.334762:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760732.334769:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2e055 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23842816->23846911] (req 23842816->23846911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.334789:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2e055 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23842816->23846911] (req 23842816->23846911) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760732.403879:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973140 is committed 00002000:00010000:0.0:1789760732.404304:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23842816 -> 23846912 00010000:00010000:0.0:1789760732.404319:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2e055 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23842816->23846911] (req 23842816->23846911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.404332:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2e055 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23842816->23846911] (req 23842816->23846911) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.404341:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2e055 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23842816->23846911] (req 23842816->23846911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.404349:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760732.404354:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2e055 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23842816->23846911] (req 23842816->23846911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.404364:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2e055 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23842816->23846911] (req 23842816->23846911) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.404382:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2e055 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23842816->23846911] (req 23842816->23846911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.404389:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2e055 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23842816->23846911] (req 23842816->23846911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760732.404497:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac531dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512604928:12345-192.168.203.56@tcp:4:dd.0 Request processed in 69824us (73066us total) trans 4294973140 rc 0/0 00000100:00100000:0.0:1789760732.404507:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5851 00000100:00100000:0.0:1789760732.404517:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 711829730153 time 711898956363 state 0x2 ref 0 00000100:00100000:3.0:1789760732.408741:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760732.409904:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512605056 00000100:00100000:0.0:1789760732.409917:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512605056 00000100:00100000:0.0:1789760732.409989:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 711829730153 time 711898956363 state 0x0 ref 1 00000100:00100000:0.0:1789760732.410005:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 711899216779 time 711898956363 state 0x1 ref 1 00000100:00100000:0.0:1789760732.410037:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 711899216779 time 711974337089 state 0x0 ref 1 00000100:00100000:0.0:1789760732.410050:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac531ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512605056:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760732.410098:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23846912, mtime: 0x6aad94db, atime: 0x0, ctime: 0x6aad94db, blocks: 0xb807 00010000:00010000:0.0:1789760732.410107:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2e05c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.410123:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=711s v=(0 0) 00010000:00010000:0.0:1789760732.410128:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760732.410135:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2e05c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23846912->23851007] (req 23846912->23851007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.410148:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2e05c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23846912->23851007] (req 23846912->23851007) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760732.456659:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23846912 -> 23851008 00010000:00010000:3.0:1789760732.456680:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2e05c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23846912->23851007] (req 23846912->23851007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.456694:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2e05c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23846912->23851007] (req 23846912->23851007) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.456733:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2e05c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23846912->23851007] (req 23846912->23851007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.456741:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760732.456746:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2e05c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23846912->23851007] (req 23846912->23851007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.456754:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2e05c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23846912->23851007] (req 23846912->23851007) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.456776:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2e05c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23846912->23851007] (req 23846912->23851007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.456783:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2e05c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23846912->23851007] (req 23846912->23851007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760732.456876:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac531ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2360:x1876699512605056:12345-192.168.203.56@tcp:4:dd.0 Request processed in 46823us (48140us total) trans 4294973141 rc 0/0 00000100:00100000:3.0:1789760732.456885:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5852 00000100:00100000:3.0:1789760732.456895:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 711899216779 time 711974337089 state 0x2 ref 0 00000001:00080000:3.0:1789760732.456932:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973141 is committed 00000100:00100000:2.0:1789760732.461675:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760732.462733:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512605184 00000100:00100000:0.0:1789760732.462746:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512605184 00000100:00100000:0.0:1789760732.462809:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 711899216779 time 711974337089 state 0x0 ref 1 00000100:00100000:0.0:1789760732.462825:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 711974597505 time 711974337089 state 0x1 ref 1 00000100:00100000:0.0:1789760732.462858:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 711974597505 time 712027156542 state 0x0 ref 1 00000100:00100000:0.0:1789760732.462871:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2fb8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512605184:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760732.463100:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23851008, mtime: 0x6aad94db, atime: 0x0, ctime: 0x6aad94db, blocks: 0xb807 00010000:00010000:1.0:1789760732.463111:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2e063 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.463126:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=712s v=(0 0) 00010000:00010000:1.0:1789760732.463141:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760732.463149:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2e063 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23851008->23855103] (req 23851008->23855103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.463163:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2e063 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23851008->23855103] (req 23851008->23855103) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760732.496656:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973142 is committed 00002000:00010000:3.0:1789760732.496874:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23851008 -> 23855104 00010000:00010000:3.0:1789760732.496890:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2e063 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23851008->23855103] (req 23851008->23855103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.496904:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2e063 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23851008->23855103] (req 23851008->23855103) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.496914:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2e063 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23851008->23855103] (req 23851008->23855103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.496921:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760732.496925:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2e063 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23851008->23855103] (req 23851008->23855103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.496940:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2e063 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23851008->23855103] (req 23851008->23855103) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.496961:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2e063 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23851008->23855103] (req 23851008->23855103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.496970:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2e063 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23851008->23855103] (req 23851008->23855103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760732.497086:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2fb8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512605184:12345-192.168.203.56@tcp:4:dd.0 Request processed in 34206us (35410us total) trans 4294973142 rc 0/0 00000100:00100000:3.0:1789760732.497095:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5853 00000100:00100000:3.0:1789760732.497104:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 711974597505 time 712027156542 state 0x2 ref 0 00000100:00100000:2.0:1789760732.500844:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760732.502942:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512605312 00000100:00100000:0.0:1789760732.502956:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512605312 00000100:00100000:1.0:1789760732.503299:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 711974597505 time 712027156542 state 0x0 ref 1 00000100:00100000:1.0:1789760732.503318:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 712027416958 time 712027156542 state 0x1 ref 1 00000100:00100000:1.0:1789760732.503394:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 712027416958 time 712067693049 state 0x0 ref 1 00000100:00100000:1.0:1789760732.503418:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2fb9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512605312:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760732.503536:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23855104, mtime: 0x6aad94db, atime: 0x0, ctime: 0x6aad94db, blocks: 0xb807 00010000:00010000:1.0:1789760732.503545:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2e06a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.503561:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=712s v=(0 0) 00010000:00010000:1.0:1789760732.503566:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760732.503572:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2e06a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23855104->23859199] (req 23855104->23859199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.503586:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2e06a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23855104->23859199] (req 23855104->23859199) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.531310:0:5836:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:2.0:1789760732.551665:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973143 is committed 00002000:00010000:1.0:1789760732.552385:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23855104 -> 23859200 00002000:00010000:1.0:1789760732.552391:0:6710:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760731 -> 1789760732 00002000:00010000:1.0:1789760732.552394:0:6710:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760731 -> 1789760732 00010000:00010000:1.0:1789760732.552418:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2e06a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23855104->23859199] (req 23855104->23859199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.552439:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2e06a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23855104->23859199] (req 23855104->23859199) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.552449:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2e06a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23855104->23859199] (req 23855104->23859199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.552456:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760732.552462:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2e06a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23855104->23859199] (req 23855104->23859199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.552472:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2e06a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23855104->23859199] (req 23855104->23859199) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.552556:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2e06a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23855104->23859199] (req 23855104->23859199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.552567:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2e06a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23855104->23859199] (req 23855104->23859199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760732.552664:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2fb9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512605312:12345-192.168.203.56@tcp:4:dd.0 Request processed in 49255us (51824us total) trans 4294973143 rc 0/0 00000100:00100000:1.0:1789760732.552674:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5854 00000100:00100000:1.0:1789760732.552683:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 712027416958 time 712067693049 state 0x2 ref 0 00000100:00100000:0.0:1789760732.555355:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760732.555459:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512605440 00000100:00100000:3.0:1789760732.555530:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512605440 00000100:00100000:3.0:1789760732.555588:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 712027416958 time 712067693049 state 0x0 ref 1 00000100:00100000:3.0:1789760732.555604:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 712067953465 time 712067693049 state 0x1 ref 1 00000100:00100000:3.0:1789760732.555638:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 712067953465 time 712119937409 state 0x0 ref 1 00000100:00100000:3.0:1789760732.555650:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2878000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512605440:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760732.555715:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23859200, mtime: 0x6aad94dc, atime: 0x0, ctime: 0x6aad94dc, blocks: 0xb807 00010000:00010000:3.0:1789760732.555723:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2e071 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.555739:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=712s v=(0 0) 00010000:00010000:3.0:1789760732.555742:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760732.555748:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2e071 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23859200->23863295] (req 23859200->23863295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.555759:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2e071 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23859200->23863295] (req 23859200->23863295) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760732.601974:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973144 is committed 00002000:00010000:1.0:1789760732.602140:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23859200 -> 23863296 00010000:00010000:1.0:1789760732.602157:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2e071 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23859200->23863295] (req 23859200->23863295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.602169:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2e071 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23859200->23863295] (req 23859200->23863295) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.602177:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2e071 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23859200->23863295] (req 23859200->23863295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.602184:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760732.602189:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2e071 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23859200->23863295] (req 23859200->23863295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.602198:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2e071 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23859200->23863295] (req 23859200->23863295) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.602215:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2e071 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23859200->23863295] (req 23859200->23863295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.602224:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2e071 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23859200->23863295] (req 23859200->23863295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760732.602327:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2878000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512605440:12345-192.168.203.56@tcp:4:dd.0 Request processed in 46675us (46976us total) trans 4294973144 rc 0/0 00000100:00100000:1.0:1789760732.602337:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5855 00000100:00100000:1.0:1789760732.602347:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 712067953465 time 712119937409 state 0x2 ref 0 00000100:00100000:2.0:1789760732.605084:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760732.605171:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512605568 00000100:00100000:3.0:1789760732.605180:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512605568 00000100:00100000:3.0:1789760732.605234:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 712067953465 time 712119937409 state 0x0 ref 1 00000100:00100000:3.0:1789760732.605249:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 712120197825 time 712119937409 state 0x1 ref 1 00000100:00100000:3.0:1789760732.605279:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 712120197825 time 712169578763 state 0x0 ref 1 00000100:00100000:3.0:1789760732.605290:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2fbaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512605568:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760732.605335:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23863296, mtime: 0x6aad94dc, atime: 0x0, ctime: 0x6aad94dc, blocks: 0xb807 00010000:00010000:3.0:1789760732.605343:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2e078 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.605358:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=712s v=(0 0) 00010000:00010000:3.0:1789760732.605362:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760732.605368:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2e078 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23863296->23867391] (req 23863296->23867391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.605386:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2e078 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23863296->23867391] (req 23863296->23867391) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760732.645364:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973145 is committed 00002000:00010000:1.0:1789760732.645603:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23863296 -> 23867392 00010000:00010000:1.0:1789760732.645618:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2e078 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23863296->23867391] (req 23863296->23867391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.645631:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2e078 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23863296->23867391] (req 23863296->23867391) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.645640:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2e078 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23863296->23867391] (req 23863296->23867391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.645648:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760732.645654:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2e078 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23863296->23867391] (req 23863296->23867391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.645663:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2e078 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23863296->23867391] (req 23863296->23867391) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.645681:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2e078 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23863296->23867391] (req 23863296->23867391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.645688:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2e078 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23863296->23867391] (req 23863296->23867391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760732.646383:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2fbaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512605568:12345-192.168.203.56@tcp:4:dd.0 Request processed in 41088us (41309us total) trans 4294973145 rc 0/0 00000100:00100000:1.0:1789760732.646395:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5856 00000100:00100000:1.0:1789760732.646416:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 712120197825 time 712169578763 state 0x2 ref 0 00000100:00100000:2.0:1789760732.650212:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760732.650292:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512605696 00000100:00100000:1.0:1789760732.650302:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512605696 00000100:00100000:1.0:1789760732.650355:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 712120197825 time 712169578763 state 0x0 ref 1 00000100:00100000:1.0:1789760732.650371:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 712169839179 time 712169578763 state 0x1 ref 1 00000100:00100000:1.0:1789760732.650411:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 712169839179 time 712214700691 state 0x0 ref 1 00000100:00100000:1.0:1789760732.650425:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febfbddc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512605696:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760732.650536:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23867392, mtime: 0x6aad94dc, atime: 0x0, ctime: 0x6aad94dc, blocks: 0xb807 00010000:00010000:1.0:1789760732.650545:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2e07f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.650561:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=712s v=(0 0) 00010000:00010000:1.0:1789760732.650565:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760732.650572:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2e07f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23867392->23871487] (req 23867392->23871487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.650585:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2e07f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23867392->23871487] (req 23867392->23871487) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760732.703221:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973146 is committed 00002000:00010000:1.0:1789760732.703393:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23867392 -> 23871488 00010000:00010000:1.0:1789760732.703419:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2e07f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23867392->23871487] (req 23867392->23871487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.703436:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2e07f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23867392->23871487] (req 23867392->23871487) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.703445:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2e07f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23867392->23871487] (req 23867392->23871487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.703453:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760732.703458:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2e07f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23867392->23871487] (req 23867392->23871487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.703468:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2e07f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23867392->23871487] (req 23867392->23871487) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.703488:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2e07f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23867392->23871487] (req 23867392->23871487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.703496:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2e07f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23867392->23871487] (req 23867392->23871487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760732.703593:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febfbddc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512605696:12345-192.168.203.56@tcp:4:dd.0 Request processed in 53166us (53386us total) trans 4294973146 rc 0/0 00000100:00100000:1.0:1789760732.703604:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5857 00000100:00100000:1.0:1789760732.703613:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 712169839179 time 712214700691 state 0x2 ref 0 00000100:00100000:0.0:1789760732.705976:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760732.706133:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512605824 00000100:00100000:1.0:1789760732.706144:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512605824 00000100:00100000:1.0:1789760732.706199:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 712169839179 time 712214700691 state 0x0 ref 1 00000100:00100000:1.0:1789760732.706214:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 712214961107 time 712214700691 state 0x1 ref 1 00000100:00100000:1.0:1789760732.706291:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 712214961107 time 712270590308 state 0x0 ref 1 00000100:00100000:1.0:1789760732.706304:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac287b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512605824:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760732.706350:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23871488, mtime: 0x6aad94dc, atime: 0x0, ctime: 0x6aad94dc, blocks: 0xb807 00010000:00010000:1.0:1789760732.706359:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2e086 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.706374:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=712s v=(0 0) 00010000:00010000:1.0:1789760732.706379:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760732.706385:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2e086 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23871488->23875583] (req 23871488->23875583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.706398:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2e086 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23871488->23875583] (req 23871488->23875583) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760732.737352:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973147 is committed 00002000:00010000:3.0:1789760732.737491:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23871488 -> 23875584 00010000:00010000:3.0:1789760732.737506:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2e086 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23871488->23875583] (req 23871488->23875583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.737519:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2e086 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23871488->23875583] (req 23871488->23875583) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.737539:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2e086 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23871488->23875583] (req 23871488->23875583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.737546:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760732.737551:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2e086 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23871488->23875583] (req 23871488->23875583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.737560:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2e086 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23871488->23875583] (req 23871488->23875583) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.737578:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2e086 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23871488->23875583] (req 23871488->23875583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.737586:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2e086 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23871488->23875583] (req 23871488->23875583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760732.737687:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac287b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512605824:12345-192.168.203.56@tcp:4:dd.0 Request processed in 31378us (31713us total) trans 4294973147 rc 0/0 00000100:00100000:3.0:1789760732.737749:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5858 00000100:00100000:3.0:1789760732.737760:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 712214961107 time 712270590308 state 0x2 ref 0 00000100:00100000:3.0:1789760732.740898:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760732.741243:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512605952 00000100:00100000:1.0:1789760732.741253:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512605952 00000100:00100000:1.0:1789760732.741308:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 712214961107 time 712270590308 state 0x0 ref 1 00000100:00100000:1.0:1789760732.741323:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 712270850724 time 712270590308 state 0x1 ref 1 00000100:00100000:1.0:1789760732.741355:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 712270850724 time 712305654882 state 0x0 ref 1 00000100:00100000:1.0:1789760732.741372:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac531f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512605952:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760732.741434:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23875584, mtime: 0x6aad94dc, atime: 0x0, ctime: 0x6aad94dc, blocks: 0xb807 00010000:00010000:1.0:1789760732.741444:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2e08d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.741459:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=712s v=(0 0) 00010000:00010000:1.0:1789760732.741463:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760732.741469:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2e08d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23875584->23879679] (req 23875584->23879679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.741484:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2e08d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23875584->23879679] (req 23875584->23879679) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760732.772582:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973148 is committed 00002000:00010000:3.0:1789760732.772689:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23875584 -> 23879680 00010000:00010000:3.0:1789760732.772731:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2e08d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23875584->23879679] (req 23875584->23879679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.772745:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2e08d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23875584->23879679] (req 23875584->23879679) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.772754:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2e08d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23875584->23879679] (req 23875584->23879679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.772761:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760732.772767:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2e08d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23875584->23879679] (req 23875584->23879679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.772776:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2e08d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23875584->23879679] (req 23875584->23879679) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.772868:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2e08d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23875584->23879679] (req 23875584->23879679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.772882:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2e08d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23875584->23879679] (req 23875584->23879679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760732.772964:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac531f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512605952:12345-192.168.203.56@tcp:4:dd.0 Request processed in 31591us (32070us total) trans 4294973148 rc 0/0 00000100:00100000:3.0:1789760732.772973:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5859 00000100:00100000:3.0:1789760732.772983:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 712270850724 time 712305654882 state 0x2 ref 0 00000100:00100000:2.0:1789760732.775684:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760732.776180:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512606080 00000100:00100000:3.0:1789760732.776189:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512606080 00000100:00100000:3.0:1789760732.776242:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 712270850724 time 712305654882 state 0x0 ref 1 00000100:00100000:3.0:1789760732.776258:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 712305915298 time 712305654882 state 0x1 ref 1 00000100:00100000:3.0:1789760732.776289:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 712305915298 time 712340588001 state 0x0 ref 1 00000100:00100000:3.0:1789760732.776303:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac437e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512606080:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760732.776352:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23879680, mtime: 0x6aad94dc, atime: 0x0, ctime: 0x6aad94dc, blocks: 0xb807 00010000:00010000:3.0:1789760732.776361:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2e094 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.776378:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=712s v=(0 0) 00010000:00010000:3.0:1789760732.776382:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760732.776389:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2e094 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23879680->23883775] (req 23879680->23883775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.776419:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2e094 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23879680->23883775] (req 23879680->23883775) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760732.809032:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973149 is committed 00002000:00010000:1.0:1789760732.809254:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23879680 -> 23883776 00010000:00010000:1.0:1789760732.809270:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2e094 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23879680->23883775] (req 23879680->23883775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.809283:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2e094 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23879680->23883775] (req 23879680->23883775) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.809293:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2e094 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23879680->23883775] (req 23879680->23883775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.809301:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760732.809306:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2e094 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23879680->23883775] (req 23879680->23883775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.809315:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2e094 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23879680->23883775] (req 23879680->23883775) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.809333:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2e094 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23879680->23883775] (req 23879680->23883775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.809340:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2e094 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23879680->23883775] (req 23879680->23883775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760732.809473:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac437e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512606080:12345-192.168.203.56@tcp:4:dd.0 Request processed in 33167us (33793us total) trans 4294973149 rc 0/0 00000100:00100000:1.0:1789760732.809484:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5860 00000100:00100000:1.0:1789760732.809494:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 712305915298 time 712340588001 state 0x2 ref 0 00000100:00100000:0.0:1789760732.812067:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760732.812146:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512606208 00000100:00100000:1.0:1789760732.812156:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512606208 00000100:00100000:1.0:1789760732.812208:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 712305915298 time 712340588001 state 0x0 ref 1 00000100:00100000:1.0:1789760732.812223:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 712340848417 time 712340588001 state 0x1 ref 1 00000100:00100000:1.0:1789760732.812255:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 712340848417 time 712376554591 state 0x0 ref 1 00000100:00100000:1.0:1789760732.812268:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2878700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512606208:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760732.812314:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23883776, mtime: 0x6aad94dc, atime: 0x0, ctime: 0x6aad94dc, blocks: 0xb807 00010000:00010000:1.0:1789760732.812323:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2e09b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.812341:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=712s v=(0 0) 00010000:00010000:1.0:1789760732.812346:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760732.812353:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2e09b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23883776->23887871] (req 23883776->23887871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.812368:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2e09b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23883776->23887871] (req 23883776->23887871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760732.859224:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973150 is committed 00002000:00010000:3.0:1789760732.859616:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23883776 -> 23887872 00010000:00010000:3.0:1789760732.859634:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2e09b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23883776->23887871] (req 23883776->23887871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.859647:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2e09b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23883776->23887871] (req 23883776->23887871) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.859656:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2e09b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23883776->23887871] (req 23883776->23887871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.859663:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760732.859668:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2e09b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23883776->23887871] (req 23883776->23887871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.859676:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2e09b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23883776->23887871] (req 23883776->23887871) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.859742:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2e09b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23883776->23887871] (req 23883776->23887871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.859754:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2e09b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23883776->23887871] (req 23883776->23887871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760732.859857:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2878700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512606208:12345-192.168.203.56@tcp:4:dd.0 Request processed in 47585us (47794us total) trans 4294973150 rc 0/0 00000100:00100000:3.0:1789760732.859866:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5861 00000100:00100000:3.0:1789760732.859890:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 712340848417 time 712376554591 state 0x2 ref 0 00000100:00100000:0.0:1789760732.863222:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760732.863309:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512606336 00000100:00100000:3.0:1789760732.863320:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512606336 00000100:00100000:3.0:1789760732.863377:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 712340848417 time 712376554591 state 0x0 ref 1 00000100:00100000:3.0:1789760732.863393:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 712376815007 time 712376554591 state 0x1 ref 1 00000100:00100000:3.0:1789760732.863434:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 712376815007 time 712427734313 state 0x0 ref 1 00000100:00100000:3.0:1789760732.863446:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac287aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512606336:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760732.863494:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23887872, mtime: 0x6aad94dc, atime: 0x0, ctime: 0x6aad94dc, blocks: 0xb807 00010000:00010000:3.0:1789760732.863503:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2e0a2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.863518:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=712s v=(0 0) 00010000:00010000:3.0:1789760732.863523:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760732.863529:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2e0a2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23887872->23891967] (req 23887872->23891967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.863543:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2e0a2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23887872->23891967] (req 23887872->23891967) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760732.910804:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973151 is committed 00002000:00010000:3.0:1789760732.911223:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23887872 -> 23891968 00010000:00010000:3.0:1789760732.911240:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2e0a2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23887872->23891967] (req 23887872->23891967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.911253:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2e0a2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23887872->23891967] (req 23887872->23891967) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.911262:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2e0a2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23887872->23891967] (req 23887872->23891967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.911270:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760732.911274:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2e0a2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23887872->23891967] (req 23887872->23891967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.911285:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2e0a2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23887872->23891967] (req 23887872->23891967) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.911317:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2e0a2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23887872->23891967] (req 23887872->23891967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.911325:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2e0a2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23887872->23891967] (req 23887872->23891967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760732.911428:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac287aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512606336:12345-192.168.203.56@tcp:4:dd.0 Request processed in 47980us (48213us total) trans 4294973151 rc 0/0 00000100:00100000:3.0:1789760732.911443:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5862 00000100:00100000:3.0:1789760732.911453:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 712376815007 time 712427734313 state 0x2 ref 0 00000100:00100000:2.0:1789760732.918445:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760732.918615:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512606464 00000100:00100000:1.0:1789760732.918626:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512606464 00000100:00100000:1.0:1789760732.918681:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 712376815007 time 712427734313 state 0x0 ref 1 00000100:00100000:1.0:1789760732.918718:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 712427994729 time 712427734313 state 0x1 ref 1 00000100:00100000:1.0:1789760732.918788:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 712427994729 time 712483087212 state 0x0 ref 1 00000100:00100000:1.0:1789760732.918802:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac437df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512606464:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760732.918852:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23891968, mtime: 0x6aad94dc, atime: 0x0, ctime: 0x6aad94dc, blocks: 0xb807 00010000:00010000:1.0:1789760732.918860:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2e0a9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.918876:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=712s v=(0 0) 00010000:00010000:1.0:1789760732.918881:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760732.918887:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2e0a9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23891968->23896063] (req 23891968->23896063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.918906:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2e0a9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23891968->23896063] (req 23891968->23896063) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760732.959848:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23891968 -> 23896064 00010000:00010000:1.0:1789760732.959868:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2e0a9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23891968->23896063] (req 23891968->23896063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.959882:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2e0a9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23891968->23896063] (req 23891968->23896063) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.959891:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2e0a9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23891968->23896063] (req 23891968->23896063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.959899:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760732.959905:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2e0a9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23891968->23896063] (req 23891968->23896063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.960823:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2e0a9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23891968->23896063] (req 23891968->23896063) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.960846:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2e0a9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23891968->23896063] (req 23891968->23896063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.960855:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2e0a9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23891968->23896063] (req 23891968->23896063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760732.961049:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973152 is committed 00000100:00100000:1.0:1789760732.961541:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac437df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512606464:12345-192.168.203.56@tcp:4:dd.0 Request processed in 42736us (43100us total) trans 4294973152 rc 0/0 00000100:00100000:1.0:1789760732.961552:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5863 00000100:00100000:1.0:1789760732.961562:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 712427994729 time 712483087212 state 0x2 ref 0 00000100:00100000:2.0:1789760732.964498:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760732.964635:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512606592 00000100:00100000:3.0:1789760732.964645:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512606592 00000100:00100000:3.0:1789760732.964736:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 712427994729 time 712483087212 state 0x0 ref 1 00000100:00100000:3.0:1789760732.964752:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 712483347628 time 712483087212 state 0x1 ref 1 00000100:00100000:3.0:1789760732.964827:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 712483347628 time 712529125493 state 0x0 ref 1 00000100:00100000:3.0:1789760732.964841:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac437fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512606592:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760732.964892:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23896064, mtime: 0x6aad94dc, atime: 0x0, ctime: 0x6aad94dc, blocks: 0xb807 00010000:00010000:3.0:1789760732.964902:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2e0b0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.964931:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=712s v=(0 0) 00010000:00010000:3.0:1789760732.964935:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760732.964941:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2e0b0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23896064->23900159] (req 23896064->23900159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.964954:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2e0b0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23896064->23900159] (req 23896064->23900159) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760732.994034:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973153 is committed 00002000:00010000:3.0:1789760732.994288:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23896064 -> 23900160 00010000:00010000:3.0:1789760732.994303:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2e0b0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23896064->23900159] (req 23896064->23900159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.994318:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2e0b0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23896064->23900159] (req 23896064->23900159) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.994328:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2e0b0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23896064->23900159] (req 23896064->23900159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.994336:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760732.994340:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2e0b0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23896064->23900159] (req 23896064->23900159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.994350:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2e0b0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23896064->23900159] (req 23896064->23900159) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.994368:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2e0b0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23896064->23900159] (req 23896064->23900159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.994375:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2e0b0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23896064->23900159] (req 23896064->23900159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760732.994470:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac437fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512606592:12345-192.168.203.56@tcp:4:dd.0 Request processed in 29627us (29976us total) trans 4294973153 rc 0/0 00000100:00100000:3.0:1789760732.994481:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5864 00000100:00100000:3.0:1789760732.994490:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 712483347628 time 712529125493 state 0x2 ref 0 00000100:00100000:0.0:1789760732.998572:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760732.998659:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512606720 00000100:00100000:3.0:1789760732.998670:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512606720 00000100:00100000:3.0:1789760732.998767:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 712483347628 time 712529125493 state 0x0 ref 1 00000100:00100000:3.0:1789760732.998782:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 712529385909 time 712529125493 state 0x1 ref 1 00000100:00100000:3.0:1789760732.998820:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 712529385909 time 712563118992 state 0x0 ref 1 00000100:00100000:3.0:1789760732.998833:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2879880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512606720:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760732.998882:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23900160, mtime: 0x6aad94dc, atime: 0x0, ctime: 0x6aad94dc, blocks: 0xb807 00010000:00010000:3.0:1789760732.998891:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2e0b7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.998916:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=712s v=(0 0) 00010000:00010000:3.0:1789760732.998921:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760732.998928:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2e0b7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23900160->23904255] (req 23900160->23904255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.998942:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2e0b7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23900160->23904255] (req 23900160->23904255) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760733.026114:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23900160 -> 23904256 00010000:00010000:3.0:1789760733.026136:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2e0b7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23900160->23904255] (req 23900160->23904255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.026150:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2e0b7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23900160->23904255] (req 23900160->23904255) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.026166:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2e0b7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23900160->23904255] (req 23900160->23904255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.026174:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760733.026179:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2e0b7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23900160->23904255] (req 23900160->23904255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.026189:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2e0b7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23900160->23904255] (req 23900160->23904255) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.026208:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2e0b7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23900160->23904255] (req 23900160->23904255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.026216:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2e0b7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23900160->23904255] (req 23900160->23904255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760733.026339:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2879880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2360:x1876699512606720:12345-192.168.203.56@tcp:4:dd.0 Request processed in 27504us (27771us total) trans 4294973154 rc 0/0 00000100:00100000:3.0:1789760733.026349:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5865 00000100:00100000:3.0:1789760733.026357:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 712529385909 time 712563118992 state 0x2 ref 0 00000001:00080000:2.0:1789760733.026500:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973154 is committed 00000100:00100000:2.0:1789760733.029995:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760733.030155:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512606848 00000100:00100000:1.0:1789760733.030164:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512606848 00000100:00100000:1.0:1789760733.030263:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 712529385909 time 712563118992 state 0x0 ref 1 00000100:00100000:1.0:1789760733.030279:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 712563379408 time 712563118992 state 0x1 ref 1 00000100:00100000:1.0:1789760733.030311:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 712563379408 time 712594611015 state 0x0 ref 1 00000100:00100000:1.0:1789760733.030324:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac437f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512606848:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760733.030374:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23904256, mtime: 0x6aad94dc, atime: 0x0, ctime: 0x6aad94dc, blocks: 0xb807 00010000:00010000:1.0:1789760733.030383:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2e0be lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.030413:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=712s v=(0 0) 00010000:00010000:1.0:1789760733.030418:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760733.030426:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2e0be lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23904256->23908351] (req 23904256->23908351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.030441:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2e0be lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23904256->23908351] (req 23904256->23908351) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1789760733.031321:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:2.0:1789760733.064246:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973155 is committed 00002000:00010000:3.0:1789760733.064363:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23904256 -> 23908352 00010000:00010000:3.0:1789760733.064380:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2e0be lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23904256->23908351] (req 23904256->23908351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.064410:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2e0be lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23904256->23908351] (req 23904256->23908351) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.064420:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2e0be lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23904256->23908351] (req 23904256->23908351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.064427:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760733.064432:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2e0be lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23904256->23908351] (req 23904256->23908351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.064442:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2e0be lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23904256->23908351] (req 23904256->23908351) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.064460:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2e0be lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23904256->23908351] (req 23904256->23908351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.064469:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2e0be lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23904256->23908351] (req 23904256->23908351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760733.064558:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac437f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512606848:12345-192.168.203.56@tcp:4:dd.0 Request processed in 34231us (34567us total) trans 4294973155 rc 0/0 00000100:00100000:3.0:1789760733.064568:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5866 00000100:00100000:3.0:1789760733.064577:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 712563379408 time 712594611015 state 0x2 ref 0 00000100:00100000:1.0:1789760733.068302:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760733.068847:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512606976 00000100:00100000:3.0:1789760733.068859:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512606976 00000100:00100000:3.0:1789760733.068917:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 712563379408 time 712594611015 state 0x0 ref 1 00000100:00100000:3.0:1789760733.068932:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 712594871431 time 712594611015 state 0x1 ref 1 00000100:00100000:3.0:1789760733.069024:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 712594871431 time 712633322304 state 0x0 ref 1 00000100:00100000:3.0:1789760733.069038:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4131f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512606976:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760733.069106:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23908352, mtime: 0x6aad94dc, atime: 0x0, ctime: 0x6aad94dc, blocks: 0xb807 00010000:00010000:3.0:1789760733.069116:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2e0c5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.069132:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=712s v=(0 0) 00010000:00010000:3.0:1789760733.069136:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760733.069143:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2e0c5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23908352->23912447] (req 23908352->23912447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.069157:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2e0c5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23908352->23912447] (req 23908352->23912447) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760733.108473:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23908352 -> 23912448 00010000:00010000:3.0:1789760733.108492:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2e0c5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23908352->23912447] (req 23908352->23912447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.108505:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2e0c5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23908352->23912447] (req 23908352->23912447) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.108520:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2e0c5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23908352->23912447] (req 23908352->23912447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.108527:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760733.108533:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2e0c5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23908352->23912447] (req 23908352->23912447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.108542:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2e0c5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23908352->23912447] (req 23908352->23912447) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.108562:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2e0c5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23908352->23912447] (req 23908352->23912447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.108571:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2e0c5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23908352->23912447] (req 23908352->23912447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760733.108667:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4131f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2358:x1876699512606976:12345-192.168.203.56@tcp:4:dd.0 Request processed in 39628us (40373us total) trans 4294973156 rc 0/0 00000100:00100000:3.0:1789760733.108678:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5867 00000100:00100000:3.0:1789760733.108687:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 712594871431 time 712633322304 state 0x2 ref 0 00000001:00080000:2.0:1789760733.109085:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973156 is committed 00000100:00100000:2.0:1789760733.113852:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760733.113962:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512607104 00000100:00100000:3.0:1789760733.113972:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512607104 00000100:00100000:3.0:1789760733.114027:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 712594871431 time 712633322304 state 0x0 ref 1 00000100:00100000:3.0:1789760733.114043:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 712633582720 time 712633322304 state 0x1 ref 1 00000100:00100000:3.0:1789760733.114160:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 712633582720 time 712678458527 state 0x0 ref 1 00000100:00100000:3.0:1789760733.114173:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac437d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512607104:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760733.114220:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23912448, mtime: 0x6aad94dc, atime: 0x0, ctime: 0x6aad94dc, blocks: 0xb807 00010000:00010000:3.0:1789760733.114228:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2e0cc lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.114243:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=712s v=(0 0) 00010000:00010000:3.0:1789760733.114247:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760733.114253:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2e0cc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23912448->23916543] (req 23912448->23916543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.114265:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2e0cc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23912448->23916543] (req 23912448->23916543) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760733.158026:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23912448 -> 23916544 00010000:00010000:3.0:1789760733.158045:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2e0cc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23912448->23916543] (req 23912448->23916543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.158058:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2e0cc lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23912448->23916543] (req 23912448->23916543) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.158066:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2e0cc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23912448->23916543] (req 23912448->23916543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.158084:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760733.158090:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2e0cc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23912448->23916543] (req 23912448->23916543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.158098:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2e0cc lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23912448->23916543] (req 23912448->23916543) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.158117:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2e0cc lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23912448->23916543] (req 23912448->23916543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.158125:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2e0cc lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23912448->23916543] (req 23912448->23916543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760733.158217:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac437d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2358:x1876699512607104:12345-192.168.203.56@tcp:4:dd.0 Request processed in 44042us (44434us total) trans 4294973157 rc 0/0 00000100:00100000:3.0:1789760733.158229:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5868 00000100:00100000:3.0:1789760733.158239:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 712633582720 time 712678458527 state 0x2 ref 0 00000001:00080000:2.0:1789760733.160898:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973157 is committed 00000100:00100000:0.0:1789760733.175615:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760733.175931:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512607232 00000100:00100000:3.0:1789760733.175941:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512607232 00000100:00100000:2.0:1789760733.176415:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 712633582720 time 712678458527 state 0x0 ref 1 00000100:00100000:2.0:1789760733.176435:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 712678718943 time 712678458527 state 0x1 ref 1 00000100:00100000:2.0:1789760733.176468:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 712678718943 time 712740767866 state 0x0 ref 1 00000100:00100000:2.0:1789760733.176481:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2879180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512607232:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760733.176652:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23916544, mtime: 0x6aad94dc, atime: 0x0, ctime: 0x6aad94dc, blocks: 0xb807 00010000:00010000:2.0:1789760733.176663:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2e0d3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.176679:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=712s v=(0 0) 00010000:00010000:2.0:1789760733.176683:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760733.176690:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2e0d3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23916544->23920639] (req 23916544->23920639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.176742:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2e0d3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23916544->23920639] (req 23916544->23920639) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760733.218315:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973158 is committed 00002000:00010000:1.0:1789760733.218436:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23916544 -> 23920640 00010000:00010000:1.0:1789760733.218622:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2e0d3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23916544->23920639] (req 23916544->23920639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.218636:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2e0d3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23916544->23920639] (req 23916544->23920639) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.218647:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2e0d3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23916544->23920639] (req 23916544->23920639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.218655:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760733.218660:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2e0d3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23916544->23920639] (req 23916544->23920639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.218669:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2e0d3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23916544->23920639] (req 23916544->23920639) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.218689:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2e0d3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23916544->23920639] (req 23916544->23920639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.218778:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2e0d3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23916544->23920639] (req 23916544->23920639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760733.218907:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2879180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512607232:12345-192.168.203.56@tcp:4:dd.0 Request processed in 42424us (43297us total) trans 4294973158 rc 0/0 00000100:00100000:1.0:1789760733.218919:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5869 00000100:00100000:1.0:1789760733.218929:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 712678718943 time 712740767866 state 0x2 ref 0 00000100:00100000:1.0:1789760733.222032:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760733.222184:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512607360 00000100:00100000:3.0:1789760733.222193:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512607360 00000100:00100000:3.0:1789760733.222245:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 712678718943 time 712740767866 state 0x0 ref 1 00000100:00100000:3.0:1789760733.222261:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 712741028282 time 712740767866 state 0x1 ref 1 00000100:00100000:3.0:1789760733.222347:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 712741028282 time 712786646022 state 0x0 ref 1 00000100:00100000:3.0:1789760733.222365:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4130380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512607360:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760733.222423:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23920640, mtime: 0x6aad94dc, atime: 0x0, ctime: 0x6aad94dc, blocks: 0xb807 00010000:00010000:3.0:1789760733.222432:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2e0da lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.222457:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=712s v=(0 0) 00010000:00010000:3.0:1789760733.222461:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760733.222468:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2e0da lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23920640->23924735] (req 23920640->23924735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.222490:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2e0da lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23920640->23924735] (req 23920640->23924735) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760733.265031:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23920640 -> 23924736 00010000:00010000:3.0:1789760733.265051:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2e0da lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23920640->23924735] (req 23920640->23924735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.265065:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2e0da lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23920640->23924735] (req 23920640->23924735) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.265087:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2e0da lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23920640->23924735] (req 23920640->23924735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.265094:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760733.265099:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2e0da lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23920640->23924735] (req 23920640->23924735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.265109:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2e0da lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23920640->23924735] (req 23920640->23924735) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.265128:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2e0da lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23920640->23924735] (req 23920640->23924735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.265137:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2e0da lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23920640->23924735] (req 23920640->23924735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760733.265228:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4130380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2358:x1876699512607360:12345-192.168.203.56@tcp:4:dd.0 Request processed in 42861us (43200us total) trans 4294973159 rc 0/0 00000100:00100000:3.0:1789760733.265239:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5870 00000100:00100000:3.0:1789760733.265248:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 712741028282 time 712786646022 state 0x2 ref 0 00000001:00080000:1.0:1789760733.265393:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973159 is committed 00000100:00100000:2.0:1789760733.268033:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760733.268199:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512607488 00000100:00100000:3.0:1789760733.268210:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512607488 00000100:00100000:3.0:1789760733.268264:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 712741028282 time 712786646022 state 0x0 ref 1 00000100:00100000:3.0:1789760733.268278:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 712786906438 time 712786646022 state 0x1 ref 1 00000100:00100000:3.0:1789760733.268303:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 712786906438 time 712832602716 state 0x0 ref 1 00000100:00100000:3.0:1789760733.268317:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4074700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512607488:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760733.268367:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23924736, mtime: 0x6aad94dc, atime: 0x0, ctime: 0x6aad94dc, blocks: 0xb807 00010000:00010000:3.0:1789760733.268376:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2e0e1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.268391:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=712s v=(0 0) 00010000:00010000:3.0:1789760733.268406:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760733.268413:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2e0e1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23924736->23928831] (req 23924736->23928831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.268427:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2e0e1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23924736->23928831] (req 23924736->23928831) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760733.294383:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973160 is committed 00002000:00010000:3.0:1789760733.294497:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23924736 -> 23928832 00010000:00010000:3.0:1789760733.294513:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2e0e1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23924736->23928831] (req 23924736->23928831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.294527:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2e0e1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23924736->23928831] (req 23924736->23928831) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.294536:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2e0e1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23924736->23928831] (req 23924736->23928831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.294543:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760733.294548:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2e0e1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23924736->23928831] (req 23924736->23928831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.294556:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2e0e1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23924736->23928831] (req 23924736->23928831) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.294576:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2e0e1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23924736->23928831] (req 23924736->23928831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.294585:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2e0e1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23924736->23928831] (req 23924736->23928831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760733.294678:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4074700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512607488:12345-192.168.203.56@tcp:4:dd.0 Request processed in 26360us (26649us total) trans 4294973160 rc 0/0 00000100:00100000:3.0:1789760733.294688:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5871 00000100:00100000:3.0:1789760733.294736:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 712786906438 time 712832602716 state 0x2 ref 0 00000100:00100000:2.0:1789760733.298095:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760733.298277:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512607616 00000100:00100000:1.0:1789760733.298287:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512607616 00000100:00100000:1.0:1789760733.298340:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 712786906438 time 712832602716 state 0x0 ref 1 00000100:00100000:1.0:1789760733.298355:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 712832863132 time 712832602716 state 0x1 ref 1 00000100:00100000:1.0:1789760733.298386:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 712832863132 time 712862685687 state 0x0 ref 1 00000100:00100000:1.0:1789760733.298407:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4074a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512607616:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760733.298452:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23928832, mtime: 0x6aad94dc, atime: 0x0, ctime: 0x6aad94dc, blocks: 0xb807 00010000:00010000:1.0:1789760733.298461:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2e0e8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.298475:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=712s v=(0 0) 00010000:00010000:1.0:1789760733.298480:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760733.298487:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2e0e8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23928832->23932927] (req 23928832->23932927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.298501:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2e0e8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23928832->23932927] (req 23928832->23932927) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760733.335076:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23928832 -> 23932928 00010000:00010000:2.0:1789760733.335098:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2e0e8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23928832->23932927] (req 23928832->23932927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.335111:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2e0e8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23928832->23932927] (req 23928832->23932927) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.335125:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2e0e8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23928832->23932927] (req 23928832->23932927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.335132:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760733.335138:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2e0e8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23928832->23932927] (req 23928832->23932927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.335147:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2e0e8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23928832->23932927] (req 23928832->23932927) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.335165:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2e0e8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23928832->23932927] (req 23928832->23932927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.335172:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2e0e8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23928832->23932927] (req 23928832->23932927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760733.335268:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4074a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2359:x1876699512607616:12345-192.168.203.56@tcp:4:dd.0 Request processed in 36867us (37179us total) trans 4294973161 rc 0/0 00000100:00100000:2.0:1789760733.335279:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5872 00000100:00100000:2.0:1789760733.335289:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 712832863132 time 712862685687 state 0x2 ref 0 00000001:00080000:3.0:1789760733.335974:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973161 is committed 00000100:00100000:1.0:1789760733.338515:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760733.338641:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512607744 00000100:00100000:3.0:1789760733.338651:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512607744 00000100:00100000:3.0:1789760733.338720:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 712832863132 time 712862685687 state 0x0 ref 1 00000100:00100000:3.0:1789760733.338735:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 712862946103 time 712862685687 state 0x1 ref 1 00000100:00100000:3.0:1789760733.338767:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 712862946103 time 712903066846 state 0x0 ref 1 00000100:00100000:3.0:1789760733.338800:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4133b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512607744:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760733.338851:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23932928, mtime: 0x6aad94dc, atime: 0x0, ctime: 0x6aad94dc, blocks: 0xb807 00010000:00010000:3.0:1789760733.338862:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2e0ef lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.338878:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=712s v=(0 0) 00010000:00010000:3.0:1789760733.338882:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760733.338888:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2e0ef lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23932928->23937023] (req 23932928->23937023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.338901:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2e0ef lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23932928->23937023] (req 23932928->23937023) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760733.374838:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973162 is committed 00002000:00010000:2.0:1789760733.374877:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23932928 -> 23937024 00010000:00010000:2.0:1789760733.374893:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2e0ef lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23932928->23937023] (req 23932928->23937023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.374910:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2e0ef lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23932928->23937023] (req 23932928->23937023) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.374919:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2e0ef lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23932928->23937023] (req 23932928->23937023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.374927:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760733.374932:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2e0ef lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23932928->23937023] (req 23932928->23937023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.374942:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2e0ef lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23932928->23937023] (req 23932928->23937023) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.374962:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2e0ef lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23932928->23937023] (req 23932928->23937023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.376675:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2e0ef lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23932928->23937023] (req 23932928->23937023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760733.376909:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4133b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512607744:12345-192.168.203.56@tcp:4:dd.0 Request processed in 38115us (38397us total) trans 4294973162 rc 0/0 00000100:00100000:2.0:1789760733.376921:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5873 00000100:00100000:2.0:1789760733.376931:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 712862946103 time 712903066846 state 0x2 ref 0 00000100:00100000:1.0:1789760733.382664:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760733.382896:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512607872 00000100:00100000:3.0:1789760733.382906:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512607872 00000100:00100000:3.0:1789760733.382962:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 712862946103 time 712903066846 state 0x0 ref 1 00000100:00100000:3.0:1789760733.382978:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 712903327262 time 712903066846 state 0x1 ref 1 00000100:00100000:3.0:1789760733.383018:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 712903327262 time 712947318027 state 0x0 ref 1 00000100:00100000:3.0:1789760733.383031:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4133800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512607872:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760733.383103:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23937024, mtime: 0x6aad94dc, atime: 0x0, ctime: 0x6aad94dc, blocks: 0xb807 00010000:00010000:3.0:1789760733.383112:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2e0f6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.383128:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=712s v=(0 0) 00010000:00010000:3.0:1789760733.383132:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760733.383138:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2e0f6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23937024->23941119] (req 23937024->23941119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.383152:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2e0f6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23937024->23941119] (req 23937024->23941119) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760733.449533:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23937024 -> 23941120 00010000:00010000:3.0:1789760733.449553:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2e0f6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23937024->23941119] (req 23937024->23941119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.449565:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2e0f6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23937024->23941119] (req 23937024->23941119) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.449575:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2e0f6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23937024->23941119] (req 23937024->23941119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.449581:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760733.449585:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2e0f6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23937024->23941119] (req 23937024->23941119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.449594:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2e0f6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23937024->23941119] (req 23937024->23941119) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.449616:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2e0f6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23937024->23941119] (req 23937024->23941119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.449623:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2e0f6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23937024->23941119] (req 23937024->23941119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760733.449771:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4133800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2359:x1876699512607872:12345-192.168.203.56@tcp:4:dd.0 Request processed in 66736us (67111us total) trans 4294973163 rc 0/0 00000100:00100000:3.0:1789760733.449787:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5874 00000100:00100000:3.0:1789760733.449800:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 712903327262 time 712947318027 state 0x2 ref 0 00000001:00080000:0.0:1789760733.449829:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973163 is committed 00000100:00100000:0.0:1789760733.454629:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760733.454720:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512608000 00000100:00100000:2.0:1789760733.454729:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512608000 00000100:00100000:2.0:1789760733.454785:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 712903327262 time 712947318027 state 0x0 ref 1 00000100:00100000:2.0:1789760733.454801:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 712947578443 time 712947318027 state 0x1 ref 1 00000100:00100000:2.0:1789760733.454826:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 712947578443 time 713019125546 state 0x0 ref 1 00000100:00100000:2.0:1789760733.454838:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2879500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512608000:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760733.454889:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23941120, mtime: 0x6aad94dc, atime: 0x0, ctime: 0x6aad94dc, blocks: 0xb807 00010000:00010000:2.0:1789760733.454899:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2e0fd lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.454914:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=713s v=(0 0) 00010000:00010000:2.0:1789760733.454918:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760733.454924:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2e0fd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23941120->23945215] (req 23941120->23945215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.454938:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2e0fd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23941120->23945215] (req 23941120->23945215) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760733.528097:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973164 is committed 00002000:00010000:3.0:1789760733.530442:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23941120 -> 23945216 00010000:00010000:3.0:1789760733.530467:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2e0fd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23941120->23945215] (req 23941120->23945215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.530481:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2e0fd lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23941120->23945215] (req 23941120->23945215) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.530491:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2e0fd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23941120->23945215] (req 23941120->23945215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.530500:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760733.530505:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2e0fd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23941120->23945215] (req 23941120->23945215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.530515:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2e0fd lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23941120->23945215] (req 23941120->23945215) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.530536:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2e0fd lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23941120->23945215] (req 23941120->23945215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.530545:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2e0fd lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23941120->23945215] (req 23941120->23945215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760733.530766:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2879500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512608000:12345-192.168.203.56@tcp:4:dd.0 Request processed in 75923us (76142us total) trans 4294973164 rc 0/0 00000100:00100000:3.0:1789760733.530778:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5875 00000100:00100000:3.0:1789760733.530788:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 712947578443 time 713019125546 state 0x2 ref 0 00000100:00100000:1.0:1789760733.536211:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760733.536304:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512608128 00000100:00100000:2.0:1789760733.536314:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512608128 00000100:00100000:2.0:1789760733.536367:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 712947578443 time 713019125546 state 0x0 ref 1 00000100:00100000:2.0:1789760733.536382:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 713019385962 time 713019125546 state 0x1 ref 1 00000100:00100000:2.0:1789760733.536469:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 713019385962 time 713100768266 state 0x0 ref 1 00000100:00100000:2.0:1789760733.536485:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4130000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512608128:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760733.536537:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23945216, mtime: 0x6aad94dc, atime: 0x0, ctime: 0x6aad94dc, blocks: 0xb807 00010000:00010000:2.0:1789760733.536546:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2e104 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.536564:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=713s v=(0 0) 00010000:00010000:2.0:1789760733.536568:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760733.536575:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2e104 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23945216->23949311] (req 23945216->23949311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.536590:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2e104 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23945216->23949311] (req 23945216->23949311) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.556223:0:5835:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:2.0:1789760733.578491:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973165 is committed 00002000:00010000:3.0:1789760733.578597:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23945216 -> 23949312 00002000:00010000:3.0:1789760733.578602:0:6709:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760732 -> 1789760733 00002000:00010000:3.0:1789760733.578605:0:6709:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760732 -> 1789760733 00010000:00010000:3.0:1789760733.578618:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2e104 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23945216->23949311] (req 23945216->23949311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.578631:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2e104 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23945216->23949311] (req 23945216->23949311) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.578640:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2e104 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23945216->23949311] (req 23945216->23949311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.578648:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760733.578652:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2e104 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23945216->23949311] (req 23945216->23949311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.578662:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2e104 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23945216->23949311] (req 23945216->23949311) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.578681:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2e104 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23945216->23949311] (req 23945216->23949311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.578690:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2e104 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23945216->23949311] (req 23945216->23949311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760733.578816:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4130000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512608128:12345-192.168.203.56@tcp:4:dd.0 Request processed in 42328us (42610us total) trans 4294973165 rc 0/0 00000100:00100000:3.0:1789760733.578826:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5876 00000100:00100000:3.0:1789760733.578835:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 713019385962 time 713100768266 state 0x2 ref 0 00000100:00100000:3.0:1789760733.581500:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760733.581595:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512608256 00000100:00100000:2.0:1789760733.581605:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512608256 00000100:00100000:2.0:1789760733.581654:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 713019385962 time 713100768266 state 0x0 ref 1 00000100:00100000:2.0:1789760733.581668:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 713101028682 time 713100768266 state 0x1 ref 1 00000100:00100000:2.0:1789760733.581729:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 713101028682 time 713146028062 state 0x0 ref 1 00000100:00100000:2.0:1789760733.581742:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac531df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512608256:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760733.581799:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23949312, mtime: 0x6aad94dd, atime: 0x0, ctime: 0x6aad94dd, blocks: 0xb807 00010000:00010000:2.0:1789760733.581808:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2e10b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.581823:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=713s v=(0 0) 00010000:00010000:2.0:1789760733.581828:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760733.581834:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2e10b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23949312->23953407] (req 23949312->23953407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.581848:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2e10b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23949312->23953407] (req 23949312->23953407) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760733.610997:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973166 is committed 00002000:00010000:3.0:1789760733.611119:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23949312 -> 23953408 00010000:00010000:3.0:1789760733.611135:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2e10b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23949312->23953407] (req 23949312->23953407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.611149:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2e10b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23949312->23953407] (req 23949312->23953407) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.611159:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2e10b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23949312->23953407] (req 23949312->23953407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.611167:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760733.611172:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2e10b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23949312->23953407] (req 23949312->23953407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.611181:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2e10b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23949312->23953407] (req 23949312->23953407) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.611205:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2e10b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23949312->23953407] (req 23949312->23953407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.611213:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2e10b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23949312->23953407] (req 23949312->23953407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760733.611310:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac531df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512608256:12345-192.168.203.56@tcp:4:dd.0 Request processed in 29566us (29814us total) trans 4294973166 rc 0/0 00000100:00100000:3.0:1789760733.611321:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5877 00000100:00100000:3.0:1789760733.611332:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 713101028682 time 713146028062 state 0x2 ref 0 00000100:00100000:3.0:1789760733.614243:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760733.614384:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512608384 00000100:00100000:0.0:1789760733.614403:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512608384 00000100:00100000:0.0:1789760733.614457:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 713101028682 time 713146028062 state 0x0 ref 1 00000100:00100000:0.0:1789760733.614473:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 713146288478 time 713146028062 state 0x1 ref 1 00000100:00100000:0.0:1789760733.614505:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 713146288478 time 713178804380 state 0x0 ref 1 00000100:00100000:0.0:1789760733.614517:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac531c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512608384:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760733.614570:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23953408, mtime: 0x6aad94dd, atime: 0x0, ctime: 0x6aad94dd, blocks: 0xb807 00010000:00010000:0.0:1789760733.614579:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2e112 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.614595:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=713s v=(0 0) 00010000:00010000:0.0:1789760733.614599:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760733.614606:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2e112 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23953408->23957503] (req 23953408->23957503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.614620:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2e112 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23953408->23957503] (req 23953408->23957503) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760733.686151:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973167 is committed 00002000:00010000:0.0:1789760733.686261:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23953408 -> 23957504 00010000:00010000:0.0:1789760733.686277:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2e112 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23953408->23957503] (req 23953408->23957503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.686290:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2e112 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23953408->23957503] (req 23953408->23957503) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.686305:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2e112 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23953408->23957503] (req 23953408->23957503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.686313:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760733.686318:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2e112 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23953408->23957503] (req 23953408->23957503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.686327:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2e112 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23953408->23957503] (req 23953408->23957503) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.686348:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2e112 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23953408->23957503] (req 23953408->23957503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.686356:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2e112 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23953408->23957503] (req 23953408->23957503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760733.686518:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac531c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512608384:12345-192.168.203.56@tcp:4:dd.0 Request processed in 71998us (72279us total) trans 4294973167 rc 0/0 00000100:00100000:0.0:1789760733.686528:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5878 00000100:00100000:0.0:1789760733.686539:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 713146288478 time 713178804380 state 0x2 ref 0 00000100:00100000:3.0:1789760733.689267:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760733.689309:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512608512 00000100:00100000:3.0:1789760733.689328:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512608512 00000100:00100000:3.0:1789760733.689378:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 713146288478 time 713178804380 state 0x0 ref 1 00000100:00100000:3.0:1789760733.689403:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 713179064796 time 713178804380 state 0x1 ref 1 00000100:00100000:3.0:1789760733.689428:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 713179064796 time 713253727388 state 0x0 ref 1 00000100:00100000:3.0:1789760733.689440:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac531e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512608512:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760733.689906:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23957504, mtime: 0x6aad94dd, atime: 0x0, ctime: 0x6aad94dd, blocks: 0xb807 00010000:00010000:3.0:1789760733.689916:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2e119 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.689937:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=713s v=(0 0) 00010000:00010000:3.0:1789760733.689942:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760733.689948:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2e119 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23957504->23961599] (req 23957504->23961599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.689962:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2e119 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23957504->23961599] (req 23957504->23961599) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760733.786731:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973168 is committed 00002000:00010000:3.0:1789760733.786932:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23957504 -> 23961600 00010000:00010000:3.0:1789760733.786949:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2e119 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23957504->23961599] (req 23957504->23961599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.786963:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2e119 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23957504->23961599] (req 23957504->23961599) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.786973:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2e119 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23957504->23961599] (req 23957504->23961599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.786981:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760733.786986:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2e119 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23957504->23961599] (req 23957504->23961599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.786996:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2e119 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23957504->23961599] (req 23957504->23961599) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.787018:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2e119 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23957504->23961599] (req 23957504->23961599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.787028:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2e119 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23957504->23961599] (req 23957504->23961599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760733.789461:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac531e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512608512:12345-192.168.203.56@tcp:4:dd.0 Request processed in 100012us (100195us total) trans 4294973168 rc 0/0 00000100:00100000:3.0:1789760733.789476:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5879 00000100:00100000:3.0:1789760733.789486:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 713179064796 time 713253727388 state 0x2 ref 0 00000100:00100000:3.0:1789760733.792383:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760733.792465:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512608640 00000100:00100000:2.0:1789760733.792475:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512608640 00000100:00100000:2.0:1789760733.792531:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 713179064796 time 713253727388 state 0x0 ref 1 00000100:00100000:2.0:1789760733.792546:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 713253987804 time 713253727388 state 0x1 ref 1 00000100:00100000:2.0:1789760733.792571:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 713253987804 time 713356871048 state 0x0 ref 1 00000100:00100000:2.0:1789760733.792584:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac531fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512608640:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760733.792633:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23961600, mtime: 0x6aad94dd, atime: 0x0, ctime: 0x6aad94dd, blocks: 0xb807 00010000:00010000:2.0:1789760733.792642:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2e120 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.792658:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=713s v=(0 0) 00010000:00010000:2.0:1789760733.792663:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760733.792669:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2e120 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23961600->23965695] (req 23961600->23965695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.792683:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2e120 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23961600->23965695] (req 23961600->23965695) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760733.845412:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973169 is committed 00002000:00010000:2.0:1789760733.845612:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23961600 -> 23965696 00010000:00010000:2.0:1789760733.845627:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2e120 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23961600->23965695] (req 23961600->23965695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.845640:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2e120 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23961600->23965695] (req 23961600->23965695) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.845649:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2e120 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23961600->23965695] (req 23961600->23965695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.845655:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760733.845660:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2e120 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23961600->23965695] (req 23961600->23965695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.845668:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2e120 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23961600->23965695] (req 23961600->23965695) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.845687:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2e120 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23961600->23965695] (req 23961600->23965695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.845714:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2e120 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23961600->23965695] (req 23961600->23965695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760733.845925:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac531fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512608640:12345-192.168.203.56@tcp:4:dd.0 Request processed in 53335us (53546us total) trans 4294973169 rc 0/0 00000100:00100000:2.0:1789760733.845935:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5880 00000100:00100000:2.0:1789760733.845944:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 713253987804 time 713356871048 state 0x2 ref 0 00000100:00100000:0.0:1789760733.850694:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760733.852875:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512608768 00000100:00100000:3.0:1789760733.852889:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512608768 00000100:00100000:2.0:1789760733.853243:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 713253987804 time 713356871048 state 0x0 ref 1 00000100:00100000:2.0:1789760733.853259:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 713357131464 time 713356871048 state 0x1 ref 1 00000100:00100000:2.0:1789760733.853294:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 713357131464 time 713417593103 state 0x0 ref 1 00000100:00100000:2.0:1789760733.853307:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2878a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512608768:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760733.853359:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23965696, mtime: 0x6aad94dd, atime: 0x0, ctime: 0x6aad94dd, blocks: 0xb807 00010000:00010000:2.0:1789760733.853368:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2e127 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.853384:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=713s v=(0 0) 00010000:00010000:2.0:1789760733.853388:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760733.853406:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2e127 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23965696->23969791] (req 23965696->23969791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.853420:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2e127 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23965696->23969791] (req 23965696->23969791) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760733.899721:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23965696 -> 23969792 00010000:00010000:2.0:1789760733.899742:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2e127 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23965696->23969791] (req 23965696->23969791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.899755:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2e127 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23965696->23969791] (req 23965696->23969791) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.899765:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2e127 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23965696->23969791] (req 23965696->23969791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.899771:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760733.899776:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2e127 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23965696->23969791] (req 23965696->23969791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760733.899878:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973170 is committed 00010000:00010000:2.0:1789760733.899978:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2e127 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23965696->23969791] (req 23965696->23969791) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.899998:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2e127 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23965696->23969791] (req 23965696->23969791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.900007:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2e127 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23965696->23969791] (req 23965696->23969791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760733.900109:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2878a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512608768:12345-192.168.203.56@tcp:4:dd.0 Request processed in 46800us (49420us total) trans 4294973170 rc 0/0 00000100:00100000:2.0:1789760733.900119:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5881 00000100:00100000:2.0:1789760733.900128:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 713357131464 time 713417593103 state 0x2 ref 0 00000100:00100000:0.0:1789760733.903324:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760733.903431:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512608896 00000100:00100000:3.0:1789760733.903441:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512608896 00000100:00100000:3.0:1789760733.903567:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 713357131464 time 713417593103 state 0x0 ref 1 00000100:00100000:3.0:1789760733.903583:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 713417853519 time 713417593103 state 0x1 ref 1 00000100:00100000:3.0:1789760733.903648:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 713417853519 time 713467946701 state 0x0 ref 1 00000100:00100000:3.0:1789760733.903661:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac287ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512608896:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760733.903736:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23969792, mtime: 0x6aad94dd, atime: 0x0, ctime: 0x6aad94dd, blocks: 0xb807 00010000:00010000:3.0:1789760733.903746:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2e12e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.903762:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=713s v=(0 0) 00010000:00010000:3.0:1789760733.903768:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760733.903775:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2e12e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23969792->23973887] (req 23969792->23973887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.903887:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2e12e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23969792->23973887] (req 23969792->23973887) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760733.955988:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973171 is committed 00002000:00010000:2.0:1789760733.956280:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23969792 -> 23973888 00010000:00010000:2.0:1789760733.956296:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2e12e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23969792->23973887] (req 23969792->23973887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.956308:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2e12e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23969792->23973887] (req 23969792->23973887) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.956316:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2e12e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23969792->23973887] (req 23969792->23973887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.956323:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760733.956327:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2e12e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23969792->23973887] (req 23969792->23973887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.956337:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2e12e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23969792->23973887] (req 23969792->23973887) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.956355:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2e12e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23969792->23973887] (req 23969792->23973887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.956362:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2e12e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23969792->23973887] (req 23969792->23973887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760733.956522:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac287ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512608896:12345-192.168.203.56@tcp:4:dd.0 Request processed in 52858us (53203us total) trans 4294973171 rc 0/0 00000100:00100000:2.0:1789760733.956534:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5882 00000100:00100000:2.0:1789760733.956545:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 713417853519 time 713467946701 state 0x2 ref 0 00000100:00100000:0.0:1789760733.959639:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760733.959749:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512609024 00000100:00100000:2.0:1789760733.959758:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512609024 00000100:00100000:2.0:1789760733.959909:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 713417853519 time 713467946701 state 0x0 ref 1 00000100:00100000:2.0:1789760733.959925:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 713468207117 time 713467946701 state 0x1 ref 1 00000100:00100000:2.0:1789760733.959998:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 713468207117 time 713524297165 state 0x0 ref 1 00000100:00100000:2.0:1789760733.960011:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac287a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512609024:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760733.960062:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23973888, mtime: 0x6aad94dd, atime: 0x0, ctime: 0x6aad94dd, blocks: 0xb807 00010000:00010000:2.0:1789760733.960070:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2e135 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.960085:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=713s v=(0 0) 00010000:00010000:2.0:1789760733.960089:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760733.960095:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2e135 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23973888->23977983] (req 23973888->23977983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.960107:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2e135 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23973888->23977983] (req 23973888->23977983) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760733.991269:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973172 is committed 00002000:00010000:2.0:1789760733.991355:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23973888 -> 23977984 00010000:00010000:2.0:1789760733.991372:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2e135 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23973888->23977983] (req 23973888->23977983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.991460:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2e135 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23973888->23977983] (req 23973888->23977983) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.991472:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2e135 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23973888->23977983] (req 23973888->23977983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.991480:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760733.991484:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2e135 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23973888->23977983] (req 23973888->23977983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.991494:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2e135 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23973888->23977983] (req 23973888->23977983) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.991514:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2e135 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23973888->23977983] (req 23973888->23977983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.991521:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2e135 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23973888->23977983] (req 23973888->23977983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760733.991657:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac287a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512609024:12345-192.168.203.56@tcp:4:dd.0 Request processed in 31642us (32021us total) trans 4294973172 rc 0/0 00000100:00100000:2.0:1789760733.991668:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5883 00000100:00100000:2.0:1789760733.991677:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 713468207117 time 713524297165 state 0x2 ref 0 00000100:00100000:2.0:1789760733.995152:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760733.995245:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512609152 00000100:00100000:0.0:1789760733.995255:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512609152 00000100:00100000:0.0:1789760733.995311:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 713468207117 time 713524297165 state 0x0 ref 1 00000100:00100000:0.0:1789760733.995325:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 713524557581 time 713524297165 state 0x1 ref 1 00000100:00100000:0.0:1789760733.995355:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 713524557581 time 713559654969 state 0x0 ref 1 00000100:00100000:0.0:1789760733.995368:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febea651500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512609152:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760733.995425:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23977984, mtime: 0x6aad94dd, atime: 0x0, ctime: 0x6aad94dd, blocks: 0xb807 00010000:00010000:0.0:1789760733.995434:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2e13c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.995449:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=713s v=(0 0) 00010000:00010000:0.0:1789760733.995453:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760733.995460:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2e13c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23977984->23982079] (req 23977984->23982079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.995473:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2e13c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23977984->23982079] (req 23977984->23982079) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760734.025804:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23977984 -> 23982080 00010000:00010000:0.0:1789760734.025823:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2e13c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23977984->23982079] (req 23977984->23982079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760734.025843:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2e13c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23977984->23982079] (req 23977984->23982079) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760734.025852:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2e13c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23977984->23982079] (req 23977984->23982079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760734.025859:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760734.025864:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2e13c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23977984->23982079] (req 23977984->23982079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760734.025873:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2e13c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23977984->23982079] (req 23977984->23982079) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760734.025891:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2e13c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23977984->23982079] (req 23977984->23982079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760734.025899:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2e13c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23977984->23982079] (req 23977984->23982079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760734.025987:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febea651500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2360:x1876699512609152:12345-192.168.203.56@tcp:4:dd.0 Request processed in 30616us (30840us total) trans 4294973173 rc 0/0 00000100:00100000:0.0:1789760734.025997:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5884 00000100:00100000:0.0:1789760734.026006:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 713524557581 time 713559654969 state 0x2 ref 0 00000001:00080000:1.0:1789760734.026741:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973173 is committed 00000100:00100000:0.0:1789760734.028740:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760734.028963:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512609280 00000100:00100000:2.0:1789760734.028972:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512609280 00000100:00100000:2.0:1789760734.029028:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 713524557581 time 713559654969 state 0x0 ref 1 00000100:00100000:2.0:1789760734.029044:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 713559915385 time 713559654969 state 0x1 ref 1 00000100:00100000:2.0:1789760734.029154:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 713559915385 time 713593452069 state 0x0 ref 1 00000100:00100000:2.0:1789760734.029172:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2879c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512609280:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760734.029217:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23982080, mtime: 0x6aad94dd, atime: 0x0, ctime: 0x6aad94dd, blocks: 0xb807 00010000:00010000:2.0:1789760734.029226:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2e143 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.029241:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=713s v=(0 0) 00010000:00010000:2.0:1789760734.029246:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760734.029252:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2e143 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23982080->23986175] (req 23982080->23986175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.029266:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2e143 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23982080->23986175] (req 23982080->23986175) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760734.057030:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973174 is committed 00002000:00010000:0.0:1789760734.057350:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23982080 -> 23986176 00010000:00010000:0.0:1789760734.057365:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2e143 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23982080->23986175] (req 23982080->23986175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760734.057378:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2e143 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23982080->23986175] (req 23982080->23986175) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760734.057386:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2e143 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23982080->23986175] (req 23982080->23986175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760734.057406:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760734.057411:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2e143 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23982080->23986175] (req 23982080->23986175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760734.057420:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2e143 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23982080->23986175] (req 23982080->23986175) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760734.057438:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2e143 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23982080->23986175] (req 23982080->23986175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760734.057446:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2e143 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23982080->23986175] (req 23982080->23986175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760734.057542:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2879c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512609280:12345-192.168.203.56@tcp:4:dd.0 Request processed in 28370us (28805us total) trans 4294973174 rc 0/0 00000100:00100000:0.0:1789760734.057553:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5885 00000100:00100000:0.0:1789760734.057563:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 713559915385 time 713593452069 state 0x2 ref 0 00000100:00100000:2.0:1789760734.060800:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760734.060932:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512609408 00000100:00100000:0.0:1789760734.060942:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512609408 00000100:00100000:0.0:1789760734.060993:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 713559915385 time 713593452069 state 0x0 ref 1 00000100:00100000:0.0:1789760734.061008:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 713593712485 time 713593452069 state 0x1 ref 1 00000100:00100000:0.0:1789760734.061078:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 713593712485 time 713625377427 state 0x0 ref 1 00000100:00100000:0.0:1789760734.061091:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febea650380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512609408:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760734.061135:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23986176, mtime: 0x6aad94dd, atime: 0x0, ctime: 0x6aad94dd, blocks: 0xb807 00010000:00010000:0.0:1789760734.061144:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2e14a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760734.061160:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=713s v=(0 0) 00010000:00010000:0.0:1789760734.061165:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760734.061171:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2e14a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23986176->23990271] (req 23986176->23990271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760734.061185:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2e14a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23986176->23990271] (req 23986176->23990271) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760734.097855:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23986176 -> 23990272 00010000:00010000:0.0:1789760734.097877:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2e14a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23986176->23990271] (req 23986176->23990271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760734.097889:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2e14a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23986176->23990271] (req 23986176->23990271) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760734.097900:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2e14a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23986176->23990271] (req 23986176->23990271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760734.097908:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760734.097913:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2e14a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23986176->23990271] (req 23986176->23990271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760734.097923:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2e14a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23986176->23990271] (req 23986176->23990271) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760734.097945:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2e14a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23986176->23990271] (req 23986176->23990271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760734.097953:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2e14a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23986176->23990271] (req 23986176->23990271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760734.098050:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febea650380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2359:x1876699512609408:12345-192.168.203.56@tcp:4:dd.0 Request processed in 36956us (37263us total) trans 4294973175 rc 0/0 00000100:00100000:0.0:1789760734.098069:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5886 00000100:00100000:0.0:1789760734.098079:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 713593712485 time 713625377427 state 0x2 ref 0 00000001:00080000:0.0:1789760734.098118:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973175 is committed 00000100:00100000:3.0:1789760734.100889:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760734.101002:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512609536 00000100:00100000:0.0:1789760734.101012:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512609536 00000100:00100000:0.0:1789760734.101065:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 713593712485 time 713625377427 state 0x0 ref 1 00000100:00100000:0.0:1789760734.101085:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 713625637843 time 713625377427 state 0x1 ref 1 00000100:00100000:0.0:1789760734.101117:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 713625637843 time 713665416056 state 0x0 ref 1 00000100:00100000:0.0:1789760734.101131:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac531f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512609536:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760734.101187:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23990272, mtime: 0x6aad94dd, atime: 0x0, ctime: 0x6aad94dd, blocks: 0xb807 00010000:00010000:0.0:1789760734.101196:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2e151 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760734.101212:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=713s v=(0 0) 00010000:00010000:0.0:1789760734.101216:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760734.101223:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2e151 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23990272->23994367] (req 23990272->23994367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760734.101238:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2e151 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23990272->23994367] (req 23990272->23994367) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760734.126690:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973176 is committed 00002000:00010000:3.0:1789760734.126857:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23990272 -> 23994368 00010000:00010000:3.0:1789760734.126877:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2e151 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23990272->23994367] (req 23990272->23994367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.126889:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2e151 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23990272->23994367] (req 23990272->23994367) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.126898:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2e151 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23990272->23994367] (req 23990272->23994367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.126905:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760734.126910:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2e151 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23990272->23994367] (req 23990272->23994367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.126919:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2e151 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23990272->23994367] (req 23990272->23994367) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.126958:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2e151 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23990272->23994367] (req 23990272->23994367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.126971:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2e151 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23990272->23994367] (req 23990272->23994367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760734.127078:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac531f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512609536:12345-192.168.203.56@tcp:4:dd.0 Request processed in 25946us (26193us total) trans 4294973176 rc 0/0 00000100:00100000:3.0:1789760734.127089:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5887 00000100:00100000:3.0:1789760734.127101:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 713625637843 time 713665416056 state 0x2 ref 0 00000100:00100000:1.0:1789760734.132250:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760734.132422:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512609664 00000100:00100000:0.0:1789760734.132433:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512609664 00000100:00100000:0.0:1789760734.132614:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 713625637843 time 713665416056 state 0x0 ref 1 00000100:00100000:0.0:1789760734.132630:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 713665676472 time 713665416056 state 0x1 ref 1 00000100:00100000:0.0:1789760734.132659:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 713665676472 time 713696959159 state 0x0 ref 1 00000100:00100000:0.0:1789760734.133060:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4133480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512609664:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760734.133124:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23994368, mtime: 0x6aad94dd, atime: 0x0, ctime: 0x6aad94dd, blocks: 0xb807 00010000:00010000:0.0:1789760734.133133:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2e158 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760734.133159:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=713s v=(0 0) 00010000:00010000:0.0:1789760734.133165:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760734.133172:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2e158 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23994368->23998463] (req 23994368->23998463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760734.133186:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2e158 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23994368->23998463] (req 23994368->23998463) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760734.180887:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973177 is committed 00002000:00010000:3.0:1789760734.180996:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23994368 -> 23998464 00010000:00010000:3.0:1789760734.181012:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2e158 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23994368->23998463] (req 23994368->23998463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.181024:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2e158 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23994368->23998463] (req 23994368->23998463) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.181032:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2e158 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23994368->23998463] (req 23994368->23998463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.181038:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760734.181043:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2e158 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23994368->23998463] (req 23994368->23998463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.181051:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2e158 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23994368->23998463] (req 23994368->23998463) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.181086:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2e158 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23994368->23998463] (req 23994368->23998463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.181094:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2e158 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23994368->23998463] (req 23994368->23998463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760734.181183:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4133480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512609664:12345-192.168.203.56@tcp:4:dd.0 Request processed in 48505us (48941us total) trans 4294973177 rc 0/0 00000100:00100000:3.0:1789760734.181193:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5888 00000100:00100000:3.0:1789760734.181202:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 713665676472 time 713696959159 state 0x2 ref 0 00000100:00100000:0.0:1789760734.183875:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760734.184113:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512609792 00000100:00100000:2.0:1789760734.184123:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512609792 00000100:00100000:2.0:1789760734.184176:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 713665676472 time 713696959159 state 0x0 ref 1 00000100:00100000:2.0:1789760734.184190:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 713697219575 time 713696959159 state 0x1 ref 1 00000100:00100000:2.0:1789760734.184293:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 713697219575 time 713748591498 state 0x0 ref 1 00000100:00100000:2.0:1789760734.184306:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2878e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512609792:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760734.184356:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23998464, mtime: 0x6aad94dd, atime: 0x0, ctime: 0x6aad94dd, blocks: 0xb807 00010000:00010000:2.0:1789760734.184375:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2e15f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.184398:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=713s v=(0 0) 00010000:00010000:2.0:1789760734.184402:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760734.184407:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2e15f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23998464->24002559] (req 23998464->24002559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.184422:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2e15f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23998464->24002559] (req 23998464->24002559) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760734.211947:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23998464 -> 24002560 00010000:00010000:2.0:1789760734.211967:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2e15f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23998464->24002559] (req 23998464->24002559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.211980:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2e15f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23998464->24002559] (req 23998464->24002559) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.211989:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2e15f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23998464->24002559] (req 23998464->24002559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.211996:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760734.212002:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2e15f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23998464->24002559] (req 23998464->24002559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.212011:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2e15f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23998464->24002559] (req 23998464->24002559) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.212031:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2e15f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23998464->24002559] (req 23998464->24002559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.212038:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2e15f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23998464->24002559] (req 23998464->24002559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760734.212140:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2878e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2358:x1876699512609792:12345-192.168.203.56@tcp:4:dd.0 Request processed in 27830us (28267us total) trans 4294973178 rc 0/0 00000100:00100000:2.0:1789760734.212151:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5889 00000100:00100000:2.0:1789760734.212161:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 713697219575 time 713748591498 state 0x2 ref 0 00000001:00080000:3.0:1789760734.213588:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973178 is committed 00000100:00100000:1.0:1789760734.216606:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760734.216671:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512609920 00000100:00100000:3.0:1789760734.216679:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512609920 00000100:00100000:3.0:1789760734.216757:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 713697219575 time 713748591498 state 0x0 ref 1 00000100:00100000:3.0:1789760734.216770:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 713748851914 time 713748591498 state 0x1 ref 1 00000100:00100000:3.0:1789760734.216898:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 713748851914 time 713781197421 state 0x0 ref 1 00000100:00100000:3.0:1789760734.216909:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4130700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512609920:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760734.216949:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24002560, mtime: 0x6aad94dd, atime: 0x0, ctime: 0x6aad94dd, blocks: 0xb807 00010000:00010000:3.0:1789760734.216956:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2e166 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.216977:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=713s v=(0 0) 00010000:00010000:3.0:1789760734.216982:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760734.216988:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2e166 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24002560->24006655] (req 24002560->24006655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.217001:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2e166 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24002560->24006655] (req 24002560->24006655) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760734.249567:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24002560 -> 24006656 00010000:00010000:3.0:1789760734.249588:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2e166 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24002560->24006655] (req 24002560->24006655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.249601:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2e166 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24002560->24006655] (req 24002560->24006655) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.249611:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2e166 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24002560->24006655] (req 24002560->24006655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.249618:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760734.249623:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2e166 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24002560->24006655] (req 24002560->24006655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.249631:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2e166 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24002560->24006655] (req 24002560->24006655) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.249648:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2e166 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24002560->24006655] (req 24002560->24006655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.249656:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2e166 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24002560->24006655] (req 24002560->24006655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760734.249791:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4130700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2359:x1876699512609920:12345-192.168.203.56@tcp:4:dd.0 Request processed in 32872us (33181us total) trans 4294973179 rc 0/0 00000100:00100000:3.0:1789760734.249802:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5890 00000100:00100000:3.0:1789760734.249812:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 713748851914 time 713781197421 state 0x2 ref 0 00000001:00080000:2.0:1789760734.250629:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973179 is committed 00000100:00100000:1.0:1789760734.252451:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760734.252509:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512610048 00000100:00100000:3.0:1789760734.252519:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512610048 00000100:00100000:3.0:1789760734.252574:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 713748851914 time 713781197421 state 0x0 ref 1 00000100:00100000:3.0:1789760734.252590:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 713781457837 time 713781197421 state 0x1 ref 1 00000100:00100000:3.0:1789760734.252624:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 713781457837 time 713816923877 state 0x0 ref 1 00000100:00100000:3.0:1789760734.252638:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4131180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512610048:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760734.252688:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24006656, mtime: 0x6aad94dd, atime: 0x0, ctime: 0x6aad94dd, blocks: 0xb807 00010000:00010000:3.0:1789760734.252722:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2e16d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.252738:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=713s v=(0 0) 00010000:00010000:3.0:1789760734.252742:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760734.252748:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2e16d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24006656->24010751] (req 24006656->24010751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.252768:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2e16d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24006656->24010751] (req 24006656->24010751) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760734.301543:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973180 is committed 00002000:00010000:3.0:1789760734.301563:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24006656 -> 24010752 00010000:00010000:3.0:1789760734.301579:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2e16d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24006656->24010751] (req 24006656->24010751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.301593:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2e16d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24006656->24010751] (req 24006656->24010751) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.301603:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2e16d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24006656->24010751] (req 24006656->24010751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.301610:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760734.301615:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2e16d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24006656->24010751] (req 24006656->24010751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.301622:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2e16d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24006656->24010751] (req 24006656->24010751) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.301636:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2e16d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24006656->24010751] (req 24006656->24010751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.301640:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2e16d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24006656->24010751] (req 24006656->24010751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760734.301763:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4131180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512610048:12345-192.168.203.56@tcp:4:dd.0 Request processed in 49123us (49316us total) trans 4294973180 rc 0/0 00000100:00100000:3.0:1789760734.301773:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5891 00000100:00100000:3.0:1789760734.301782:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 713781457837 time 713816923877 state 0x2 ref 0 00000100:00100000:1.0:1789760734.304118:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760734.304209:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512610176 00000100:00100000:2.0:1789760734.304218:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512610176 00000100:00100000:2.0:1789760734.304270:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 713781457837 time 713816923877 state 0x0 ref 1 00000100:00100000:2.0:1789760734.304284:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 713817184293 time 713816923877 state 0x1 ref 1 00000100:00100000:2.0:1789760734.304315:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 713817184293 time 713868615276 state 0x0 ref 1 00000100:00100000:2.0:1789760734.304327:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4130a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512610176:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760734.304373:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24010752, mtime: 0x6aad94dd, atime: 0x0, ctime: 0x6aad94dd, blocks: 0xb807 00010000:00010000:2.0:1789760734.304382:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2e174 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.304412:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=713s v=(0 0) 00010000:00010000:2.0:1789760734.304417:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760734.304592:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2e174 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24010752->24014847] (req 24010752->24014847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.304608:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2e174 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24010752->24014847] (req 24010752->24014847) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760734.350200:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973181 is committed 00002000:00010000:2.0:1789760734.350299:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24010752 -> 24014848 00010000:00010000:2.0:1789760734.350314:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2e174 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24010752->24014847] (req 24010752->24014847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.350328:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2e174 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24010752->24014847] (req 24010752->24014847) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.350337:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2e174 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24010752->24014847] (req 24010752->24014847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.350344:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760734.350349:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2e174 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24010752->24014847] (req 24010752->24014847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.350358:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2e174 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24010752->24014847] (req 24010752->24014847) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.350377:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2e174 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24010752->24014847] (req 24010752->24014847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.350385:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2e174 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24010752->24014847] (req 24010752->24014847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760734.350612:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4130a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512610176:12345-192.168.203.56@tcp:4:dd.0 Request processed in 46281us (46498us total) trans 4294973181 rc 0/0 00000100:00100000:2.0:1789760734.350622:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5892 00000100:00100000:2.0:1789760734.350632:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 713817184293 time 713868615276 state 0x2 ref 0 00000100:00100000:1.0:1789760734.353290:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760734.353396:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512610304 00000100:00100000:3.0:1789760734.353408:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512610304 00000100:00100000:3.0:1789760734.353460:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 713817184293 time 713868615276 state 0x0 ref 1 00000100:00100000:3.0:1789760734.353475:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 713868875692 time 713868615276 state 0x1 ref 1 00000100:00100000:3.0:1789760734.353507:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 713868875692 time 713917806714 state 0x0 ref 1 00000100:00100000:3.0:1789760734.353519:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4131500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512610304:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760734.353565:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24014848, mtime: 0x6aad94dd, atime: 0x0, ctime: 0x6aad94dd, blocks: 0xb807 00010000:00010000:3.0:1789760734.353574:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2e17b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.353592:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=713s v=(0 0) 00010000:00010000:3.0:1789760734.353597:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760734.353603:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2e17b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24014848->24018943] (req 24014848->24018943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.353618:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2e17b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24014848->24018943] (req 24014848->24018943) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760734.397772:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973182 is committed 00002000:00010000:1.0:1789760734.397875:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24014848 -> 24018944 00010000:00010000:1.0:1789760734.397889:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2e17b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24014848->24018943] (req 24014848->24018943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.397902:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2e17b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24014848->24018943] (req 24014848->24018943) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.397912:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2e17b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24014848->24018943] (req 24014848->24018943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.397919:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760734.397925:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2e17b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24014848->24018943] (req 24014848->24018943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.397933:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2e17b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24014848->24018943] (req 24014848->24018943) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.397953:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2e17b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24014848->24018943] (req 24014848->24018943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.397960:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2e17b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24014848->24018943] (req 24014848->24018943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760734.398056:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4131500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512610304:12345-192.168.203.56@tcp:4:dd.0 Request processed in 44533us (44769us total) trans 4294973182 rc 0/0 00000100:00100000:1.0:1789760734.398973:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5893 00000100:00100000:1.0:1789760734.398983:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 713868875692 time 713917806714 state 0x2 ref 0 00000100:00100000:0.0:1789760734.403800:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760734.404312:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512610432 00000100:00100000:3.0:1789760734.404323:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512610432 00000100:00100000:3.0:1789760734.404385:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 713868875692 time 713917806714 state 0x0 ref 1 00000100:00100000:3.0:1789760734.404412:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 713918067130 time 713917806714 state 0x1 ref 1 00000100:00100000:3.0:1789760734.404481:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 713918067130 time 713968780305 state 0x0 ref 1 00000100:00100000:3.0:1789760734.404495:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac287b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512610432:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760734.404545:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24018944, mtime: 0x6aad94dd, atime: 0x0, ctime: 0x6aad94dd, blocks: 0xb807 00010000:00010000:3.0:1789760734.404554:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2e182 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.404570:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=713s v=(0 0) 00010000:00010000:3.0:1789760734.404574:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760734.404581:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2e182 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24018944->24023039] (req 24018944->24023039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.404595:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2e182 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24018944->24023039] (req 24018944->24023039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760734.457459:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24018944 -> 24023040 00010000:00010000:1.0:1789760734.457480:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2e182 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24018944->24023039] (req 24018944->24023039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.457494:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2e182 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24018944->24023039] (req 24018944->24023039) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.457504:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2e182 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24018944->24023039] (req 24018944->24023039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.457512:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760734.457518:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2e182 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24018944->24023039] (req 24018944->24023039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.457526:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2e182 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24018944->24023039] (req 24018944->24023039) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.457547:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2e182 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24018944->24023039] (req 24018944->24023039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760734.457737:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973183 is committed 00010000:00010000:1.0:1789760734.457834:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2e182 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24018944->24023039] (req 24018944->24023039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760734.457918:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac287b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512610432:12345-192.168.203.56@tcp:4:dd.0 Request processed in 53421us (54133us total) trans 4294973183 rc 0/0 00000100:00100000:1.0:1789760734.457930:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5894 00000100:00100000:1.0:1789760734.457940:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 713918067130 time 713968780305 state 0x2 ref 0 00000100:00100000:3.0:1789760734.461059:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760734.461226:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512610560 00000100:00100000:2.0:1789760734.461236:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512610560 00000100:00100000:2.0:1789760734.461286:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 713918067130 time 713968780305 state 0x0 ref 1 00000100:00100000:2.0:1789760734.461301:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 713969040721 time 713968780305 state 0x1 ref 1 00000100:00100000:2.0:1789760734.461369:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 713969040721 time 714025668629 state 0x0 ref 1 00000100:00100000:2.0:1789760734.461382:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac531c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512610560:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760734.461440:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24023040, mtime: 0x6aad94dd, atime: 0x0, ctime: 0x6aad94dd, blocks: 0xb807 00010000:00010000:2.0:1789760734.461449:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2e189 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.461465:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=714s v=(0 0) 00010000:00010000:2.0:1789760734.461470:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760734.461486:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2e189 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24023040->24027135] (req 24023040->24027135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.461501:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2e189 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24023040->24027135] (req 24023040->24027135) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760734.472672:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760734.472735:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760734.472790:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760734.472830:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512610688 00000100:00100000:2.0:1789760734.472839:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512610688 00000100:00100000:2.0:1789760734.472886:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 394 00000100:00100000:2.0:1789760734.472896:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febe0503c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:e6cf7267-9b19-462a-a263-14115df15dab+13:2361:x1876699512610688:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:2.0:1789760734.472962:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febe0503c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:e6cf7267-9b19-462a-a263-14115df15dab+13:2361:x1876699512610688:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 67us (226us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760734.472970:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 394 00000100:00100000:2.0:1789760734.473003:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512610816 00000100:00100000:2.0:1789760734.473007:0:5847:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512610816 00000100:00100000:2.0:1789760734.473041:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512610944 00000100:00100000:2.0:1789760734.473043:0:5847:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512610944 00000100:00100000:2.0:1789760734.473081:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 464 00000100:00100000:2.0:1789760734.473088:0:5847:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febffb9d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+8:2360:x1876699512610816:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:2.0:1789760734.473120:0:5847:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febffb9d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+8:2360:x1876699512610816:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 33us (461us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760734.473127:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 464 00000100:00100000:2.0:1789760734.473138:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 465 00000100:00100000:2.0:1789760734.473146:0:5847:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febffb9f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:d2c40150-e97b-40da-952b-cb9f18bf7e96+5:2361:x1876699512610944:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:2.0:1789760734.473174:0:5847:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febffb9f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:d2c40150-e97b-40da-952b-cb9f18bf7e96+5:2361:x1876699512610944:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 29us (398us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760734.473180:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 465 00000100:00080000:2.0:1789760734.473298:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000001:00080000:3.0:1789760734.532583:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973184 is committed 00002000:00010000:1.0:1789760734.532665:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24023040 -> 24027136 00010000:00010000:1.0:1789760734.532680:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2e189 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24023040->24027135] (req 24023040->24027135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.532693:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2e189 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24023040->24027135] (req 24023040->24027135) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.532727:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2e189 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24023040->24027135] (req 24023040->24027135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.532740:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760734.532745:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2e189 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24023040->24027135] (req 24023040->24027135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.532753:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2e189 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24023040->24027135] (req 24023040->24027135) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.532771:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2e189 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24023040->24027135] (req 24023040->24027135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.532777:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2e189 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24023040->24027135] (req 24023040->24027135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760734.532867:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac531c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512610560:12345-192.168.203.56@tcp:4:dd.0 Request processed in 71483us (71813us total) trans 4294973184 rc 0/0 00000100:00100000:1.0:1789760734.532876:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5895 00000100:00100000:1.0:1789760734.532887:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 713969040721 time 714025668629 state 0x2 ref 0 00000100:00100000:2.0:1789760734.537157:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760734.537247:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512611072 00000100:00100000:1.0:1789760734.537258:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512611072 00000100:00100000:1.0:1789760734.537313:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 713969040721 time 714025668629 state 0x0 ref 1 00000100:00100000:1.0:1789760734.537328:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 714025929045 time 714025668629 state 0x1 ref 1 00000100:00100000:1.0:1789760734.537370:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 714025929045 time 714101669606 state 0x0 ref 1 00000100:00100000:1.0:1789760734.537394:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febffb9f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512611072:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760734.537607:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24027136, mtime: 0x6aad94dd, atime: 0x0, ctime: 0x6aad94dd, blocks: 0xb807 00010000:00010000:1.0:1789760734.537616:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2e190 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.537631:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=714s v=(0 0) 00010000:00010000:1.0:1789760734.537636:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760734.537643:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2e190 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24027136->24031231] (req 24027136->24031231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.537657:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2e190 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24027136->24031231] (req 24027136->24031231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.579031:0:7647:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:2.0:1789760734.583273:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973185 is committed 00002000:00010000:1.0:1789760734.583357:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24027136 -> 24031232 00002000:00010000:1.0:1789760734.583362:0:6710:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760733 -> 1789760734 00002000:00010000:1.0:1789760734.583364:0:6710:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760733 -> 1789760734 00010000:00010000:1.0:1789760734.583377:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2e190 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24027136->24031231] (req 24027136->24031231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.583401:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2e190 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24027136->24031231] (req 24027136->24031231) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.583419:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2e190 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24027136->24031231] (req 24027136->24031231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.583426:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760734.583432:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2e190 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24027136->24031231] (req 24027136->24031231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.583439:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2e190 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24027136->24031231] (req 24027136->24031231) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.583457:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2e190 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24027136->24031231] (req 24027136->24031231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.583464:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2e190 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24027136->24031231] (req 24027136->24031231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760734.583603:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febffb9f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512611072:12345-192.168.203.56@tcp:4:dd.0 Request processed in 46215us (46451us total) trans 4294973185 rc 0/0 00000100:00100000:1.0:1789760734.583614:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5896 00000100:00100000:1.0:1789760734.583625:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 714025929045 time 714101669606 state 0x2 ref 0 00000100:00100000:3.0:1789760734.590446:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760734.592256:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512611200 00000100:00100000:1.0:1789760734.592271:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512611200 00000100:00100000:1.0:1789760734.592330:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 714025929045 time 714101669606 state 0x0 ref 1 00000100:00100000:1.0:1789760734.592346:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 714101930022 time 714101669606 state 0x1 ref 1 00000100:00100000:1.0:1789760734.592445:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 714101930022 time 714156744228 state 0x0 ref 1 00000100:00100000:1.0:1789760734.592459:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febea4b6c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512611200:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760734.592509:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24031232, mtime: 0x6aad94de, atime: 0x0, ctime: 0x6aad94de, blocks: 0xb807 00010000:00010000:1.0:1789760734.592519:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2e197 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.592536:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=714s v=(0 0) 00010000:00010000:1.0:1789760734.592540:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760734.592546:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2e197 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24031232->24035327] (req 24031232->24035327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.592560:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2e197 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24031232->24035327] (req 24031232->24035327) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760734.638400:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24031232 -> 24035328 00010000:00010000:1.0:1789760734.638427:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2e197 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24031232->24035327] (req 24031232->24035327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.638440:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2e197 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24031232->24035327] (req 24031232->24035327) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.638449:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2e197 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24031232->24035327] (req 24031232->24035327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.638455:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760734.638460:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2e197 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24031232->24035327] (req 24031232->24035327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.638468:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2e197 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24031232->24035327] (req 24031232->24035327) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.638491:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2e197 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24031232->24035327] (req 24031232->24035327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.638498:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2e197 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24031232->24035327] (req 24031232->24035327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760734.638742:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febea4b6c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2358:x1876699512611200:12345-192.168.203.56@tcp:4:dd.0 Request processed in 46240us (48259us total) trans 4294973186 rc 0/0 00000100:00100000:1.0:1789760734.638753:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5897 00000100:00100000:1.0:1789760734.638763:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 714101930022 time 714156744228 state 0x2 ref 0 00000001:00080000:2.0:1789760734.639342:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973186 is committed 00000100:00100000:2.0:1789760734.642216:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760734.642427:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512611328 00000100:00100000:1.0:1789760734.642437:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512611328 00000100:00100000:1.0:1789760734.642486:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 714101930022 time 714156744228 state 0x0 ref 1 00000100:00100000:1.0:1789760734.642500:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 714157004644 time 714156744228 state 0x1 ref 1 00000100:00100000:1.0:1789760734.642532:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 714157004644 time 714206831951 state 0x0 ref 1 00000100:00100000:1.0:1789760734.642544:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febffb9f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512611328:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760734.642591:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24035328, mtime: 0x6aad94de, atime: 0x0, ctime: 0x6aad94de, blocks: 0xb807 00010000:00010000:1.0:1789760734.642600:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2e19e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.642616:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=714s v=(0 0) 00010000:00010000:1.0:1789760734.642621:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760734.642628:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2e19e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24035328->24039423] (req 24035328->24039423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.642642:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2e19e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24035328->24039423] (req 24035328->24039423) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760734.671974:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973187 is committed 00002000:00010000:0.0:1789760734.672253:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24035328 -> 24039424 00010000:00010000:0.0:1789760734.672270:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2e19e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24035328->24039423] (req 24035328->24039423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760734.672282:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2e19e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24035328->24039423] (req 24035328->24039423) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760734.672291:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2e19e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24035328->24039423] (req 24035328->24039423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760734.672299:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760734.672305:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2e19e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24035328->24039423] (req 24035328->24039423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760734.672314:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2e19e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24035328->24039423] (req 24035328->24039423) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760734.672335:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2e19e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24035328->24039423] (req 24035328->24039423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760734.672342:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2e19e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24035328->24039423] (req 24035328->24039423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760734.672471:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febffb9f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512611328:12345-192.168.203.56@tcp:4:dd.0 Request processed in 29924us (30261us total) trans 4294973187 rc 0/0 00000100:00100000:0.0:1789760734.672481:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5898 00000100:00100000:0.0:1789760734.672491:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 714157004644 time 714206831951 state 0x2 ref 0 00000100:00100000:0.0:1789760734.675602:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760734.676823:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512611456 00000100:00100000:1.0:1789760734.676837:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512611456 00000100:00100000:1.0:1789760734.676900:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 714157004644 time 714206831951 state 0x0 ref 1 00000100:00100000:1.0:1789760734.676914:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 714207092367 time 714206831951 state 0x1 ref 1 00000100:00100000:1.0:1789760734.677007:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 714207092367 time 714241305294 state 0x0 ref 1 00000100:00100000:1.0:1789760734.677021:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac287b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512611456:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760734.677079:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24039424, mtime: 0x6aad94de, atime: 0x0, ctime: 0x6aad94de, blocks: 0xb807 00010000:00010000:1.0:1789760734.677088:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2e1a5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.677103:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=714s v=(0 0) 00010000:00010000:1.0:1789760734.677107:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760734.677114:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2e1a5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24039424->24043519] (req 24039424->24043519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.677128:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2e1a5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24039424->24043519] (req 24039424->24043519) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760734.728860:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973188 is committed 00002000:00010000:1.0:1789760734.729041:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24039424 -> 24043520 00010000:00010000:1.0:1789760734.729068:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2e1a5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24039424->24043519] (req 24039424->24043519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.729082:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2e1a5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24039424->24043519] (req 24039424->24043519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.729092:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2e1a5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24039424->24043519] (req 24039424->24043519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.729101:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760734.729106:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2e1a5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24039424->24043519] (req 24039424->24043519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.729116:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2e1a5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24039424->24043519] (req 24039424->24043519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.729135:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2e1a5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24039424->24043519] (req 24039424->24043519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.729144:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2e1a5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24039424->24043519] (req 24039424->24043519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760734.729243:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac287b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512611456:12345-192.168.203.56@tcp:4:dd.0 Request processed in 52220us (53647us total) trans 4294973188 rc 0/0 00000100:00100000:1.0:1789760734.729258:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5899 00000100:00100000:1.0:1789760734.729269:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 714207092367 time 714241305294 state 0x2 ref 0 00000100:00100000:3.0:1789760734.731953:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760734.732024:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512611584 00000100:00100000:1.0:1789760734.732034:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512611584 00000100:00100000:1.0:1789760734.732098:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 714207092367 time 714241305294 state 0x0 ref 1 00000100:00100000:1.0:1789760734.732115:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 714241565710 time 714241305294 state 0x1 ref 1 00000100:00100000:1.0:1789760734.732147:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 714241565710 time 714296446389 state 0x0 ref 1 00000100:00100000:1.0:1789760734.732160:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac531ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512611584:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760734.732206:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24043520, mtime: 0x6aad94de, atime: 0x0, ctime: 0x6aad94de, blocks: 0xb807 00010000:00010000:1.0:1789760734.732215:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2e1ac lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.732232:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=714s v=(0 0) 00010000:00010000:1.0:1789760734.732237:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760734.732244:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2e1ac lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24043520->24047615] (req 24043520->24047615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.732258:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2e1ac lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24043520->24047615] (req 24043520->24047615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760734.763162:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24043520 -> 24047616 00010000:00010000:3.0:1789760734.763181:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2e1ac lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24043520->24047615] (req 24043520->24047615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.763195:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2e1ac lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24043520->24047615] (req 24043520->24047615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.763205:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2e1ac lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24043520->24047615] (req 24043520->24047615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.763212:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760734.763218:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2e1ac lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24043520->24047615] (req 24043520->24047615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.763225:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2e1ac lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24043520->24047615] (req 24043520->24047615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.763244:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2e1ac lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24043520->24047615] (req 24043520->24047615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.763252:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2e1ac lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24043520->24047615] (req 24043520->24047615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760734.763343:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac531ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2358:x1876699512611584:12345-192.168.203.56@tcp:4:dd.0 Request processed in 31179us (31394us total) trans 4294973189 rc 0/0 00000100:00100000:3.0:1789760734.763354:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5900 00000100:00100000:3.0:1789760734.763363:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 714241565710 time 714296446389 state 0x2 ref 0 00000001:00080000:1.0:1789760734.763439:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973189 is committed 00000100:00100000:3.0:1789760734.766990:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760734.767485:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512611712 00000100:00100000:0.0:1789760734.767496:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512611712 00000100:00100000:0.0:1789760734.767551:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 714241565710 time 714296446389 state 0x0 ref 1 00000100:00100000:0.0:1789760734.767568:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 714296706805 time 714296446389 state 0x1 ref 1 00000100:00100000:0.0:1789760734.767631:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 714296706805 time 714331930367 state 0x0 ref 1 00000100:00100000:0.0:1789760734.767644:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac531e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512611712:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760734.767716:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24047616, mtime: 0x6aad94de, atime: 0x0, ctime: 0x6aad94de, blocks: 0xb807 00010000:00010000:0.0:1789760734.767726:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2e1b3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760734.767742:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=714s v=(0 0) 00010000:00010000:0.0:1789760734.767747:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760734.767753:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2e1b3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24047616->24051711] (req 24047616->24051711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760734.767767:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2e1b3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24047616->24051711] (req 24047616->24051711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760734.819034:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973190 is committed 00002000:00010000:3.0:1789760734.819375:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24047616 -> 24051712 00010000:00010000:3.0:1789760734.819402:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2e1b3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24047616->24051711] (req 24047616->24051711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.819415:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2e1b3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24047616->24051711] (req 24047616->24051711) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.819423:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2e1b3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24047616->24051711] (req 24047616->24051711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.819431:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760734.819436:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2e1b3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24047616->24051711] (req 24047616->24051711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.819450:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2e1b3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24047616->24051711] (req 24047616->24051711) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.819547:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2e1b3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24047616->24051711] (req 24047616->24051711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.819562:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2e1b3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24047616->24051711] (req 24047616->24051711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760734.819755:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac531e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512611712:12345-192.168.203.56@tcp:4:dd.0 Request processed in 52107us (52770us total) trans 4294973190 rc 0/0 00000100:00100000:3.0:1789760734.819766:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5901 00000100:00100000:3.0:1789760734.819794:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 714296706805 time 714331930367 state 0x2 ref 0 00000100:00100000:0.0:1789760734.822342:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760734.822447:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512611840 00000100:00100000:1.0:1789760734.822456:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512611840 00000100:00100000:1.0:1789760734.822513:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 714296706805 time 714331930367 state 0x0 ref 1 00000100:00100000:1.0:1789760734.822527:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 714332190783 time 714331930367 state 0x1 ref 1 00000100:00100000:1.0:1789760734.822592:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 714332190783 time 714386891106 state 0x0 ref 1 00000100:00100000:1.0:1789760734.822604:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2878000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512611840:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760734.822654:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24051712, mtime: 0x6aad94de, atime: 0x0, ctime: 0x6aad94de, blocks: 0xb807 00010000:00010000:1.0:1789760734.822663:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2e1ba lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.822679:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=714s v=(0 0) 00010000:00010000:1.0:1789760734.822683:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760734.822689:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2e1ba lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24051712->24055807] (req 24051712->24055807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.822738:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2e1ba lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24051712->24055807] (req 24051712->24055807) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760734.888399:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973191 is committed 00002000:00010000:1.0:1789760734.888463:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24051712 -> 24055808 00010000:00010000:1.0:1789760734.888478:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2e1ba lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24051712->24055807] (req 24051712->24055807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.888491:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2e1ba lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24051712->24055807] (req 24051712->24055807) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.888500:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2e1ba lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24051712->24055807] (req 24051712->24055807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.888507:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760734.888512:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2e1ba lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24051712->24055807] (req 24051712->24055807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.888521:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2e1ba lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24051712->24055807] (req 24051712->24055807) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.888539:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2e1ba lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24051712->24055807] (req 24051712->24055807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.888546:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2e1ba lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24051712->24055807] (req 24051712->24055807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760734.888636:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2878000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512611840:12345-192.168.203.56@tcp:4:dd.0 Request processed in 66029us (66299us total) trans 4294973191 rc 0/0 00000100:00100000:1.0:1789760734.888646:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5902 00000100:00100000:1.0:1789760734.888656:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 714332190783 time 714386891106 state 0x2 ref 0 00000100:00100000:1.0:1789760734.899076:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760734.899148:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512611968 00000100:00100000:3.0:1789760734.899158:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512611968 00000100:00100000:3.0:1789760734.899209:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 714332190783 time 714386891106 state 0x0 ref 1 00000100:00100000:3.0:1789760734.899225:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 714387151522 time 714386891106 state 0x1 ref 1 00000100:00100000:3.0:1789760734.899254:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 714387151522 time 714463553888 state 0x0 ref 1 00000100:00100000:3.0:1789760734.899265:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4132d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512611968:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760734.899309:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24055808, mtime: 0x6aad94de, atime: 0x0, ctime: 0x6aad94de, blocks: 0xb807 00010000:00010000:3.0:1789760734.899317:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2e1c1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.899332:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=714s v=(0 0) 00010000:00010000:3.0:1789760734.899337:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760734.899342:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2e1c1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24055808->24059903] (req 24055808->24059903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.899362:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2e1c1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24055808->24059903] (req 24055808->24059903) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760734.956341:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973192 is committed 00002000:00010000:1.0:1789760734.959792:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24055808 -> 24059904 00010000:00010000:1.0:1789760734.959812:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2e1c1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24055808->24059903] (req 24055808->24059903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.959830:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2e1c1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24055808->24059903] (req 24055808->24059903) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.959848:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2e1c1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24055808->24059903] (req 24055808->24059903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.959855:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760734.959860:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2e1c1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24055808->24059903] (req 24055808->24059903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.959876:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2e1c1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24055808->24059903] (req 24055808->24059903) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.960148:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2e1c1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24055808->24059903] (req 24055808->24059903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.960158:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2e1c1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24055808->24059903] (req 24055808->24059903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760734.960262:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4132d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512611968:12345-192.168.203.56@tcp:4:dd.0 Request processed in 60989us (61186us total) trans 4294973192 rc 0/0 00000100:00100000:1.0:1789760734.960273:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5903 00000100:00100000:1.0:1789760734.960283:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 714387151522 time 714463553888 state 0x2 ref 0 00000100:00100000:3.0:1789760734.963951:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760734.964109:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512612096 00000100:00100000:0.0:1789760734.964119:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512612096 00000100:00100000:0.0:1789760734.964172:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 714387151522 time 714463553888 state 0x0 ref 1 00000100:00100000:0.0:1789760734.964188:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 714463814304 time 714463553888 state 0x1 ref 1 00000100:00100000:0.0:1789760734.964263:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 714463814304 time 714528562158 state 0x0 ref 1 00000100:00100000:0.0:1789760734.964277:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac531dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512612096:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760734.964327:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24059904, mtime: 0x6aad94de, atime: 0x0, ctime: 0x6aad94de, blocks: 0xb807 00010000:00010000:0.0:1789760734.964336:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2e1c8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760734.964356:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=714s v=(0 0) 00010000:00010000:0.0:1789760734.964361:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760734.964368:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2e1c8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24059904->24063999] (req 24059904->24063999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760734.964381:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2e1c8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24059904->24063999] (req 24059904->24063999) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760735.017513:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24059904 -> 24064000 00010000:00010000:0.0:1789760735.017533:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2e1c8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24059904->24063999] (req 24059904->24063999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.017547:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2e1c8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24059904->24063999] (req 24059904->24063999) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.017556:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2e1c8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24059904->24063999] (req 24059904->24063999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.017563:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760735.017568:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2e1c8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24059904->24063999] (req 24059904->24063999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.017577:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2e1c8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24059904->24063999] (req 24059904->24063999) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.017596:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2e1c8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24059904->24063999] (req 24059904->24063999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.017604:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2e1c8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24059904->24063999] (req 24059904->24063999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760735.017724:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac531dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2359:x1876699512612096:12345-192.168.203.56@tcp:4:dd.0 Request processed in 53419us (53752us total) trans 4294973193 rc 0/0 00000100:00100000:0.0:1789760735.017735:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5904 00000100:00100000:0.0:1789760735.017753:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 714463814304 time 714528562158 state 0x2 ref 0 00000001:00080000:3.0:1789760735.018689:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973193 is committed 00000100:00100000:0.0:1789760735.020779:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760735.020975:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512612224 00000100:00100000:3.0:1789760735.020985:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512612224 00000100:00100000:3.0:1789760735.021047:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 714463814304 time 714528562158 state 0x0 ref 1 00000100:00100000:3.0:1789760735.021076:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 714528822574 time 714528562158 state 0x1 ref 1 00000100:00100000:3.0:1789760735.021107:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 714528822574 time 714585407412 state 0x0 ref 1 00000100:00100000:3.0:1789760735.021122:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac287b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512612224:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760735.021168:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24064000, mtime: 0x6aad94de, atime: 0x0, ctime: 0x6aad94de, blocks: 0xb807 00010000:00010000:3.0:1789760735.021178:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2e1cf lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.021194:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=714s v=(0 0) 00010000:00010000:3.0:1789760735.021199:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760735.021205:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2e1cf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24064000->24068095] (req 24064000->24068095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.021218:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2e1cf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24064000->24068095] (req 24064000->24068095) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1789760735.021818:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00010000:1.0:1789760735.054884:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24064000 -> 24068096 00010000:00010000:1.0:1789760735.054907:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2e1cf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24064000->24068095] (req 24064000->24068095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.054920:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2e1cf lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24064000->24068095] (req 24064000->24068095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.054930:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2e1cf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24064000->24068095] (req 24064000->24068095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.054937:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760735.054943:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2e1cf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24064000->24068095] (req 24064000->24068095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.054951:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2e1cf lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24064000->24068095] (req 24064000->24068095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.054979:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2e1cf lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24064000->24068095] (req 24064000->24068095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.054988:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2e1cf lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24064000->24068095] (req 24064000->24068095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760735.055244:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac287b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2358:x1876699512612224:12345-192.168.203.56@tcp:4:dd.0 Request processed in 34120us (34475us total) trans 4294973194 rc 0/0 00000100:00100000:1.0:1789760735.055255:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5905 00000100:00100000:1.0:1789760735.055264:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 714528822574 time 714585407412 state 0x2 ref 0 00000001:00080000:0.0:1789760735.055424:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973194 is committed 00000100:00100000:2.0:1789760735.058607:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760735.058691:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512612352 00000100:00100000:1.0:1789760735.058732:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512612352 00000100:00100000:1.0:1789760735.058785:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 714528822574 time 714585407412 state 0x0 ref 1 00000100:00100000:1.0:1789760735.058799:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 714585667828 time 714585407412 state 0x1 ref 1 00000100:00100000:1.0:1789760735.058832:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 714585667828 time 714623130445 state 0x0 ref 1 00000100:00100000:1.0:1789760735.058844:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febffb9ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512612352:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760735.058973:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24068096, mtime: 0x6aad94de, atime: 0x0, ctime: 0x6aad94de, blocks: 0xb807 00010000:00010000:1.0:1789760735.058982:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2e1d6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.058998:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=714s v=(0 0) 00010000:00010000:1.0:1789760735.059003:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760735.059009:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2e1d6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24068096->24072191] (req 24068096->24072191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.059023:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2e1d6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24068096->24072191] (req 24068096->24072191) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760735.117752:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973195 is committed 00002000:00010000:2.0:1789760735.117856:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24068096 -> 24072192 00010000:00010000:2.0:1789760735.117872:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2e1d6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24068096->24072191] (req 24068096->24072191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.117886:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2e1d6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24068096->24072191] (req 24068096->24072191) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.117896:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2e1d6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24068096->24072191] (req 24068096->24072191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.117904:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760735.117909:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2e1d6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24068096->24072191] (req 24068096->24072191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.117918:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2e1d6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24068096->24072191] (req 24068096->24072191) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.117944:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2e1d6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24068096->24072191] (req 24068096->24072191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.117953:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2e1d6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24068096->24072191] (req 24068096->24072191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760735.118686:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febffb9ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512612352:12345-192.168.203.56@tcp:4:dd.0 Request processed in 59194us (59439us total) trans 4294973195 rc 0/0 00000100:00100000:2.0:1789760735.118743:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5906 00000100:00100000:2.0:1789760735.118754:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 714585667828 time 714623130445 state 0x2 ref 0 00000100:00100000:3.0:1789760735.124621:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760735.124734:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512612480 00000100:00100000:0.0:1789760735.124744:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512612480 00000100:00100000:0.0:1789760735.124809:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 714585667828 time 714623130445 state 0x0 ref 1 00000100:00100000:0.0:1789760735.124824:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 714623390861 time 714623130445 state 0x1 ref 1 00000100:00100000:0.0:1789760735.124857:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 714623390861 time 714689156258 state 0x0 ref 1 00000100:00100000:0.0:1789760735.124869:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac531ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512612480:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760735.124916:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24072192, mtime: 0x6aad94de, atime: 0x0, ctime: 0x6aad94de, blocks: 0xb807 00010000:00010000:0.0:1789760735.124926:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2e1dd lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.124943:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=714s v=(0 0) 00010000:00010000:0.0:1789760735.124947:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760735.124953:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2e1dd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24072192->24076287] (req 24072192->24076287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.124966:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2e1dd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24072192->24076287] (req 24072192->24076287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760735.169853:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24072192 -> 24076288 00010000:00010000:0.0:1789760735.169871:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2e1dd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24072192->24076287] (req 24072192->24076287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.169884:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2e1dd lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24072192->24076287] (req 24072192->24076287) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.169893:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2e1dd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24072192->24076287] (req 24072192->24076287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.169899:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760735.169904:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2e1dd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24072192->24076287] (req 24072192->24076287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.169913:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2e1dd lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24072192->24076287] (req 24072192->24076287) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.169930:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2e1dd lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24072192->24076287] (req 24072192->24076287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.169942:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2e1dd lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24072192->24076287] (req 24072192->24076287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760735.170030:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac531ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2359:x1876699512612480:12345-192.168.203.56@tcp:4:dd.0 Request processed in 45158us (45413us total) trans 4294973196 rc 0/0 00000100:00100000:0.0:1789760735.170040:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5907 00000100:00100000:0.0:1789760735.170049:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 714623390861 time 714689156258 state 0x2 ref 0 00000001:00080000:0.0:1789760735.170095:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973196 is committed 00000100:00100000:1.0:1789760735.175219:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760735.176164:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512612608 00000100:00100000:2.0:1789760735.176176:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512612608 00000100:00100000:2.0:1789760735.176236:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 714623390861 time 714689156258 state 0x0 ref 1 00000100:00100000:2.0:1789760735.176252:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 714689416674 time 714689156258 state 0x1 ref 1 00000100:00100000:2.0:1789760735.176282:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 714689416674 time 714740581297 state 0x0 ref 1 00000100:00100000:2.0:1789760735.176294:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4132300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512612608:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760735.176345:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24076288, mtime: 0x6aad94de, atime: 0x0, ctime: 0x6aad94de, blocks: 0xb807 00010000:00010000:2.0:1789760735.176354:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2e1e4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.176370:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=714s v=(0 0) 00010000:00010000:2.0:1789760735.176375:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760735.176391:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2e1e4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24076288->24080383] (req 24076288->24080383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.176407:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2e1e4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24076288->24080383] (req 24076288->24080383) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760735.217158:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973197 is committed 00002000:00010000:2.0:1789760735.217240:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24076288 -> 24080384 00010000:00010000:2.0:1789760735.217256:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2e1e4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24076288->24080383] (req 24076288->24080383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.217270:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2e1e4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24076288->24080383] (req 24076288->24080383) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.217279:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2e1e4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24076288->24080383] (req 24076288->24080383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.217286:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760735.217291:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2e1e4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24076288->24080383] (req 24076288->24080383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.217300:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2e1e4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24076288->24080383] (req 24076288->24080383) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.217317:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2e1e4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24076288->24080383] (req 24076288->24080383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.217324:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2e1e4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24076288->24080383] (req 24076288->24080383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760735.217519:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4132300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512612608:12345-192.168.203.56@tcp:4:dd.0 Request processed in 41222us (42307us total) trans 4294973197 rc 0/0 00000100:00100000:2.0:1789760735.217528:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5908 00000100:00100000:2.0:1789760735.217538:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 714689416674 time 714740581297 state 0x2 ref 0 00000100:00100000:1.0:1789760735.224953:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760735.225041:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512612736 00000100:00100000:2.0:1789760735.225050:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512612736 00000100:00100000:2.0:1789760735.225112:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 714689416674 time 714740581297 state 0x0 ref 1 00000100:00100000:2.0:1789760735.225128:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 714740841713 time 714740581297 state 0x1 ref 1 00000100:00100000:2.0:1789760735.225161:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 714740841713 time 714789461812 state 0x0 ref 1 00000100:00100000:2.0:1789760735.225173:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4131f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512612736:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760735.225232:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24080384, mtime: 0x6aad94de, atime: 0x0, ctime: 0x6aad94de, blocks: 0xb807 00010000:00010000:2.0:1789760735.225241:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2e1eb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.225256:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=714s v=(0 0) 00010000:00010000:2.0:1789760735.225260:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760735.225267:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2e1eb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24080384->24084479] (req 24080384->24084479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.225280:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2e1eb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24080384->24084479] (req 24080384->24084479) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760735.257618:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973198 is committed 00002000:00010000:2.0:1789760735.257721:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24080384 -> 24084480 00010000:00010000:2.0:1789760735.257739:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2e1eb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24080384->24084479] (req 24080384->24084479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.257751:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2e1eb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24080384->24084479] (req 24080384->24084479) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.257760:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2e1eb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24080384->24084479] (req 24080384->24084479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.257767:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760735.257772:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2e1eb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24080384->24084479] (req 24080384->24084479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.257780:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2e1eb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24080384->24084479] (req 24080384->24084479) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.257800:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2e1eb lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24080384->24084479] (req 24080384->24084479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.257807:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2e1eb lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24080384->24084479] (req 24080384->24084479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760735.257930:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4131f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512612736:12345-192.168.203.56@tcp:4:dd.0 Request processed in 32753us (32981us total) trans 4294973198 rc 0/0 00000100:00100000:2.0:1789760735.257944:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5909 00000100:00100000:2.0:1789760735.257953:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 714740841713 time 714789461812 state 0x2 ref 0 00000100:00100000:3.0:1789760735.264062:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760735.267540:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512612864 00000100:00100000:0.0:1789760735.267557:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512612864 00000100:00100000:1.0:1789760735.267720:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 714740841713 time 714789461812 state 0x0 ref 1 00000100:00100000:1.0:1789760735.267736:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 714789722228 time 714789461812 state 0x1 ref 1 00000100:00100000:1.0:1789760735.267838:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 714789722228 time 714832136892 state 0x0 ref 1 00000100:00100000:1.0:1789760735.267854:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac531f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512612864:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760735.267902:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24084480, mtime: 0x6aad94de, atime: 0x0, ctime: 0x6aad94de, blocks: 0xb807 00010000:00010000:1.0:1789760735.267910:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2e1f2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.267938:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=714s v=(0 0) 00010000:00010000:1.0:1789760735.267943:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760735.267949:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2e1f2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24084480->24088575] (req 24084480->24088575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.267963:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2e1f2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24084480->24088575] (req 24084480->24088575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760735.304765:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24084480 -> 24088576 00010000:00010000:1.0:1789760735.304785:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2e1f2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24084480->24088575] (req 24084480->24088575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.304797:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2e1f2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24084480->24088575] (req 24084480->24088575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.304807:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2e1f2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24084480->24088575] (req 24084480->24088575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.304814:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760735.304819:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2e1f2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24084480->24088575] (req 24084480->24088575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.304828:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2e1f2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24084480->24088575] (req 24084480->24088575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.304846:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2e1f2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24084480->24088575] (req 24084480->24088575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.304854:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2e1f2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24084480->24088575] (req 24084480->24088575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760735.304932:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac531f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2358:x1876699512612864:12345-192.168.203.56@tcp:4:dd.0 Request processed in 37079us (40877us total) trans 4294973199 rc 0/0 00000100:00100000:1.0:1789760735.304943:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5910 00000100:00100000:1.0:1789760735.304953:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 714789722228 time 714832136892 state 0x2 ref 0 00000001:00080000:2.0:1789760735.305454:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973199 is committed 00000100:00100000:1.0:1789760735.309330:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760735.309547:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512612992 00000100:00100000:2.0:1789760735.309559:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512612992 00000100:00100000:2.0:1789760735.309618:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 714789722228 time 714832136892 state 0x0 ref 1 00000100:00100000:2.0:1789760735.309634:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 714832397308 time 714832136892 state 0x1 ref 1 00000100:00100000:2.0:1789760735.309670:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 714832397308 time 714873969192 state 0x0 ref 1 00000100:00100000:2.0:1789760735.309682:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4130380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512612992:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760735.309897:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24088576, mtime: 0x6aad94de, atime: 0x0, ctime: 0x6aad94de, blocks: 0xb807 00010000:00010000:2.0:1789760735.309907:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2e1f9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.309922:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=714s v=(0 0) 00010000:00010000:2.0:1789760735.309926:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760735.309932:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2e1f9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24088576->24092671] (req 24088576->24092671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.309945:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2e1f9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24088576->24092671] (req 24088576->24092671) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760735.356399:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24088576 -> 24092672 00010000:00010000:2.0:1789760735.356420:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2e1f9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24088576->24092671] (req 24088576->24092671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.356440:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2e1f9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24088576->24092671] (req 24088576->24092671) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.356449:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2e1f9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24088576->24092671] (req 24088576->24092671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.356455:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760735.356459:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2e1f9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24088576->24092671] (req 24088576->24092671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.356474:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2e1f9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24088576->24092671] (req 24088576->24092671) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.356492:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2e1f9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24088576->24092671] (req 24088576->24092671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.356499:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2e1f9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24088576->24092671] (req 24088576->24092671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760735.356588:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4130380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2358:x1876699512612992:12345-192.168.203.56@tcp:4:dd.0 Request processed in 46903us (47263us total) trans 4294973200 rc 0/0 00000100:00100000:2.0:1789760735.356597:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5911 00000100:00100000:2.0:1789760735.356607:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 714832397308 time 714873969192 state 0x2 ref 0 00000001:00080000:2.0:1789760735.356645:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973200 is committed 00000100:00100000:3.0:1789760735.361046:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760735.361197:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512613120 00000100:00100000:2.0:1789760735.361207:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512613120 00000100:00100000:2.0:1789760735.361262:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 714832397308 time 714873969192 state 0x0 ref 1 00000100:00100000:2.0:1789760735.361278:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 714874229608 time 714873969192 state 0x1 ref 1 00000100:00100000:2.0:1789760735.361304:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 714874229608 time 714925604070 state 0x0 ref 1 00000100:00100000:2.0:1789760735.361316:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac531df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512613120:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760735.361363:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24092672, mtime: 0x6aad94de, atime: 0x0, ctime: 0x6aad94de, blocks: 0xb807 00010000:00010000:2.0:1789760735.361390:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2e200 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.361406:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=714s v=(0 0) 00010000:00010000:2.0:1789760735.361410:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760735.361417:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2e200 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24092672->24096767] (req 24092672->24096767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.361443:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2e200 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24092672->24096767] (req 24092672->24096767) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760735.405037:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24092672 -> 24096768 00010000:00010000:3.0:1789760735.405067:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2e200 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24092672->24096767] (req 24092672->24096767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.405080:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2e200 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24092672->24096767] (req 24092672->24096767) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.405089:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2e200 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24092672->24096767] (req 24092672->24096767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.405096:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760735.405100:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2e200 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24092672->24096767] (req 24092672->24096767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.405109:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2e200 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24092672->24096767] (req 24092672->24096767) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.405130:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2e200 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24092672->24096767] (req 24092672->24096767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.405137:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2e200 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24092672->24096767] (req 24092672->24096767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760735.405235:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac531df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2358:x1876699512613120:12345-192.168.203.56@tcp:4:dd.0 Request processed in 43916us (44193us total) trans 4294973201 rc 0/0 00000100:00100000:3.0:1789760735.405248:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5912 00000100:00100000:3.0:1789760735.405258:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 714874229608 time 714925604070 state 0x2 ref 0 00000001:00080000:0.0:1789760735.406918:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973201 is committed 00000100:00100000:0.0:1789760735.408324:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760735.408434:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512613248 00000100:00100000:1.0:1789760735.408443:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512613248 00000100:00100000:1.0:1789760735.408497:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 714874229608 time 714925604070 state 0x0 ref 1 00000100:00100000:1.0:1789760735.408511:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 714925864486 time 714925604070 state 0x1 ref 1 00000100:00100000:1.0:1789760735.408586:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 714925864486 time 714972885108 state 0x0 ref 1 00000100:00100000:1.0:1789760735.408600:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2878700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512613248:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760735.408658:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24096768, mtime: 0x6aad94de, atime: 0x0, ctime: 0x6aad94de, blocks: 0xb807 00010000:00010000:1.0:1789760735.408667:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2e207 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.408683:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=714s v=(0 0) 00010000:00010000:1.0:1789760735.408688:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760735.408738:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2e207 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24096768->24100863] (req 24096768->24100863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.408754:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2e207 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24096768->24100863] (req 24096768->24100863) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760735.460412:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973202 is committed 00002000:00010000:0.0:1789760735.460527:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24096768 -> 24100864 00010000:00010000:0.0:1789760735.460542:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2e207 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24096768->24100863] (req 24096768->24100863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.460554:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2e207 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24096768->24100863] (req 24096768->24100863) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.460563:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2e207 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24096768->24100863] (req 24096768->24100863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.460569:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760735.460573:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2e207 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24096768->24100863] (req 24096768->24100863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.460582:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2e207 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24096768->24100863] (req 24096768->24100863) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.460602:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2e207 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24096768->24100863] (req 24096768->24100863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.460609:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2e207 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24096768->24100863] (req 24096768->24100863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760735.460742:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2878700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512613248:12345-192.168.203.56@tcp:4:dd.0 Request processed in 52140us (52420us total) trans 4294973202 rc 0/0 00000100:00100000:0.0:1789760735.460752:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5913 00000100:00100000:0.0:1789760735.460762:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 714925864486 time 714972885108 state 0x2 ref 0 00000100:00100000:1.0:1789760735.464885:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760735.465068:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512613376 00000100:00100000:2.0:1789760735.465080:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512613376 00000100:00100000:2.0:1789760735.465141:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 714925864486 time 714972885108 state 0x0 ref 1 00000100:00100000:2.0:1789760735.465155:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 714973145524 time 714972885108 state 0x1 ref 1 00000100:00100000:2.0:1789760735.465187:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 714973145524 time 715029487344 state 0x0 ref 1 00000100:00100000:2.0:1789760735.465199:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4133b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512613376:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760735.465251:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24100864, mtime: 0x6aad94de, atime: 0x0, ctime: 0x6aad94de, blocks: 0xb807 00010000:00010000:2.0:1789760735.465259:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2e20e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.465274:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=715s v=(0 0) 00010000:00010000:2.0:1789760735.465278:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760735.465285:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2e20e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24100864->24104959] (req 24100864->24104959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.465298:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2e20e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24100864->24104959] (req 24100864->24104959) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760735.522619:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24100864 -> 24104960 00010000:00010000:3.0:1789760735.522644:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2e20e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24100864->24104959] (req 24100864->24104959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.522660:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2e20e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24100864->24104959] (req 24100864->24104959) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.522671:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2e20e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24100864->24104959] (req 24100864->24104959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.522679:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760735.522684:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2e20e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24100864->24104959] (req 24100864->24104959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.522720:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2e20e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24100864->24104959] (req 24100864->24104959) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.522744:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2e20e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24100864->24104959] (req 24100864->24104959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.522758:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2e20e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24100864->24104959] (req 24100864->24104959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760735.522863:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4133b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2358:x1876699512613376:12345-192.168.203.56@tcp:4:dd.0 Request processed in 57660us (57984us total) trans 4294973203 rc 0/0 00000100:00100000:3.0:1789760735.522874:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5914 00000100:00100000:3.0:1789760735.522883:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 714973145524 time 715029487344 state 0x2 ref 0 00000001:00080000:1.0:1789760735.523943:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973203 is committed 00000100:00100000:1.0:1789760735.528261:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760735.528732:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512613504 00000100:00100000:3.0:1789760735.528743:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512613504 00000100:00100000:3.0:1789760735.528861:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 714973145524 time 715029487344 state 0x0 ref 1 00000100:00100000:3.0:1789760735.528878:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 715029747760 time 715029487344 state 0x1 ref 1 00000100:00100000:3.0:1789760735.528901:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 715029747760 time 715093200969 state 0x0 ref 1 00000100:00100000:3.0:1789760735.528916:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4133800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512613504:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760735.528967:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24104960, mtime: 0x6aad94de, atime: 0x0, ctime: 0x6aad94de, blocks: 0xb807 00010000:00010000:3.0:1789760735.528978:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2e215 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.528995:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=715s v=(0 0) 00010000:00010000:3.0:1789760735.528999:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760735.529006:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2e215 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24104960->24109055] (req 24104960->24109055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.529025:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2e215 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24104960->24109055] (req 24104960->24109055) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760735.573843:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24104960 -> 24109056 00002000:00010000:1.0:1789760735.573854:0:6709:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760734 -> 1789760735 00002000:00010000:1.0:1789760735.573856:0:6709:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760734 -> 1789760735 00010000:00010000:1.0:1789760735.573870:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2e215 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24104960->24109055] (req 24104960->24109055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.573883:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2e215 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24104960->24109055] (req 24104960->24109055) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.573893:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2e215 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24104960->24109055] (req 24104960->24109055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.573907:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760735.573912:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2e215 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24104960->24109055] (req 24104960->24109055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.573921:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2e215 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24104960->24109055] (req 24104960->24109055) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.573940:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2e215 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24104960->24109055] (req 24104960->24109055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.573948:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2e215 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24104960->24109055] (req 24104960->24109055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760735.574045:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4133800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2361:x1876699512613504:12345-192.168.203.56@tcp:4:dd.0 Request processed in 45128us (45791us total) trans 4294973204 rc 0/0 00000100:00100000:1.0:1789760735.574065:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5915 00000100:00100000:1.0:1789760735.574075:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 715029747760 time 715093200969 state 0x2 ref 0 00000001:00080000:1.0:1789760735.574112:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973204 is committed 00000100:00100000:0.0:1789760735.577441:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760735.577545:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512613632 00000100:00100000:1.0:1789760735.577556:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512613632 00000100:00100000:1.0:1789760735.577620:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 715029747760 time 715093200969 state 0x0 ref 1 00000100:00100000:1.0:1789760735.577638:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 715093461385 time 715093200969 state 0x1 ref 1 00000100:00100000:1.0:1789760735.577670:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 715093461385 time 715141969309 state 0x0 ref 1 00000100:00100000:1.0:1789760735.577684:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac287aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512613632:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760735.577925:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24109056, mtime: 0x6aad94df, atime: 0x0, ctime: 0x6aad94df, blocks: 0xb807 00010000:00010000:0.0:1789760735.577935:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2e21c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.577951:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=715s v=(0 0) 00010000:00010000:0.0:1789760735.577955:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760735.577962:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2e21c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24109056->24113151] (req 24109056->24113151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.577975:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2e21c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24109056->24113151] (req 24109056->24113151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.606393:0:5836:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:0.0:1789760735.614679:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973205 is committed 00002000:00010000:0.0:1789760735.614958:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24109056 -> 24113152 00010000:00010000:0.0:1789760735.615030:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2e21c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24109056->24113151] (req 24109056->24113151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.615046:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2e21c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24109056->24113151] (req 24109056->24113151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.615056:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2e21c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24109056->24113151] (req 24109056->24113151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.615063:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760735.615068:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2e21c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24109056->24113151] (req 24109056->24113151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.615078:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2e21c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24109056->24113151] (req 24109056->24113151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.615102:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2e21c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24109056->24113151] (req 24109056->24113151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.615110:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2e21c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24109056->24113151] (req 24109056->24113151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760735.615220:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac287aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512613632:12345-192.168.203.56@tcp:4:dd.0 Request processed in 37532us (37782us total) trans 4294973205 rc 0/0 00000100:00100000:0.0:1789760735.615230:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5916 00000100:00100000:0.0:1789760735.615240:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 715093461385 time 715141969309 state 0x2 ref 0 00000100:00100000:0.0:1789760735.618724:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760735.618785:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512613760 00000100:00100000:3.0:1789760735.618795:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512613760 00000100:00100000:3.0:1789760735.618852:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 715093461385 time 715141969309 state 0x0 ref 1 00000100:00100000:3.0:1789760735.618868:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 715142229725 time 715141969309 state 0x1 ref 1 00000100:00100000:3.0:1789760735.618895:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 715142229725 time 715183194531 state 0x0 ref 1 00000100:00100000:3.0:1789760735.618908:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2879880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512613760:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760735.618954:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24113152, mtime: 0x6aad94df, atime: 0x0, ctime: 0x6aad94df, blocks: 0xb807 00010000:00010000:3.0:1789760735.618963:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2e223 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.618977:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=715s v=(0 0) 00010000:00010000:3.0:1789760735.618982:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760735.618988:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2e223 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24113152->24117247] (req 24113152->24117247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.619002:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2e223 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24113152->24117247] (req 24113152->24117247) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760735.668415:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973206 is committed 00002000:00010000:1.0:1789760735.668718:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24113152 -> 24117248 00010000:00010000:1.0:1789760735.668736:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2e223 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24113152->24117247] (req 24113152->24117247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.668749:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2e223 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24113152->24117247] (req 24113152->24117247) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.668759:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2e223 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24113152->24117247] (req 24113152->24117247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.668768:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760735.668782:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2e223 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24113152->24117247] (req 24113152->24117247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.668791:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2e223 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24113152->24117247] (req 24113152->24117247) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.668809:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2e223 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24113152->24117247] (req 24113152->24117247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.668817:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2e223 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24113152->24117247] (req 24113152->24117247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760735.668899:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2879880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512613760:12345-192.168.203.56@tcp:4:dd.0 Request processed in 49989us (50204us total) trans 4294973206 rc 0/0 00000100:00100000:1.0:1789760735.668909:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5917 00000100:00100000:1.0:1789760735.668919:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 715142229725 time 715183194531 state 0x2 ref 0 00000100:00100000:1.0:1789760735.673684:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760735.674182:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512613888 00000100:00100000:3.0:1789760735.674192:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512613888 00000100:00100000:3.0:1789760735.674242:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 715142229725 time 715183194531 state 0x0 ref 1 00000100:00100000:3.0:1789760735.674254:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 715183454947 time 715183194531 state 0x1 ref 1 00000100:00100000:3.0:1789760735.674433:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 715183454947 time 715238730743 state 0x0 ref 1 00000100:00100000:3.0:1789760735.674525:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4130000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512613888:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760735.674576:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24117248, mtime: 0x6aad94df, atime: 0x0, ctime: 0x6aad94df, blocks: 0xb807 00010000:00010000:3.0:1789760735.674585:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2e22a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.674601:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=715s v=(0 0) 00010000:00010000:3.0:1789760735.674606:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760735.674612:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2e22a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24117248->24121343] (req 24117248->24121343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.674626:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2e22a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24117248->24121343] (req 24117248->24121343) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760735.713318:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973207 is committed 00002000:00010000:3.0:1789760735.713593:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24117248 -> 24121344 00002000:00010000:3.0:1789760735.713599:0:6709:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb blocks from disk: 47111 -> 49159 00010000:00010000:3.0:1789760735.713612:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2e22a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24117248->24121343] (req 24117248->24121343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.713624:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2e22a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24117248->24121343] (req 24117248->24121343) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.713639:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2e22a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24117248->24121343] (req 24117248->24121343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.713645:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760735.713651:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2e22a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24117248->24121343] (req 24117248->24121343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.713661:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2e22a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24117248->24121343] (req 24117248->24121343) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.713681:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2e22a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24117248->24121343] (req 24117248->24121343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.713690:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2e22a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24117248->24121343] (req 24117248->24121343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760735.714030:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4130000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512613888:12345-192.168.203.56@tcp:4:dd.0 Request processed in 39583us (40353us total) trans 4294973207 rc 0/0 00000100:00100000:3.0:1789760735.714037:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5918 00000100:00100000:3.0:1789760735.714044:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 715183454947 time 715238730743 state 0x2 ref 0 00000100:00100000:0.0:1789760735.717503:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760735.717654:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512614016 00000100:00100000:2.0:1789760735.717664:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512614016 00000100:00100000:2.0:1789760735.717752:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 715183454947 time 715238730743 state 0x0 ref 1 00000100:00100000:2.0:1789760735.717768:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 715238991159 time 715238730743 state 0x1 ref 1 00000100:00100000:2.0:1789760735.717788:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 715238991159 time 715282088282 state 0x0 ref 1 00000100:00100000:2.0:1789760735.717800:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2879180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512614016:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760735.717846:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24121344, mtime: 0x6aad94df, atime: 0x0, ctime: 0x6aad94df, blocks: 0xc007 00010000:00010000:2.0:1789760735.717855:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2e231 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.717870:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=715s v=(0 0) 00010000:00010000:2.0:1789760735.717875:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760735.717882:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2e231 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24121344->24125439] (req 24121344->24125439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.717895:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2e231 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24121344->24125439] (req 24121344->24125439) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760735.756644:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973208 is committed 00002000:00010000:0.0:1789760735.756770:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24121344 -> 24125440 00010000:00010000:0.0:1789760735.756788:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2e231 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24121344->24125439] (req 24121344->24125439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.756803:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2e231 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24121344->24125439] (req 24121344->24125439) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.756813:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2e231 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24121344->24125439] (req 24121344->24125439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.756825:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760735.756830:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2e231 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24121344->24125439] (req 24121344->24125439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.756840:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2e231 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24121344->24125439] (req 24121344->24125439) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.756863:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2e231 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24121344->24125439] (req 24121344->24125439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.756878:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2e231 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24121344->24125439] (req 24121344->24125439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760735.756978:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2879180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512614016:12345-192.168.203.56@tcp:4:dd.0 Request processed in 39175us (39478us total) trans 4294973208 rc 0/0 00000100:00100000:0.0:1789760735.756988:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5919 00000100:00100000:0.0:1789760735.756998:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 715238991159 time 715282088282 state 0x2 ref 0 00000100:00100000:0.0:1789760735.760017:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760735.760084:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512614144 00000100:00100000:1.0:1789760735.760095:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512614144 00000100:00100000:1.0:1789760735.760151:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 715238991159 time 715282088282 state 0x0 ref 1 00000100:00100000:1.0:1789760735.760168:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 715282348698 time 715282088282 state 0x1 ref 1 00000100:00100000:1.0:1789760735.760200:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 715282348698 time 715324499459 state 0x0 ref 1 00000100:00100000:1.0:1789760735.760214:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2879500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512614144:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760735.760267:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24125440, mtime: 0x6aad94df, atime: 0x0, ctime: 0x6aad94df, blocks: 0xc007 00010000:00010000:1.0:1789760735.760277:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2e238 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.760294:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=715s v=(0 0) 00010000:00010000:1.0:1789760735.760299:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760735.760305:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2e238 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24125440->24129535] (req 24125440->24129535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.760320:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2e238 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24125440->24129535] (req 24125440->24129535) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760735.803819:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973209 is committed 00002000:00010000:2.0:1789760735.803919:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24125440 -> 24129536 00010000:00010000:2.0:1789760735.803933:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2e238 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24125440->24129535] (req 24125440->24129535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.803945:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2e238 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24125440->24129535] (req 24125440->24129535) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.803954:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2e238 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24125440->24129535] (req 24125440->24129535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.803961:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760735.803966:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2e238 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24125440->24129535] (req 24125440->24129535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.803976:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2e238 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24125440->24129535] (req 24125440->24129535) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.803999:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2e238 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24125440->24129535] (req 24125440->24129535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.804006:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2e238 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24125440->24129535] (req 24125440->24129535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760735.804138:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2879500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512614144:12345-192.168.203.56@tcp:4:dd.0 Request processed in 43923us (44127us total) trans 4294973209 rc 0/0 00000100:00100000:2.0:1789760735.804148:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5920 00000100:00100000:2.0:1789760735.804158:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 715282348698 time 715324499459 state 0x2 ref 0 00000100:00100000:1.0:1789760735.807216:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760735.807727:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512614272 00000100:00100000:3.0:1789760735.807748:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512614272 00000100:00100000:3.0:1789760735.807805:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 715282348698 time 715324499459 state 0x0 ref 1 00000100:00100000:3.0:1789760735.807821:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 715324759875 time 715324499459 state 0x1 ref 1 00000100:00100000:3.0:1789760735.807853:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 715324759875 time 715372152429 state 0x0 ref 1 00000100:00100000:3.0:1789760735.807865:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4133480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512614272:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760735.807911:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24129536, mtime: 0x6aad94df, atime: 0x0, ctime: 0x6aad94df, blocks: 0xc007 00010000:00010000:3.0:1789760735.807921:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2e23f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.807937:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=715s v=(0 0) 00010000:00010000:3.0:1789760735.807942:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760735.807948:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2e23f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24129536->24133631] (req 24129536->24133631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.807962:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2e23f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24129536->24133631] (req 24129536->24133631) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760735.835269:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24129536 -> 24133632 00010000:00010000:3.0:1789760735.835289:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2e23f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24129536->24133631] (req 24129536->24133631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.835314:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2e23f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24129536->24133631] (req 24129536->24133631) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.835324:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2e23f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24129536->24133631] (req 24129536->24133631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760735.835328:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973210 is committed 00010000:00010000:3.0:1789760735.835336:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760735.835341:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2e23f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24129536->24133631] (req 24129536->24133631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.835350:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2e23f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24129536->24133631] (req 24129536->24133631) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.835368:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2e23f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24129536->24133631] (req 24129536->24133631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.835374:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2e23f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24129536->24133631] (req 24129536->24133631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760735.835509:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4133480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512614272:12345-192.168.203.56@tcp:4:dd.0 Request processed in 27641us (28297us total) trans 4294973210 rc 0/0 00000100:00100000:3.0:1789760735.835520:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5921 00000100:00100000:3.0:1789760735.835530:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 715324759875 time 715372152429 state 0x2 ref 0 00000100:00100000:0.0:1789760735.841001:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760735.841152:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512614400 00000100:00100000:2.0:1789760735.841162:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512614400 00000100:00100000:2.0:1789760735.841218:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 715324759875 time 715372152429 state 0x0 ref 1 00000100:00100000:2.0:1789760735.841233:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 715372412845 time 715372152429 state 0x1 ref 1 00000100:00100000:2.0:1789760735.841261:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 715372412845 time 715405560436 state 0x0 ref 1 00000100:00100000:2.0:1789760735.841276:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2878a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512614400:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760735.841411:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24133632, mtime: 0x6aad94df, atime: 0x0, ctime: 0x6aad94df, blocks: 0xc007 00010000:00010000:2.0:1789760735.841420:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2e246 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.841441:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=715s v=(0 0) 00010000:00010000:2.0:1789760735.841446:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760735.841451:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2e246 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24133632->24137727] (req 24133632->24137727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.841466:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2e246 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24133632->24137727] (req 24133632->24137727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760735.881897:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973211 is committed 00002000:00010000:3.0:1789760735.882008:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24133632 -> 24137728 00010000:00010000:3.0:1789760735.882025:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2e246 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24133632->24137727] (req 24133632->24137727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.882037:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2e246 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24133632->24137727] (req 24133632->24137727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.882050:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2e246 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24133632->24137727] (req 24133632->24137727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.882058:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760735.882062:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2e246 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24133632->24137727] (req 24133632->24137727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.882071:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2e246 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24133632->24137727] (req 24133632->24137727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.882231:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2e246 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24133632->24137727] (req 24133632->24137727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.882242:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2e246 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24133632->24137727] (req 24133632->24137727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760735.882337:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2878a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512614400:12345-192.168.203.56@tcp:4:dd.0 Request processed in 41060us (41339us total) trans 4294973211 rc 0/0 00000100:00100000:3.0:1789760735.882348:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5922 00000100:00100000:3.0:1789760735.882368:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 715372412845 time 715405560436 state 0x2 ref 0 00000100:00100000:0.0:1789760735.886411:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760735.886500:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512614528 00000100:00100000:2.0:1789760735.886510:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512614528 00000100:00100000:2.0:1789760735.886558:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 715372412845 time 715405560436 state 0x0 ref 1 00000100:00100000:2.0:1789760735.886572:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 715405820852 time 715405560436 state 0x1 ref 1 00000100:00100000:2.0:1789760735.886604:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 715405820852 time 715450904143 state 0x0 ref 1 00000100:00100000:2.0:1789760735.886616:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac287ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512614528:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760735.886671:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24137728, mtime: 0x6aad94df, atime: 0x0, ctime: 0x6aad94df, blocks: 0xc007 00010000:00010000:2.0:1789760735.886680:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2e24d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.886730:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=715s v=(0 0) 00010000:00010000:2.0:1789760735.886735:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760735.886742:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2e24d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24137728->24141823] (req 24137728->24141823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.886770:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2e24d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24137728->24141823] (req 24137728->24141823) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760735.938480:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973212 is committed 00002000:00010000:3.0:1789760735.938683:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24137728 -> 24141824 00010000:00010000:3.0:1789760735.938734:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2e24d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24137728->24141823] (req 24137728->24141823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.938753:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2e24d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24137728->24141823] (req 24137728->24141823) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.938763:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2e24d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24137728->24141823] (req 24137728->24141823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.938770:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760735.938775:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2e24d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24137728->24141823] (req 24137728->24141823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.938784:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2e24d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24137728->24141823] (req 24137728->24141823) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.938805:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2e24d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24137728->24141823] (req 24137728->24141823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.938938:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2e24d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24137728->24141823] (req 24137728->24141823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760735.939036:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac287ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512614528:12345-192.168.203.56@tcp:4:dd.0 Request processed in 52417us (52630us total) trans 4294973212 rc 0/0 00000100:00100000:3.0:1789760735.939203:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5923 00000100:00100000:3.0:1789760735.939214:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 715405820852 time 715450904143 state 0x2 ref 0 00000100:00100000:1.0:1789760735.942076:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760735.942221:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512614656 00000100:00100000:3.0:1789760735.942231:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512614656 00000100:00100000:3.0:1789760735.942282:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 715405820852 time 715450904143 state 0x0 ref 1 00000100:00100000:3.0:1789760735.942297:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 715451164559 time 715450904143 state 0x1 ref 1 00000100:00100000:3.0:1789760735.942328:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 715451164559 time 715506628082 state 0x0 ref 1 00000100:00100000:3.0:1789760735.942340:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4130700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512614656:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760735.942397:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24141824, mtime: 0x6aad94df, atime: 0x0, ctime: 0x6aad94df, blocks: 0xc007 00010000:00010000:3.0:1789760735.942406:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2e254 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.942423:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=715s v=(0 0) 00010000:00010000:3.0:1789760735.942427:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760735.942433:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2e254 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24141824->24145919] (req 24141824->24145919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.942446:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2e254 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24141824->24145919] (req 24141824->24145919) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760735.982753:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24141824 -> 24145920 00010000:00010000:3.0:1789760735.982772:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2e254 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24141824->24145919] (req 24141824->24145919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.982785:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2e254 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24141824->24145919] (req 24141824->24145919) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.982800:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2e254 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24141824->24145919] (req 24141824->24145919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.982903:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760735.982909:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2e254 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24141824->24145919] (req 24141824->24145919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.982918:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2e254 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24141824->24145919] (req 24141824->24145919) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.982938:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2e254 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24141824->24145919] (req 24141824->24145919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.982946:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2e254 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24141824->24145919] (req 24141824->24145919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760735.983037:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4130700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2358:x1876699512614656:12345-192.168.203.56@tcp:4:dd.0 Request processed in 40693us (40963us total) trans 4294973213 rc 0/0 00000100:00100000:3.0:1789760735.983047:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5924 00000100:00100000:3.0:1789760735.983065:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 715451164559 time 715506628082 state 0x2 ref 0 00000001:00080000:0.0:1789760735.984124:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973213 is committed 00000100:00100000:0.0:1789760735.986243:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760735.986337:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512614784 00000100:00100000:3.0:1789760735.986348:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512614784 00000100:00100000:3.0:1789760735.986409:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 715451164559 time 715506628082 state 0x0 ref 1 00000100:00100000:3.0:1789760735.986424:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 715506888498 time 715506628082 state 0x1 ref 1 00000100:00100000:3.0:1789760735.986556:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 715506888498 time 715550854623 state 0x0 ref 1 00000100:00100000:3.0:1789760735.986570:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac287a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512614784:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760735.986618:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24145920, mtime: 0x6aad94df, atime: 0x0, ctime: 0x6aad94df, blocks: 0xc007 00010000:00010000:3.0:1789760735.986627:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2e25b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.986643:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=715s v=(0 0) 00010000:00010000:3.0:1789760735.986648:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760735.986654:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2e25b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24145920->24150015] (req 24145920->24150015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.986668:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2e25b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24145920->24150015] (req 24145920->24150015) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760736.020720:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973214 is committed 00002000:00010000:3.0:1789760736.020910:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24145920 -> 24150016 00010000:00010000:3.0:1789760736.020925:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2e25b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24145920->24150015] (req 24145920->24150015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.020936:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2e25b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24145920->24150015] (req 24145920->24150015) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.020943:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2e25b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24145920->24150015] (req 24145920->24150015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.020947:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760736.020951:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2e25b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24145920->24150015] (req 24145920->24150015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.020958:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2e25b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24145920->24150015] (req 24145920->24150015) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.020971:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2e25b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24145920->24150015] (req 24145920->24150015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.020975:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2e25b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24145920->24150015] (req 24145920->24150015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760736.021108:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac287a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512614784:12345-192.168.203.56@tcp:4:dd.0 Request processed in 34532us (34864us total) trans 4294973214 rc 0/0 00000100:00100000:3.0:1789760736.021119:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5925 00000100:00100000:3.0:1789760736.021128:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 715506888498 time 715550854623 state 0x2 ref 0 00000100:00100000:1.0:1789760736.024385:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760736.024578:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512614912 00000100:00100000:3.0:1789760736.024587:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512614912 00000100:00100000:3.0:1789760736.024638:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 715506888498 time 715550854623 state 0x0 ref 1 00000100:00100000:3.0:1789760736.024653:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 715551115039 time 715550854623 state 0x1 ref 1 00000100:00100000:2.0:1789760736.024969:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 715551115039 time 715589267013 state 0x0 ref 1 00000100:00100000:2.0:1789760736.024984:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4131180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512614912:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760736.025028:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24150016, mtime: 0x6aad94df, atime: 0x0, ctime: 0x6aad94df, blocks: 0xc007 00010000:00010000:2.0:1789760736.025036:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2e262 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.025183:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=715s v=(0 0) 00010000:00010000:2.0:1789760736.025189:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760736.025196:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2e262 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24150016->24154111] (req 24150016->24154111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.025209:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2e262 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24150016->24154111] (req 24150016->24154111) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760736.076162:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973215 is committed 00002000:00010000:0.0:1789760736.076833:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24150016 -> 24154112 00010000:00010000:0.0:1789760736.076850:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2e262 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24150016->24154111] (req 24150016->24154111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.076873:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2e262 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24150016->24154111] (req 24150016->24154111) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.076882:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2e262 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24150016->24154111] (req 24150016->24154111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.076890:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760736.076895:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2e262 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24150016->24154111] (req 24150016->24154111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.076904:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2e262 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24150016->24154111] (req 24150016->24154111) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.076923:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2e262 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24150016->24154111] (req 24150016->24154111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.076930:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2e262 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24150016->24154111] (req 24150016->24154111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760736.077023:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4131180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512614912:12345-192.168.203.56@tcp:4:dd.0 Request processed in 52038us (52652us total) trans 4294973215 rc 0/0 00000100:00100000:0.0:1789760736.077033:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5926 00000100:00100000:0.0:1789760736.077042:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 715551115039 time 715589267013 state 0x2 ref 0 00000100:00100000:1.0:1789760736.080167:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760736.081118:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512615040 00000100:00100000:3.0:1789760736.081129:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512615040 00000100:00100000:3.0:1789760736.081184:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 715551115039 time 715589267013 state 0x0 ref 1 00000100:00100000:3.0:1789760736.081198:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 715589527429 time 715589267013 state 0x1 ref 1 00000100:00100000:3.0:1789760736.081231:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 715589527429 time 715645531388 state 0x0 ref 1 00000100:00100000:3.0:1789760736.081244:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4130a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512615040:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760736.081291:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24154112, mtime: 0x6aad94df, atime: 0x0, ctime: 0x6aad94df, blocks: 0xc007 00010000:00010000:3.0:1789760736.081300:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2e269 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.081317:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=715s v=(0 0) 00010000:00010000:3.0:1789760736.081321:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760736.081327:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2e269 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24154112->24158207] (req 24154112->24158207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.081341:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2e269 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24154112->24158207] (req 24154112->24158207) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760736.131390:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973216 is committed 00002000:00010000:2.0:1789760736.131624:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24154112 -> 24158208 00010000:00010000:2.0:1789760736.131643:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2e269 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24154112->24158207] (req 24154112->24158207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.131657:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2e269 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24154112->24158207] (req 24154112->24158207) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.131674:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2e269 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24154112->24158207] (req 24154112->24158207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.131682:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760736.131688:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2e269 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24154112->24158207] (req 24154112->24158207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.131715:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2e269 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24154112->24158207] (req 24154112->24158207) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.131740:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2e269 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24154112->24158207] (req 24154112->24158207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.131748:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2e269 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24154112->24158207] (req 24154112->24158207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760736.132140:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4130a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512615040:12345-192.168.203.56@tcp:4:dd.0 Request processed in 50892us (51976us total) trans 4294973216 rc 0/0 00000100:00100000:2.0:1789760736.132152:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5927 00000100:00100000:2.0:1789760736.132163:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 715589527429 time 715645531388 state 0x2 ref 0 00000100:00100000:3.0:1789760736.137658:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760736.137882:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512615168 00000100:00100000:2.0:1789760736.137893:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512615168 00000100:00100000:2.0:1789760736.137949:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 715589527429 time 715645531388 state 0x0 ref 1 00000100:00100000:2.0:1789760736.137965:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 715645791804 time 715645531388 state 0x1 ref 1 00000100:00100000:2.0:1789760736.138054:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 715645791804 time 715702346752 state 0x0 ref 1 00000100:00100000:2.0:1789760736.138070:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac531c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512615168:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760736.138123:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24158208, mtime: 0x6aad94df, atime: 0x0, ctime: 0x6aad94df, blocks: 0xc007 00010000:00010000:2.0:1789760736.138135:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2e270 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.138154:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=715s v=(0 0) 00010000:00010000:2.0:1789760736.138158:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760736.138165:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2e270 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24158208->24162303] (req 24158208->24162303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.138181:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2e270 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24158208->24162303] (req 24158208->24162303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760736.174431:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24158208 -> 24162304 00010000:00010000:2.0:1789760736.174451:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2e270 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24158208->24162303] (req 24158208->24162303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.174464:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2e270 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24158208->24162303] (req 24158208->24162303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.174473:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2e270 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24158208->24162303] (req 24158208->24162303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.174481:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760736.174487:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2e270 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24158208->24162303] (req 24158208->24162303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.174497:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2e270 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24158208->24162303] (req 24158208->24162303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.174517:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2e270 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24158208->24162303] (req 24158208->24162303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.174525:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2e270 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24158208->24162303] (req 24158208->24162303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760736.174621:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac531c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2358:x1876699512615168:12345-192.168.203.56@tcp:4:dd.0 Request processed in 36550us (36970us total) trans 4294973217 rc 0/0 00000100:00100000:2.0:1789760736.174631:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5928 00000100:00100000:2.0:1789760736.174641:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 715645791804 time 715702346752 state 0x2 ref 0 00000001:00080000:1.0:1789760736.175177:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973217 is committed 00000100:00100000:2.0:1789760736.179127:0:3311:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8febffb9c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3311:1876699540222848:0@lo:400:kworker.0 00000100:00100000:2.0:1789760736.179198:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760736.179249:0:3311:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8feac28f2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3311:1876699540223360:0@lo:400:kworker.0 00000100:00100000:2.0:1789760736.179276:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760736.179306:0:3311:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1789760736.179313:0:3311:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8febffb9f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3311:1876699540222976:0@lo:400:kworker.0 00000100:00100000:2.0:1789760736.179335:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760736.179367:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540222848 00000100:00100000:2.0:1789760736.179386:0:5847:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540222848 00000100:00100000:2.0:1789760736.179427:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540222976 00000100:00100000:2.0:1789760736.179431:0:5847:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540222976 00000100:00100000:2.0:1789760736.179459:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 466 00000100:00100000:2.0:1789760736.179468:0:5847:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac28f1180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3311:x1876699540222848:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760736.179518:0:5847:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac28f1180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3311:x1876699540222848:12345-0@lo:400:kworker.0 Request processed in 51us (320us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760736.179525:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 466 00000100:00100000:2.0:1789760736.179540:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 467 00000100:00100000:2.0:1789760736.179547:0:5847:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febfbeae300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3311:x1876699540222976:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760736.179574:0:5847:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febfbeae300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3311:x1876699540222976:12345-0@lo:400:kworker.0 Request processed in 29us (239us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760736.179580:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 467 00000100:00100000:2.0:1789760736.179598:0:6705:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540223360 00000100:00100000:2.0:1789760736.179601:0:6705:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540223360 00000100:00100000:2.0:1789760736.179632:0:6705:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 179 00000100:00100000:2.0:1789760736.179639:0:6705:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac28f0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3311:x1876699540223360:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760736.179668:0:6705:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac28f0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3311:x1876699540223360:12345-0@lo:400:kworker.0 Request processed in 30us (391us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760736.179674:0:6705:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 179 00000100:00080000:2.0:1789760736.179730:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:2.0:1789760736.179734:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789760736.179766:0:3311:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8febffb9f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3311:1876699540222976:0@lo:400:kworker.0 00000100:00100000:2.0:1789760736.179777:0:3311:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8febffb9c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3311:1876699540222848:0@lo:400:kworker.0 00000100:00100000:2.0:1789760736.179786:0:3311:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8feac28f2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3311:1876699540223360:0@lo:400:kworker.0 00000100:00100000:1.0:1789760736.180280:0:3309:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8feac38b6300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3309:1876699540223104:0@lo:400:kworker.0 00000100:00100000:1.0:1789760736.180320:0:3309:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760736.180354:0:3309:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:1.0:1789760736.180360:0:3309:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8febffb9c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:6df98d54-eb52-4c5c-8fc9-ee9d74f90f98:3309:1876699540222720:0@lo:400:kworker.0 00000100:00100000:1.0:1789760736.180399:0:3309:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760736.180422:0:3309:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8feac28f0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3309:1876699540223232:0@lo:400:kworker.0 00000100:00100000:1.0:1789760736.180444:0:3309:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760736.180735:0:3312:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8feac531e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3312:1876699540222592:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789760736.180768:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760736.180917:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540223232 00000100:00100000:2.0:1789760736.180923:0:5847:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540223232 00000100:00100000:2.0:1789760736.181210:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 468 00000100:00100000:2.0:1789760736.181219:0:5847:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2fb9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3309:x1876699540223232:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760736.181269:0:5847:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2fb9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3309:x1876699540223232:12345-0@lo:400:kworker.0 Request processed in 51us (823us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760736.181276:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 468 00000100:00080000:2.0:1789760736.181297:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789760736.181414:0:3309:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8feac28f0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3309:1876699540223232:0@lo:400:kworker.0 00000100:00100000:3.0:1789760736.181751:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540223104 00000100:00100000:3.0:1789760736.181756:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540223104 00000100:00100000:3.0:1789760736.181794:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 180 00000100:00100000:3.0:1789760736.181802:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4131500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3309:x1876699540223104:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760736.181840:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4131500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3309:x1876699540223104:12345-0@lo:400:kworker.0 Request processed in 40us (1519us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760736.181846:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 180 00000100:00100000:3.0:1789760736.181864:0:5842:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540222720 00000100:00100000:3.0:1789760736.181867:0:5842:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540222720 00000100:00100000:3.0:1789760736.181886:0:5842:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 395 00000100:00100000:3.0:1789760736.181892:0:5842:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febe0502050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:6df98d54-eb52-4c5c-8fc9-ee9d74f90f98+19:3309:x1876699540222720:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760736.181920:0:5842:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febe0502050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:6df98d54-eb52-4c5c-8fc9-ee9d74f90f98+19:3309:x1876699540222720:12345-0@lo:400:kworker.0 Request processed in 28us (1521us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760736.181926:0:5842:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 395 00000100:00080000:2.0:1789760736.182028:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1789760736.182384:0:3309:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8febffb9c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:6df98d54-eb52-4c5c-8fc9-ee9d74f90f98:3309:1876699540222720:0@lo:400:kworker.0 00000100:00100000:2.0:1789760736.182399:0:3309:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8feac38b6300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3309:1876699540223104:0@lo:400:kworker.0 00000100:00100000:1.0:1789760736.182476:0:6706:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540222592 00000100:00100000:1.0:1789760736.182481:0:6706:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540222592 00000100:00100000:1.0:1789760736.182519:0:6706:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 194 00000100:00100000:1.0:1789760736.182527:0:6706:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac28f2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3312:x1876699540222592:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1789760736.182543:0:6706:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 526188544 pending 0 free 3834642432 avail 3796893696 00000100:00100000:1.0:1789760736.182583:0:6706:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac28f2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3312:x1876699540222592:12345-0@lo:13:osp-pre-0-0.0 Request processed in 56us (1813us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760736.182589:0:6706:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 194 00000100:00100000:2.0:1789760736.183367:0:3312:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8feac531e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3312:1876699540222592:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1789760736.185232:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760736.185369:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512615296 00000100:00100000:3.0:1789760736.185386:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512615296 00000100:00100000:3.0:1789760736.185436:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 715645791804 time 715702346752 state 0x0 ref 1 00000100:00100000:3.0:1789760736.185450:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 715702607168 time 715702346752 state 0x1 ref 1 00000100:00100000:3.0:1789760736.185480:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 715702607168 time 715749779820 state 0x0 ref 1 00000100:00100000:3.0:1789760736.185493:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2fb8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512615296:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760736.185538:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24162304, mtime: 0x6aad94df, atime: 0x0, ctime: 0x6aad94df, blocks: 0xc007 00010000:00010000:3.0:1789760736.185547:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2e277 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.185567:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=715s v=(0 0) 00010000:00010000:3.0:1789760736.185571:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760736.185576:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2e277 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24162304->24166399] (req 24162304->24166399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.185591:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2e277 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24162304->24166399] (req 24162304->24166399) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760736.223859:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973218 is committed 00002000:00010000:2.0:1789760736.228059:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24162304 -> 24166400 00010000:00010000:2.0:1789760736.228083:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2e277 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24162304->24166399] (req 24162304->24166399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.228098:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2e277 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24162304->24166399] (req 24162304->24166399) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.228108:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2e277 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24162304->24166399] (req 24162304->24166399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.228115:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760736.228120:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2e277 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24162304->24166399] (req 24162304->24166399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.228134:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2e277 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24162304->24166399] (req 24162304->24166399) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.228154:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2e277 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24162304->24166399] (req 24162304->24166399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.228162:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2e277 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24162304->24166399] (req 24162304->24166399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760736.228258:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2fb8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512615296:12345-192.168.203.56@tcp:4:dd.0 Request processed in 42762us (43029us total) trans 4294973218 rc 0/0 00000100:00100000:2.0:1789760736.228270:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5929 00000100:00100000:2.0:1789760736.228280:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 715702607168 time 715749779820 state 0x2 ref 0 00000100:00100000:3.0:1789760736.232131:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760736.232212:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512615424 00000100:00100000:2.0:1789760736.232222:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512615424 00000100:00100000:2.0:1789760736.232277:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 715702607168 time 715749779820 state 0x0 ref 1 00000100:00100000:2.0:1789760736.232293:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 715750040236 time 715749779820 state 0x1 ref 1 00000100:00100000:2.0:1789760736.232326:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 715750040236 time 715796625766 state 0x0 ref 1 00000100:00100000:2.0:1789760736.232339:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac531fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512615424:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760736.232397:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24166400, mtime: 0x6aad94df, atime: 0x0, ctime: 0x6aad94df, blocks: 0xc007 00010000:00010000:2.0:1789760736.232406:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2e27e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.232422:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=715s v=(0 0) 00010000:00010000:2.0:1789760736.232427:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760736.232432:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2e27e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24166400->24170495] (req 24166400->24170495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.232445:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2e27e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24166400->24170495] (req 24166400->24170495) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760736.266292:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973219 is committed 00002000:00010000:2.0:1789760736.266408:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24166400 -> 24170496 00010000:00010000:2.0:1789760736.266425:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2e27e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24166400->24170495] (req 24166400->24170495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.266436:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2e27e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24166400->24170495] (req 24166400->24170495) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.266445:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2e27e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24166400->24170495] (req 24166400->24170495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.266452:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760736.266457:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2e27e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24166400->24170495] (req 24166400->24170495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.266467:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2e27e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24166400->24170495] (req 24166400->24170495) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.266486:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2e27e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24166400->24170495] (req 24166400->24170495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.266494:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2e27e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24166400->24170495] (req 24166400->24170495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760736.266582:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac531fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512615424:12345-192.168.203.56@tcp:4:dd.0 Request processed in 34241us (34457us total) trans 4294973219 rc 0/0 00000100:00100000:2.0:1789760736.266593:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5930 00000100:00100000:2.0:1789760736.266602:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 715750040236 time 715796625766 state 0x2 ref 0 00000100:00100000:3.0:1789760736.270261:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760736.270645:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512615552 00000100:00100000:2.0:1789760736.270654:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512615552 00000100:00100000:2.0:1789760736.271159:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 715750040236 time 715796625766 state 0x0 ref 1 00000100:00100000:2.0:1789760736.271176:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 715796886182 time 715796625766 state 0x1 ref 1 00000100:00100000:2.0:1789760736.271246:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 715796886182 time 715835544928 state 0x0 ref 1 00000100:00100000:2.0:1789760736.271260:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac531f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512615552:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760736.271318:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24170496, mtime: 0x6aad94df, atime: 0x0, ctime: 0x6aad94df, blocks: 0xc007 00010000:00010000:2.0:1789760736.271327:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2e285 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.271342:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=715s v=(0 0) 00010000:00010000:2.0:1789760736.271347:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760736.271353:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2e285 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24170496->24174591] (req 24170496->24174591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.271387:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2e285 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24170496->24174591] (req 24170496->24174591) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760736.324541:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24170496 -> 24174592 00010000:00010000:0.0:1789760736.324564:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2e285 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24170496->24174591] (req 24170496->24174591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.324576:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2e285 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24170496->24174591] (req 24170496->24174591) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.324585:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2e285 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24170496->24174591] (req 24170496->24174591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.324599:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760736.324603:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2e285 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24170496->24174591] (req 24170496->24174591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.324613:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2e285 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24170496->24174591] (req 24170496->24174591) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.324632:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2e285 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24170496->24174591] (req 24170496->24174591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.324640:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2e285 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24170496->24174591] (req 24170496->24174591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760736.324769:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac531f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2359:x1876699512615552:12345-192.168.203.56@tcp:4:dd.0 Request processed in 53508us (54513us total) trans 4294973220 rc 0/0 00000100:00100000:0.0:1789760736.324779:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5931 00000100:00100000:0.0:1789760736.324789:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 715796886182 time 715835544928 state 0x2 ref 0 00000001:00080000:1.0:1789760736.325222:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973220 is committed 00000100:00100000:1.0:1789760736.327190:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760736.327335:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512615680 00000100:00100000:2.0:1789760736.327344:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512615680 00000100:00100000:2.0:1789760736.327406:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 715796886182 time 715835544928 state 0x0 ref 1 00000100:00100000:2.0:1789760736.327420:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 715835805344 time 715835544928 state 0x1 ref 1 00000100:00100000:2.0:1789760736.327449:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 715835805344 time 715891748563 state 0x0 ref 1 00000100:00100000:2.0:1789760736.327625:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2fba680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512615680:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760736.327674:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24174592, mtime: 0x6aad94df, atime: 0x0, ctime: 0x6aad94df, blocks: 0xc007 00010000:00010000:2.0:1789760736.327683:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2e28c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.327731:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=715s v=(0 0) 00010000:00010000:2.0:1789760736.327736:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760736.327742:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2e28c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24174592->24178687] (req 24174592->24178687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.327755:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2e28c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24174592->24178687] (req 24174592->24178687) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760736.389059:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973221 is committed 00002000:00010000:3.0:1789760736.389271:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24174592 -> 24178688 00010000:00010000:3.0:1789760736.389286:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2e28c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24174592->24178687] (req 24174592->24178687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.389300:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2e28c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24174592->24178687] (req 24174592->24178687) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.389309:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2e28c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24174592->24178687] (req 24174592->24178687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.389316:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760736.389321:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2e28c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24174592->24178687] (req 24174592->24178687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.389329:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2e28c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24174592->24178687] (req 24174592->24178687) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.389349:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2e28c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24174592->24178687] (req 24174592->24178687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.389357:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2e28c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24174592->24178687] (req 24174592->24178687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760736.389560:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2fba680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512615680:12345-192.168.203.56@tcp:4:dd.0 Request processed in 61934us (62372us total) trans 4294973221 rc 0/0 00000100:00100000:3.0:1789760736.389570:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5932 00000100:00100000:3.0:1789760736.389581:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 715835805344 time 715891748563 state 0x2 ref 0 00000100:00100000:2.0:1789760736.401073:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760736.401162:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512615808 00000100:00100000:0.0:1789760736.401173:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512615808 00000100:00100000:0.0:1789760736.401228:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 715835805344 time 715891748563 state 0x0 ref 1 00000100:00100000:0.0:1789760736.401245:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 715892008979 time 715891748563 state 0x1 ref 1 00000100:00100000:0.0:1789760736.401276:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 715892008979 time 715965575422 state 0x0 ref 1 00000100:00100000:0.0:1789760736.401289:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac28f0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512615808:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760736.401337:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24178688, mtime: 0x6aad94df, atime: 0x0, ctime: 0x6aad94df, blocks: 0xc007 00010000:00010000:0.0:1789760736.401347:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2e293 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.401365:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=715s v=(0 0) 00010000:00010000:0.0:1789760736.401370:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760736.401388:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2e293 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24178688->24182783] (req 24178688->24182783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.401402:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2e293 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24178688->24182783] (req 24178688->24182783) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760736.454214:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24178688 -> 24182784 00010000:00010000:0.0:1789760736.454230:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2e293 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24178688->24182783] (req 24178688->24182783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.454240:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2e293 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24178688->24182783] (req 24178688->24182783) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.454246:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2e293 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24178688->24182783] (req 24178688->24182783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.454250:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760736.454254:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2e293 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24178688->24182783] (req 24178688->24182783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.454261:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2e293 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24178688->24182783] (req 24178688->24182783) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.454276:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2e293 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24178688->24182783] (req 24178688->24182783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.454281:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2e293 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24178688->24182783] (req 24178688->24182783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760736.454356:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac28f0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2359:x1876699512615808:12345-192.168.203.56@tcp:4:dd.0 Request processed in 53065us (53289us total) trans 4294973222 rc 0/0 00000100:00100000:0.0:1789760736.454382:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5933 00000100:00100000:0.0:1789760736.454392:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 715892008979 time 715965575422 state 0x2 ref 0 00000001:00080000:1.0:1789760736.454842:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973222 is committed 00000100:00100000:0.0:1789760736.457689:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760736.457946:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512615936 00000100:00100000:2.0:1789760736.457956:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512615936 00000100:00100000:2.0:1789760736.458014:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 715892008979 time 715965575422 state 0x0 ref 1 00000100:00100000:2.0:1789760736.458028:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 715965835838 time 715965575422 state 0x1 ref 1 00000100:00100000:2.0:1789760736.458047:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 715965835838 time 716022347828 state 0x0 ref 1 00000100:00100000:2.0:1789760736.458058:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2879c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512615936:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760736.458261:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24182784, mtime: 0x6aad94df, atime: 0x0, ctime: 0x6aad94df, blocks: 0xc007 00010000:00010000:2.0:1789760736.458270:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2e29a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.458285:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=716s v=(0 0) 00010000:00010000:2.0:1789760736.458289:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760736.458295:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2e29a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24182784->24186879] (req 24182784->24186879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.458308:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2e29a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24182784->24186879] (req 24182784->24186879) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760736.497834:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973223 is committed 00002000:00010000:2.0:1789760736.498018:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24182784 -> 24186880 00010000:00010000:2.0:1789760736.498033:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2e29a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24182784->24186879] (req 24182784->24186879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.498057:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2e29a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24182784->24186879] (req 24182784->24186879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.498072:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2e29a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24182784->24186879] (req 24182784->24186879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.498079:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760736.498084:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2e29a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24182784->24186879] (req 24182784->24186879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.498535:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2e29a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24182784->24186879] (req 24182784->24186879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.498560:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2e29a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24182784->24186879] (req 24182784->24186879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.498569:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2e29a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24182784->24186879] (req 24182784->24186879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760736.498673:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2879c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512615936:12345-192.168.203.56@tcp:4:dd.0 Request processed in 40610us (40987us total) trans 4294973223 rc 0/0 00000100:00100000:2.0:1789760736.498684:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5934 00000100:00100000:2.0:1789760736.498693:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 715965835838 time 716022347828 state 0x2 ref 0 00000100:00100000:3.0:1789760736.503208:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760736.503389:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512616064 00000100:00100000:2.0:1789760736.503400:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512616064 00000100:00100000:2.0:1789760736.503454:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 715965835838 time 716022347828 state 0x0 ref 1 00000100:00100000:2.0:1789760736.503471:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 716022608244 time 716022347828 state 0x1 ref 1 00000100:00100000:2.0:1789760736.503541:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 716022608244 time 716067840370 state 0x0 ref 1 00000100:00100000:2.0:1789760736.503555:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac531c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512616064:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760736.503604:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24186880, mtime: 0x6aad94df, atime: 0x0, ctime: 0x6aad94df, blocks: 0xc007 00010000:00010000:2.0:1789760736.503613:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2e2a1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.503629:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=716s v=(0 0) 00010000:00010000:2.0:1789760736.503634:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760736.503640:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2e2a1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24186880->24190975] (req 24186880->24190975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.503654:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2e2a1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24186880->24190975] (req 24186880->24190975) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760736.532418:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973224 is committed 00002000:00010000:0.0:1789760736.532837:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24186880 -> 24190976 00002000:00010000:0.0:1789760736.532841:0:6710:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760735 -> 1789760736 00002000:00010000:0.0:1789760736.532844:0:6710:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760735 -> 1789760736 00010000:00010000:0.0:1789760736.532857:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2e2a1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24186880->24190975] (req 24186880->24190975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.532870:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2e2a1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24186880->24190975] (req 24186880->24190975) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.532879:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2e2a1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24186880->24190975] (req 24186880->24190975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.532886:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760736.532891:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2e2a1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24186880->24190975] (req 24186880->24190975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.532900:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2e2a1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24186880->24190975] (req 24186880->24190975) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.532919:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2e2a1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24186880->24190975] (req 24186880->24190975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.532928:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2e2a1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24186880->24190975] (req 24186880->24190975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760736.533008:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac531c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512616064:12345-192.168.203.56@tcp:4:dd.0 Request processed in 29453us (29806us total) trans 4294973224 rc 0/0 00000100:00100000:0.0:1789760736.533015:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5935 00000100:00100000:0.0:1789760736.533023:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 716022608244 time 716067840370 state 0x2 ref 0 00000100:00100000:2.0:1789760736.537439:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760736.538236:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512616192 00000100:00100000:0.0:1789760736.538246:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512616192 00000100:00100000:0.0:1789760736.538298:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 716022608244 time 716067840370 state 0x0 ref 1 00000100:00100000:0.0:1789760736.538313:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 716068100786 time 716067840370 state 0x1 ref 1 00000100:00100000:0.0:1789760736.538343:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 716068100786 time 716102643417 state 0x0 ref 1 00000100:00100000:0.0:1789760736.538360:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac38b6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512616192:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760736.538416:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24190976, mtime: 0x6aad94e0, atime: 0x0, ctime: 0x6aad94e0, blocks: 0xc007 00010000:00010000:0.0:1789760736.538425:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2e2a8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.538441:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=716s v=(0 0) 00010000:00010000:0.0:1789760736.538446:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760736.538451:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2e2a8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24190976->24195071] (req 24190976->24195071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.538474:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2e2a8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24190976->24195071] (req 24190976->24195071) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760736.570693:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24190976 -> 24195072 00010000:00010000:0.0:1789760736.570741:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2e2a8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24190976->24195071] (req 24190976->24195071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.570754:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2e2a8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24190976->24195071] (req 24190976->24195071) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.570837:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2e2a8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24190976->24195071] (req 24190976->24195071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.570846:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760736.570851:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2e2a8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24190976->24195071] (req 24190976->24195071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.570861:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2e2a8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24190976->24195071] (req 24190976->24195071) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.570880:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2e2a8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24190976->24195071] (req 24190976->24195071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.570889:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2e2a8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24190976->24195071] (req 24190976->24195071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760736.570984:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac38b6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2358:x1876699512616192:12345-192.168.203.56@tcp:4:dd.0 Request processed in 32622us (33551us total) trans 4294973225 rc 0/0 00000100:00100000:0.0:1789760736.570994:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5936 00000100:00100000:0.0:1789760736.571004:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 716068100786 time 716102643417 state 0x2 ref 0 00000001:00080000:1.0:1789760736.571328:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973225 is committed 00000100:00100000:3.0:1789760736.575073:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760736.575280:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512616320 00000100:00100000:2.0:1789760736.575290:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512616320 00000100:00100000:2.0:1789760736.575353:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 716068100786 time 716102643417 state 0x0 ref 1 00000100:00100000:2.0:1789760736.575369:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 716102903833 time 716102643417 state 0x1 ref 1 00000100:00100000:2.0:1789760736.575444:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 716102903833 time 716139743317 state 0x0 ref 1 00000100:00100000:2.0:1789760736.575457:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac531ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512616320:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760736.575513:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24195072, mtime: 0x6aad94e0, atime: 0x0, ctime: 0x6aad94e0, blocks: 0xc007 00010000:00010000:2.0:1789760736.575523:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2e2af lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.575539:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=716s v=(0 0) 00010000:00010000:2.0:1789760736.575543:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760736.575549:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2e2af lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24195072->24199167] (req 24195072->24199167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.575563:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2e2af lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24195072->24199167] (req 24195072->24199167) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760736.617347:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973226 is committed 00002000:00010000:3.0:1789760736.617458:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24195072 -> 24199168 00010000:00010000:3.0:1789760736.617473:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2e2af lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24195072->24199167] (req 24195072->24199167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.617484:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2e2af lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24195072->24199167] (req 24195072->24199167) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.617492:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2e2af lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24195072->24199167] (req 24195072->24199167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.617498:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760736.617502:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2e2af lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24195072->24199167] (req 24195072->24199167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.617510:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2e2af lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24195072->24199167] (req 24195072->24199167) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.617528:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2e2af lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24195072->24199167] (req 24195072->24199167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.617535:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2e2af lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24195072->24199167] (req 24195072->24199167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760736.617642:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac531ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512616320:12345-192.168.203.56@tcp:4:dd.0 Request processed in 42181us (42572us total) trans 4294973226 rc 0/0 00000100:00100000:3.0:1789760736.617652:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5937 00000100:00100000:3.0:1789760736.617661:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 716102903833 time 716139743317 state 0x2 ref 0 00000100:00100000:2.0:1789760736.620607:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760736.620820:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512616448 00000100:00100000:0.0:1789760736.620831:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512616448 00000100:00100000:0.0:1789760736.620884:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 716102903833 time 716139743317 state 0x0 ref 1 00000100:00100000:0.0:1789760736.620899:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 716140003733 time 716139743317 state 0x1 ref 1 00000100:00100000:0.0:1789760736.620958:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 716140003733 time 716185257378 state 0x0 ref 1 00000100:00100000:0.0:1789760736.620971:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febffb9c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512616448:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760736.621018:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24199168, mtime: 0x6aad94e0, atime: 0x0, ctime: 0x6aad94e0, blocks: 0xc007 00010000:00010000:0.0:1789760736.621027:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2e2b6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.621101:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=716s v=(0 0) 00010000:00010000:0.0:1789760736.621106:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760736.621113:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2e2b6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24199168->24203263] (req 24199168->24203263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.621125:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2e2b6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24199168->24203263] (req 24199168->24203263) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.628077:0:5835:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:1.0:1789760736.647520:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973227 is committed 00002000:00010000:2.0:1789760736.647618:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24199168 -> 24203264 00010000:00010000:2.0:1789760736.647634:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2e2b6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24199168->24203263] (req 24199168->24203263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.647647:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2e2b6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24199168->24203263] (req 24199168->24203263) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.647656:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2e2b6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24199168->24203263] (req 24199168->24203263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.647664:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760736.647669:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2e2b6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24199168->24203263] (req 24199168->24203263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.647683:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2e2b6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24199168->24203263] (req 24199168->24203263) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.647730:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2e2b6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24199168->24203263] (req 24199168->24203263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.647739:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2e2b6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24199168->24203263] (req 24199168->24203263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760736.647929:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febffb9c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512616448:12345-192.168.203.56@tcp:4:dd.0 Request processed in 26955us (27327us total) trans 4294973227 rc 0/0 00000100:00100000:2.0:1789760736.647940:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5938 00000100:00100000:2.0:1789760736.647950:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 716140003733 time 716185257378 state 0x2 ref 0 00000100:00100000:0.0:1789760736.650953:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760736.651189:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512616576 00000100:00100000:2.0:1789760736.651198:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512616576 00000100:00100000:2.0:1789760736.651252:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 716140003733 time 716185257378 state 0x0 ref 1 00000100:00100000:2.0:1789760736.651267:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 716185517794 time 716185257378 state 0x1 ref 1 00000100:00100000:0.0:1789760736.651333:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 716185517794 time 716215631065 state 0x0 ref 1 00000100:00100000:0.0:1789760736.651348:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2878e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512616576:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760736.651404:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24203264, mtime: 0x6aad94e0, atime: 0x0, ctime: 0x6aad94e0, blocks: 0xc007 00010000:00010000:0.0:1789760736.651413:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2e2bd lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.651428:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=716s v=(0 0) 00010000:00010000:0.0:1789760736.651433:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760736.651440:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2e2bd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24203264->24207359] (req 24203264->24207359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.651454:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2e2bd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24203264->24207359] (req 24203264->24207359) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760736.678212:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973228 is committed 00002000:00010000:1.0:1789760736.678427:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24203264 -> 24207360 00010000:00010000:1.0:1789760736.678504:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2e2bd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24203264->24207359] (req 24203264->24207359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.678520:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2e2bd lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24203264->24207359] (req 24203264->24207359) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.678529:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2e2bd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24203264->24207359] (req 24203264->24207359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.678537:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760736.678542:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2e2bd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24203264->24207359] (req 24203264->24207359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.678551:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2e2bd lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24203264->24207359] (req 24203264->24207359) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.678571:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2e2bd lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24203264->24207359] (req 24203264->24207359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.678579:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2e2bd lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24203264->24207359] (req 24203264->24207359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760736.678672:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2878e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512616576:12345-192.168.203.56@tcp:4:dd.0 Request processed in 27322us (27724us total) trans 4294973228 rc 0/0 00000100:00100000:1.0:1789760736.678683:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5939 00000100:00100000:1.0:1789760736.678693:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 716185517794 time 716215631065 state 0x2 ref 0 00000100:00100000:0.0:1789760736.681862:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760736.681984:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512616704 00000100:00100000:3.0:1789760736.681994:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512616704 00000100:00100000:3.0:1789760736.682056:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 716185517794 time 716215631065 state 0x0 ref 1 00000100:00100000:3.0:1789760736.682072:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 716215891481 time 716215631065 state 0x1 ref 1 00000100:00100000:3.0:1789760736.682138:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 716215891481 time 716246437488 state 0x0 ref 1 00000100:00100000:3.0:1789760736.682151:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac287b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512616704:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760736.682197:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24207360, mtime: 0x6aad94e0, atime: 0x0, ctime: 0x6aad94e0, blocks: 0xc007 00010000:00010000:3.0:1789760736.682206:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2e2c4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.682222:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=716s v=(0 0) 00010000:00010000:3.0:1789760736.682226:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760736.682233:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2e2c4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24207360->24211455] (req 24207360->24211455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.682247:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2e2c4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24207360->24211455] (req 24207360->24211455) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760736.693967:0:3310:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1789760736.693990:0:3310:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8feac2fb8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3310:1876699540223488:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1789760736.694070:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760736.694263:0:6706:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540223488 00000100:00100000:1.0:1789760736.694270:0:6706:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540223488 00000100:00100000:1.0:1789760736.694319:0:6706:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 195 00000100:00100000:1.0:1789760736.694329:0:6706:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2879f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3310:x1876699540223488:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1789760736.694361:0:6706:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3859808256 avail 3857711104 00000100:00100000:1.0:1789760736.694422:0:6706:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2879f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3310:x1876699540223488:12345-0@lo:13:osp-pre-1-0.0 Request processed in 94us (351us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760736.694429:0:6706:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 195 00000100:00100000:0.0:1789760736.694582:0:3310:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8feac2fb8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3310:1876699540223488:0@lo:13:osp-pre-1-0.0 00000001:00080000:1.0:1789760736.718955:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973229 is committed 00002000:00010000:2.0:1789760736.719109:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24207360 -> 24211456 00010000:00010000:2.0:1789760736.719125:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2e2c4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24207360->24211455] (req 24207360->24211455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.719138:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2e2c4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24207360->24211455] (req 24207360->24211455) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.719154:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2e2c4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24207360->24211455] (req 24207360->24211455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.719162:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760736.719167:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2e2c4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24207360->24211455] (req 24207360->24211455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.719177:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2e2c4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24207360->24211455] (req 24207360->24211455) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.719196:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2e2c4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24207360->24211455] (req 24207360->24211455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.719204:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2e2c4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24207360->24211455] (req 24207360->24211455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760736.719298:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac287b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512616704:12345-192.168.203.56@tcp:4:dd.0 Request processed in 37144us (37439us total) trans 4294973229 rc 0/0 00000100:00100000:2.0:1789760736.719307:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5940 00000100:00100000:2.0:1789760736.719317:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 716215891481 time 716246437488 state 0x2 ref 0 00000100:00100000:2.0:1789760736.724011:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760736.724217:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512616832 00000100:00100000:0.0:1789760736.724228:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512616832 00000100:00100000:0.0:1789760736.724287:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 716215891481 time 716246437488 state 0x0 ref 1 00000100:00100000:0.0:1789760736.724302:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 716246697904 time 716246437488 state 0x1 ref 1 00000100:00100000:0.0:1789760736.724331:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 716246697904 time 716288630776 state 0x0 ref 1 00000100:00100000:0.0:1789760736.724344:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febffb9c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512616832:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760736.724403:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24211456, mtime: 0x6aad94e0, atime: 0x0, ctime: 0x6aad94e0, blocks: 0xc007 00010000:00010000:0.0:1789760736.724411:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2e2cb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.724427:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=716s v=(0 0) 00010000:00010000:0.0:1789760736.724431:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760736.724448:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2e2cb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24211456->24215551] (req 24211456->24215551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.724462:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2e2cb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24211456->24215551] (req 24211456->24215551) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760736.760753:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973230 is committed 00002000:00010000:0.0:1789760736.762344:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24211456 -> 24215552 00010000:00010000:0.0:1789760736.762364:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2e2cb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24211456->24215551] (req 24211456->24215551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.762385:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2e2cb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24211456->24215551] (req 24211456->24215551) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.762394:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2e2cb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24211456->24215551] (req 24211456->24215551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.762401:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760736.762406:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2e2cb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24211456->24215551] (req 24211456->24215551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.762414:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2e2cb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24211456->24215551] (req 24211456->24215551) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.762484:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2e2cb lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24211456->24215551] (req 24211456->24215551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.762498:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2e2cb lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24211456->24215551] (req 24211456->24215551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760736.762638:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febffb9c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512616832:12345-192.168.203.56@tcp:4:dd.0 Request processed in 38290us (38631us total) trans 4294973230 rc 0/0 00000100:00100000:0.0:1789760736.762652:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5941 00000100:00100000:0.0:1789760736.762661:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 716246697904 time 716288630776 state 0x2 ref 0 00000100:00100000:3.0:1789760736.765068:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760736.765143:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512616960 00000100:00100000:1.0:1789760736.765154:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512616960 00000100:00100000:1.0:1789760736.765212:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 716246697904 time 716288630776 state 0x0 ref 1 00000100:00100000:1.0:1789760736.765228:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 716288891192 time 716288630776 state 0x1 ref 1 00000100:00100000:1.0:1789760736.765260:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 716288891192 time 716329559753 state 0x0 ref 1 00000100:00100000:1.0:1789760736.765272:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac531d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512616960:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760736.765319:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24215552, mtime: 0x6aad94e0, atime: 0x0, ctime: 0x6aad94e0, blocks: 0xc007 00010000:00010000:1.0:1789760736.765327:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2e2d2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.765347:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=716s v=(0 0) 00010000:00010000:1.0:1789760736.765352:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760736.765358:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2e2d2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24215552->24219647] (req 24215552->24219647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.765381:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2e2d2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24215552->24219647] (req 24215552->24219647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760736.812286:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973231 is committed 00002000:00010000:1.0:1789760736.812308:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24215552 -> 24219648 00010000:00010000:1.0:1789760736.812325:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2e2d2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24215552->24219647] (req 24215552->24219647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.812337:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2e2d2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24215552->24219647] (req 24215552->24219647) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.812345:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2e2d2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24215552->24219647] (req 24215552->24219647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.812351:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760736.812356:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2e2d2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24215552->24219647] (req 24215552->24219647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.812365:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2e2d2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24215552->24219647] (req 24215552->24219647) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.812448:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2e2d2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24215552->24219647] (req 24215552->24219647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.812455:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2e2d2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24215552->24219647] (req 24215552->24219647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760736.812654:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac531d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512616960:12345-192.168.203.56@tcp:4:dd.0 Request processed in 47379us (47590us total) trans 4294973231 rc 0/0 00000100:00100000:1.0:1789760736.812664:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5942 00000100:00100000:1.0:1789760736.812673:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 716288891192 time 716329559753 state 0x2 ref 0 00000100:00100000:3.0:1789760736.815485:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760736.815679:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512617088 00000100:00100000:2.0:1789760736.815688:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512617088 00000100:00100000:2.0:1789760736.815760:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 716288891192 time 716329559753 state 0x0 ref 1 00000100:00100000:2.0:1789760736.815845:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 716329820169 time 716329559753 state 0x1 ref 1 00000100:00100000:2.0:1789760736.815881:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 716329820169 time 716380181486 state 0x0 ref 1 00000100:00100000:2.0:1789760736.815893:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac531e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512617088:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760736.816011:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24219648, mtime: 0x6aad94e0, atime: 0x0, ctime: 0x6aad94e0, blocks: 0xc007 00010000:00010000:2.0:1789760736.816020:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2e2d9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.816035:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=716s v=(0 0) 00010000:00010000:2.0:1789760736.816100:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760736.816108:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2e2d9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24219648->24223743] (req 24219648->24223743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.816122:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2e2d9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24219648->24223743] (req 24219648->24223743) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760736.858501:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973232 is committed 00002000:00010000:2.0:1789760736.858862:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24219648 -> 24223744 00010000:00010000:2.0:1789760736.858878:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2e2d9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24219648->24223743] (req 24219648->24223743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.858891:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2e2d9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24219648->24223743] (req 24219648->24223743) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.858899:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2e2d9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24219648->24223743] (req 24219648->24223743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.858912:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760736.858917:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2e2d9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24219648->24223743] (req 24219648->24223743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.858936:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2e2d9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24219648->24223743] (req 24219648->24223743) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.858956:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2e2d9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24219648->24223743] (req 24219648->24223743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.858964:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2e2d9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24219648->24223743] (req 24219648->24223743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760736.859062:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac531e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512617088:12345-192.168.203.56@tcp:4:dd.0 Request processed in 43165us (43579us total) trans 4294973232 rc 0/0 00000100:00100000:2.0:1789760736.859072:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5943 00000100:00100000:2.0:1789760736.859082:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 716329820169 time 716380181486 state 0x2 ref 0 00000100:00100000:2.0:1789760736.870305:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760736.870791:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512617216 00000100:00100000:0.0:1789760736.870801:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512617216 00000100:00100000:0.0:1789760736.870954:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 716329820169 time 716380181486 state 0x0 ref 1 00000100:00100000:0.0:1789760736.871225:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 716380441902 time 716380181486 state 0x1 ref 1 00000100:00100000:0.0:1789760736.871510:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 716380441902 time 716435808489 state 0x0 ref 1 00000100:00100000:0.0:1789760736.872279:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febffb9f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512617216:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760736.872334:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24223744, mtime: 0x6aad94e0, atime: 0x0, ctime: 0x6aad94e0, blocks: 0xc007 00010000:00010000:0.0:1789760736.872343:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2e2e0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.872359:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=716s v=(0 0) 00010000:00010000:0.0:1789760736.872363:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760736.872367:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2e2e0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24223744->24227839] (req 24223744->24227839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.872401:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2e2e0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24223744->24227839] (req 24223744->24227839) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760736.903989:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973233 is committed 00002000:00010000:1.0:1789760736.904129:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24223744 -> 24227840 00010000:00010000:1.0:1789760736.904145:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2e2e0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24223744->24227839] (req 24223744->24227839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.904158:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2e2e0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24223744->24227839] (req 24223744->24227839) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.904168:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2e2e0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24223744->24227839] (req 24223744->24227839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.904174:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760736.904179:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2e2e0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24223744->24227839] (req 24223744->24227839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.904188:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2e2e0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24223744->24227839] (req 24223744->24227839) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.904212:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2e2e0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24223744->24227839] (req 24223744->24227839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.904220:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2e2e0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24223744->24227839] (req 24223744->24227839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760736.904307:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febffb9f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512617216:12345-192.168.203.56@tcp:4:dd.0 Request processed in 32780us (34031us total) trans 4294973233 rc 0/0 00000100:00100000:1.0:1789760736.904317:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5944 00000100:00100000:1.0:1789760736.904327:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 716380441902 time 716435808489 state 0x2 ref 0 00000100:00100000:3.0:1789760736.907125:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760736.907362:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512617344 00000100:00100000:1.0:1789760736.907379:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512617344 00000100:00100000:1.0:1789760736.907440:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 716380441902 time 716435808489 state 0x0 ref 1 00000100:00100000:1.0:1789760736.907456:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 716436068905 time 716435808489 state 0x1 ref 1 00000100:00100000:1.0:1789760736.907485:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 716436068905 time 716471785370 state 0x0 ref 1 00000100:00100000:1.0:1789760736.907497:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febfc0a3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512617344:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760736.907551:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24227840, mtime: 0x6aad94e0, atime: 0x0, ctime: 0x6aad94e0, blocks: 0xc007 00010000:00010000:1.0:1789760736.907563:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2e2e7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.907581:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=716s v=(0 0) 00010000:00010000:1.0:1789760736.907588:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760736.907594:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2e2e7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24227840->24231935] (req 24227840->24231935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.907611:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2e2e7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24227840->24231935] (req 24227840->24231935) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760736.937615:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973234 is committed 00002000:00010000:3.0:1789760736.937678:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24227840 -> 24231936 00010000:00010000:3.0:1789760736.937694:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2e2e7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24227840->24231935] (req 24227840->24231935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.937738:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2e2e7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24227840->24231935] (req 24227840->24231935) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.937747:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2e2e7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24227840->24231935] (req 24227840->24231935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.937755:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760736.937760:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2e2e7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24227840->24231935] (req 24227840->24231935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.937770:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2e2e7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24227840->24231935] (req 24227840->24231935) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.937789:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2e2e7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24227840->24231935] (req 24227840->24231935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.937802:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2e2e7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24227840->24231935] (req 24227840->24231935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760736.937961:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febfc0a3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512617344:12345-192.168.203.56@tcp:4:dd.0 Request processed in 30460us (30842us total) trans 4294973234 rc 0/0 00000100:00100000:3.0:1789760736.937971:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5945 00000100:00100000:3.0:1789760736.937982:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 716436068905 time 716471785370 state 0x2 ref 0 00000100:00100000:3.0:1789760736.943101:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760736.943424:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512617472 00000100:00100000:2.0:1789760736.943434:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512617472 00000100:00100000:2.0:1789760736.943488:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 716436068905 time 716471785370 state 0x0 ref 1 00000100:00100000:2.0:1789760736.943505:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 716472045786 time 716471785370 state 0x1 ref 1 00000100:00100000:2.0:1789760736.943537:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 716472045786 time 716507836767 state 0x0 ref 1 00000100:00100000:2.0:1789760736.943551:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febfc0a3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512617472:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760736.943600:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24231936, mtime: 0x6aad94e0, atime: 0x0, ctime: 0x6aad94e0, blocks: 0xc007 00010000:00010000:2.0:1789760736.943610:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2e2ee lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.943626:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=716s v=(0 0) 00010000:00010000:2.0:1789760736.943631:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760736.943638:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2e2ee lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24231936->24236031] (req 24231936->24236031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.943651:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2e2ee lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24231936->24236031] (req 24231936->24236031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760736.982927:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973235 is committed 00002000:00010000:2.0:1789760736.983107:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24231936 -> 24236032 00010000:00010000:2.0:1789760736.983124:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2e2ee lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24231936->24236031] (req 24231936->24236031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.983136:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2e2ee lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24231936->24236031] (req 24231936->24236031) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.983145:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2e2ee lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24231936->24236031] (req 24231936->24236031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.983152:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760736.983157:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2e2ee lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24231936->24236031] (req 24231936->24236031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.983166:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2e2ee lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24231936->24236031] (req 24231936->24236031) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.983186:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2e2ee lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24231936->24236031] (req 24231936->24236031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.983335:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2e2ee lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24231936->24236031] (req 24231936->24236031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760736.983530:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febfc0a3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512617472:12345-192.168.203.56@tcp:4:dd.0 Request processed in 39976us (40434us total) trans 4294973235 rc 0/0 00000100:00100000:2.0:1789760736.983540:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5946 00000100:00100000:2.0:1789760736.983549:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 716472045786 time 716507836767 state 0x2 ref 0 00000100:00100000:0.0:1789760736.990333:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760736.990447:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512617600 00000100:00100000:1.0:1789760736.990469:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512617600 00000100:00100000:1.0:1789760736.990530:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 716472045786 time 716507836767 state 0x0 ref 1 00000100:00100000:1.0:1789760736.990546:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 716508097183 time 716507836767 state 0x1 ref 1 00000100:00100000:1.0:1789760736.990576:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 716508097183 time 716554875947 state 0x0 ref 1 00000100:00100000:1.0:1789760736.990589:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2878380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512617600:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760736.990638:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24236032, mtime: 0x6aad94e0, atime: 0x0, ctime: 0x6aad94e0, blocks: 0xc007 00010000:00010000:1.0:1789760736.990657:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2e2f5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.990674:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=716s v=(0 0) 00010000:00010000:1.0:1789760736.990678:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760736.990684:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2e2f5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24236032->24240127] (req 24236032->24240127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.990733:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2e2f5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24236032->24240127] (req 24236032->24240127) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760737.026217:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973236 is committed 00002000:00010000:3.0:1789760737.026358:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24236032 -> 24240128 00010000:00010000:3.0:1789760737.026384:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2e2f5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24236032->24240127] (req 24236032->24240127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.026397:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2e2f5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24236032->24240127] (req 24236032->24240127) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.026407:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2e2f5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24236032->24240127] (req 24236032->24240127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.026414:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760737.026419:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2e2f5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24236032->24240127] (req 24236032->24240127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.026428:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2e2f5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24236032->24240127] (req 24236032->24240127) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.026452:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2e2f5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24236032->24240127] (req 24236032->24240127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.026470:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2e2f5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24236032->24240127] (req 24236032->24240127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760737.026675:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2878380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512617600:12345-192.168.203.56@tcp:4:dd.0 Request processed in 36083us (36348us total) trans 4294973236 rc 0/0 00000100:00100000:3.0:1789760737.026686:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5947 00000100:00100000:3.0:1789760737.026733:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 716508097183 time 716554875947 state 0x2 ref 0 00000100:00100000:3.0:1789760737.030314:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760737.030524:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512617728 00000100:00100000:1.0:1789760737.030534:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512617728 00000100:00100000:1.0:1789760737.030603:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 716508097183 time 716554875947 state 0x0 ref 1 00000100:00100000:1.0:1789760737.030619:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 716555136363 time 716554875947 state 0x1 ref 1 00000100:00100000:1.0:1789760737.030652:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 716555136363 time 716594952358 state 0x0 ref 1 00000100:00100000:1.0:1789760737.030665:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac28a2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512617728:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760737.030743:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24240128, mtime: 0x6aad94e0, atime: 0x0, ctime: 0x6aad94e0, blocks: 0xc007 00010000:00010000:1.0:1789760737.030752:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2e2fc lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.030767:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=716s v=(0 0) 00010000:00010000:1.0:1789760737.030770:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760737.030775:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2e2fc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24240128->24244223] (req 24240128->24244223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.030785:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2e2fc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24240128->24244223] (req 24240128->24244223) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1789760737.031411:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00010000:1.0:1789760737.072743:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24240128 -> 24244224 00000001:00080000:3.0:1789760737.072747:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973237 is committed 00010000:00010000:1.0:1789760737.072762:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2e2fc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24240128->24244223] (req 24240128->24244223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.072779:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2e2fc lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24240128->24244223] (req 24240128->24244223) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.072788:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2e2fc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24240128->24244223] (req 24240128->24244223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.072795:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760737.072799:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2e2fc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24240128->24244223] (req 24240128->24244223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.072809:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2e2fc lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24240128->24244223] (req 24240128->24244223) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.072828:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2e2fc lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24240128->24244223] (req 24240128->24244223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.072835:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2e2fc lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24240128->24244223] (req 24240128->24244223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760737.072922:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac28a2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512617728:12345-192.168.203.56@tcp:4:dd.0 Request processed in 42253us (42611us total) trans 4294973237 rc 0/0 00000100:00100000:1.0:1789760737.072932:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5948 00000100:00100000:1.0:1789760737.072944:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 716555136363 time 716594952358 state 0x2 ref 0 00000100:00100000:0.0:1789760737.077389:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760737.078808:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512617856 00000100:00100000:2.0:1789760737.078819:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512617856 00000100:00100000:3.0:1789760737.079541:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 716555136363 time 716594952358 state 0x0 ref 1 00000100:00100000:3.0:1789760737.079558:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 716595212774 time 716594952358 state 0x1 ref 1 00000100:00100000:3.0:1789760737.079596:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 716595212774 time 716643894986 state 0x0 ref 1 00000100:00100000:3.0:1789760737.079610:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac287a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512617856:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760737.079660:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24244224, mtime: 0x6aad94e0, atime: 0x0, ctime: 0x6aad94e0, blocks: 0xc007 00010000:00010000:3.0:1789760737.079668:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2e303 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.079690:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=716s v=(0 0) 00010000:00010000:3.0:1789760737.079716:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760737.079723:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2e303 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24244224->24248319] (req 24244224->24248319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.079737:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2e303 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24244224->24248319] (req 24244224->24248319) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760737.138764:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24244224 -> 24248320 00010000:00010000:3.0:1789760737.138784:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2e303 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24244224->24248319] (req 24244224->24248319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.138802:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2e303 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24244224->24248319] (req 24244224->24248319) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.138812:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2e303 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24244224->24248319] (req 24244224->24248319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.138820:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760737.138825:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2e303 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24244224->24248319] (req 24244224->24248319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.138834:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2e303 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24244224->24248319] (req 24244224->24248319) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.138853:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2e303 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24244224->24248319] (req 24244224->24248319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.138862:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2e303 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24244224->24248319] (req 24244224->24248319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760737.138946:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac287a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2358:x1876699512617856:12345-192.168.203.56@tcp:4:dd.0 Request processed in 59336us (61572us total) trans 4294973238 rc 0/0 00000100:00100000:3.0:1789760737.138957:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5949 00000100:00100000:3.0:1789760737.138967:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 716595212774 time 716643894986 state 0x2 ref 0 00000001:00080000:1.0:1789760737.139058:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973238 is committed 00000100:00100000:0.0:1789760737.148505:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760737.148887:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512617984 00000100:00100000:3.0:1789760737.148897:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512617984 00000100:00100000:3.0:1789760737.148950:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 716595212774 time 716643894986 state 0x0 ref 1 00000100:00100000:3.0:1789760737.148965:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 716644155402 time 716643894986 state 0x1 ref 1 00000100:00100000:3.0:1789760737.149022:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 716644155402 time 716713321795 state 0x0 ref 1 00000100:00100000:3.0:1789760737.149034:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febea651500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512617984:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760737.149146:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24248320, mtime: 0x6aad94e0, atime: 0x0, ctime: 0x6aad94e0, blocks: 0xc007 00010000:00010000:3.0:1789760737.149155:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2e30a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.149170:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=716s v=(0 0) 00010000:00010000:3.0:1789760737.149175:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760737.149181:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2e30a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24248320->24252415] (req 24248320->24252415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.149199:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2e30a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24248320->24252415] (req 24248320->24252415) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760737.206997:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973239 is committed 00002000:00010000:1.0:1789760737.207104:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24248320 -> 24252416 00010000:00010000:1.0:1789760737.207120:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2e30a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24248320->24252415] (req 24248320->24252415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.207133:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2e30a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24248320->24252415] (req 24248320->24252415) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.207141:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2e30a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24248320->24252415] (req 24248320->24252415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.207148:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760737.207152:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2e30a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24248320->24252415] (req 24248320->24252415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.207161:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2e30a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24248320->24252415] (req 24248320->24252415) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.207179:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2e30a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24248320->24252415] (req 24248320->24252415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.207186:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2e30a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24248320->24252415] (req 24248320->24252415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760737.207279:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febea651500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512617984:12345-192.168.203.56@tcp:4:dd.0 Request processed in 58243us (58780us total) trans 4294973239 rc 0/0 00000100:00100000:1.0:1789760737.207289:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5950 00000100:00100000:1.0:1789760737.207298:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 716644155402 time 716713321795 state 0x2 ref 0 00000100:00100000:0.0:1789760737.209546:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760737.209625:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512618112 00000100:00100000:1.0:1789760737.209635:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512618112 00000100:00100000:1.0:1789760737.209682:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 716644155402 time 716713321795 state 0x0 ref 1 00000100:00100000:1.0:1789760737.209726:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 716713582211 time 716713321795 state 0x1 ref 1 00000100:00100000:1.0:1789760737.209756:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 716713582211 time 716774055113 state 0x0 ref 1 00000100:00100000:1.0:1789760737.209776:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febea650380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512618112:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760737.209825:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24252416, mtime: 0x6aad94e0, atime: 0x0, ctime: 0x6aad94e0, blocks: 0xc007 00010000:00010000:1.0:1789760737.209833:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2e311 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.209847:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=716s v=(0 0) 00010000:00010000:1.0:1789760737.209851:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760737.209857:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2e311 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24252416->24256511] (req 24252416->24256511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.209871:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2e311 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24252416->24256511] (req 24252416->24256511) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760737.236973:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24252416 -> 24256512 00010000:00010000:2.0:1789760737.237621:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2e311 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24252416->24256511] (req 24252416->24256511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.237634:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2e311 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24252416->24256511] (req 24252416->24256511) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.237643:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2e311 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24252416->24256511] (req 24252416->24256511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.237655:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760737.237660:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2e311 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24252416->24256511] (req 24252416->24256511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.237668:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2e311 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24252416->24256511] (req 24252416->24256511) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.237714:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2e311 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24252416->24256511] (req 24252416->24256511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.237724:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2e311 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24252416->24256511] (req 24252416->24256511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760737.237829:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febea650380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2361:x1876699512618112:12345-192.168.203.56@tcp:4:dd.0 Request processed in 28049us (28287us total) trans 4294973240 rc 0/0 00000100:00100000:2.0:1789760737.237839:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5951 00000100:00100000:2.0:1789760737.237849:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 716713582211 time 716774055113 state 0x2 ref 0 00000001:00080000:0.0:1789760737.242286:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973240 is committed 00000100:00100000:3.0:1789760737.244923:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760737.245215:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512618240 00000100:00100000:0.0:1789760737.245226:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512618240 00000100:00100000:0.0:1789760737.245284:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 716713582211 time 716774055113 state 0x0 ref 1 00000100:00100000:0.0:1789760737.245299:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 716774315529 time 716774055113 state 0x1 ref 1 00000100:00100000:0.0:1789760737.245330:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 716774315529 time 716809630141 state 0x0 ref 1 00000100:00100000:0.0:1789760737.245343:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac28a1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512618240:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760737.245402:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24256512, mtime: 0x6aad94e0, atime: 0x0, ctime: 0x6aad94e0, blocks: 0xc007 00010000:00010000:0.0:1789760737.245412:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2e318 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.245428:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=716s v=(0 0) 00010000:00010000:0.0:1789760737.245432:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760737.245438:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2e318 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24256512->24260607] (req 24256512->24260607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.245452:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2e318 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24256512->24260607] (req 24256512->24260607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760737.285449:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973241 is committed 00002000:00010000:3.0:1789760737.285561:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24256512 -> 24260608 00010000:00010000:3.0:1789760737.285578:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2e318 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24256512->24260607] (req 24256512->24260607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.285591:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2e318 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24256512->24260607] (req 24256512->24260607) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.285599:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2e318 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24256512->24260607] (req 24256512->24260607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.285606:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760737.285612:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2e318 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24256512->24260607] (req 24256512->24260607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.285621:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2e318 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24256512->24260607] (req 24256512->24260607) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.285639:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2e318 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24256512->24260607] (req 24256512->24260607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.285646:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2e318 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24256512->24260607] (req 24256512->24260607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760737.285764:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac28a1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512618240:12345-192.168.203.56@tcp:4:dd.0 Request processed in 40419us (40846us total) trans 4294973241 rc 0/0 00000100:00100000:3.0:1789760737.285775:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5952 00000100:00100000:3.0:1789760737.285784:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 716774315529 time 716809630141 state 0x2 ref 0 00000100:00100000:3.0:1789760737.288404:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760737.288643:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512618368 00000100:00100000:0.0:1789760737.288653:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512618368 00000100:00100000:0.0:1789760737.288742:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 716774315529 time 716809630141 state 0x0 ref 1 00000100:00100000:0.0:1789760737.288756:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 716809890557 time 716809630141 state 0x1 ref 1 00000100:00100000:0.0:1789760737.288782:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 716809890557 time 716853082336 state 0x0 ref 1 00000100:00100000:0.0:1789760737.288794:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac28a3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512618368:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760737.288844:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24260608, mtime: 0x6aad94e0, atime: 0x0, ctime: 0x6aad94e0, blocks: 0xc007 00010000:00010000:0.0:1789760737.288853:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2e31f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.288869:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=716s v=(0 0) 00010000:00010000:0.0:1789760737.288873:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760737.288879:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2e31f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24260608->24264703] (req 24260608->24264703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.288893:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2e31f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24260608->24264703] (req 24260608->24264703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760737.318960:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973242 is committed 00002000:00010000:2.0:1789760737.319171:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24260608 -> 24264704 00010000:00010000:2.0:1789760737.319187:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2e31f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24260608->24264703] (req 24260608->24264703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.319199:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2e31f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24260608->24264703] (req 24260608->24264703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.319212:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2e31f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24260608->24264703] (req 24260608->24264703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.319219:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760737.319224:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2e31f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24260608->24264703] (req 24260608->24264703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.319233:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2e31f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24260608->24264703] (req 24260608->24264703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.319252:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2e31f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24260608->24264703] (req 24260608->24264703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.319259:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2e31f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24260608->24264703] (req 24260608->24264703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760737.319349:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac28a3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512618368:12345-192.168.203.56@tcp:4:dd.0 Request processed in 30552us (30948us total) trans 4294973242 rc 0/0 00000100:00100000:2.0:1789760737.319450:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5953 00000100:00100000:2.0:1789760737.319460:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 716809890557 time 716853082336 state 0x2 ref 0 00000100:00100000:0.0:1789760737.322970:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760737.323104:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512618496 00000100:00100000:2.0:1789760737.323113:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512618496 00000100:00100000:2.0:1789760737.323160:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 716809890557 time 716853082336 state 0x0 ref 1 00000100:00100000:2.0:1789760737.323174:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 716853342752 time 716853082336 state 0x1 ref 1 00000100:00100000:2.0:1789760737.323237:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 716853342752 time 716887536178 state 0x0 ref 1 00000100:00100000:2.0:1789760737.323250:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febea4b7050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512618496:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760737.323300:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24264704, mtime: 0x6aad94e0, atime: 0x0, ctime: 0x6aad94e0, blocks: 0xc007 00010000:00010000:2.0:1789760737.323309:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2e326 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.323325:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=716s v=(0 0) 00010000:00010000:2.0:1789760737.323329:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760737.323335:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2e326 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24264704->24268799] (req 24264704->24268799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.323349:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2e326 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24264704->24268799] (req 24264704->24268799) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760737.355605:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973243 is committed 00002000:00010000:2.0:1789760737.355692:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24264704 -> 24268800 00010000:00010000:2.0:1789760737.355841:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2e326 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24264704->24268799] (req 24264704->24268799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.355854:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2e326 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24264704->24268799] (req 24264704->24268799) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.355863:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2e326 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24264704->24268799] (req 24264704->24268799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.355870:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760737.355875:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2e326 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24264704->24268799] (req 24264704->24268799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.355883:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2e326 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24264704->24268799] (req 24264704->24268799) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.355902:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2e326 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24264704->24268799] (req 24264704->24268799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.355910:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2e326 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24264704->24268799] (req 24264704->24268799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760737.356034:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febea4b7050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512618496:12345-192.168.203.56@tcp:4:dd.0 Request processed in 32781us (33067us total) trans 4294973243 rc 0/0 00000100:00100000:2.0:1789760737.356044:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5954 00000100:00100000:2.0:1789760737.356054:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 716853342752 time 716887536178 state 0x2 ref 0 00000100:00100000:3.0:1789760737.360093:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760737.360157:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512618624 00000100:00100000:1.0:1789760737.360166:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512618624 00000100:00100000:1.0:1789760737.360223:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 716853342752 time 716887536178 state 0x0 ref 1 00000100:00100000:1.0:1789760737.360243:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 716887796594 time 716887536178 state 0x1 ref 1 00000100:00100000:1.0:1789760737.360272:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 716887796594 time 716924572256 state 0x0 ref 1 00000100:00100000:1.0:1789760737.360284:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac28a2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512618624:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760737.360331:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24268800, mtime: 0x6aad94e0, atime: 0x0, ctime: 0x6aad94e0, blocks: 0xc007 00010000:00010000:1.0:1789760737.360344:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2e32d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.360379:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=716s v=(0 0) 00010000:00010000:1.0:1789760737.360385:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760737.360391:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2e32d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24268800->24272895] (req 24268800->24272895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.360406:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2e32d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24268800->24272895] (req 24268800->24272895) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760737.389375:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973244 is committed 00002000:00010000:1.0:1789760737.389507:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24268800 -> 24272896 00010000:00010000:1.0:1789760737.389522:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2e32d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24268800->24272895] (req 24268800->24272895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.389535:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2e32d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24268800->24272895] (req 24268800->24272895) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.389544:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2e32d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24268800->24272895] (req 24268800->24272895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.389556:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760737.389562:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2e32d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24268800->24272895] (req 24268800->24272895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.389571:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2e32d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24268800->24272895] (req 24268800->24272895) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.389592:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2e32d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24268800->24272895] (req 24268800->24272895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.389600:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2e32d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24268800->24272895] (req 24268800->24272895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760737.389760:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac28a2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512618624:12345-192.168.203.56@tcp:4:dd.0 Request processed in 29473us (29722us total) trans 4294973244 rc 0/0 00000100:00100000:1.0:1789760737.389770:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5955 00000100:00100000:1.0:1789760737.389779:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 716887796594 time 716924572256 state 0x2 ref 0 00000100:00100000:0.0:1789760737.393074:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760737.393154:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512618752 00000100:00100000:2.0:1789760737.393162:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512618752 00000100:00100000:2.0:1789760737.393211:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 716887796594 time 716924572256 state 0x0 ref 1 00000100:00100000:2.0:1789760737.393225:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 716924832672 time 716924572256 state 0x1 ref 1 00000100:00100000:2.0:1789760737.393252:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 716924832672 time 716957552197 state 0x0 ref 1 00000100:00100000:2.0:1789760737.393264:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febea651500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512618752:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760737.393313:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24272896, mtime: 0x6aad94e0, atime: 0x0, ctime: 0x6aad94e0, blocks: 0xc007 00010000:00010000:2.0:1789760737.393322:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2e334 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.393338:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=716s v=(0 0) 00010000:00010000:2.0:1789760737.393343:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760737.393349:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2e334 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24272896->24276991] (req 24272896->24276991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.393362:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2e334 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24272896->24276991] (req 24272896->24276991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760737.433900:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973245 is committed 00002000:00010000:2.0:1789760737.433998:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24272896 -> 24276992 00010000:00010000:2.0:1789760737.434014:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2e334 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24272896->24276991] (req 24272896->24276991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.434026:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2e334 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24272896->24276991] (req 24272896->24276991) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.434089:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2e334 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24272896->24276991] (req 24272896->24276991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.434098:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760737.434103:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2e334 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24272896->24276991] (req 24272896->24276991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.434110:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2e334 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24272896->24276991] (req 24272896->24276991) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.434132:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2e334 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24272896->24276991] (req 24272896->24276991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.434139:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2e334 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24272896->24276991] (req 24272896->24276991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760737.434252:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febea651500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512618752:12345-192.168.203.56@tcp:4:dd.0 Request processed in 40985us (41182us total) trans 4294973245 rc 0/0 00000100:00100000:2.0:1789760737.434261:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5956 00000100:00100000:2.0:1789760737.434270:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 716924832672 time 716957552197 state 0x2 ref 0 00000100:00100000:3.0:1789760737.436935:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760737.437034:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512618880 00000100:00100000:2.0:1789760737.437051:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512618880 00000100:00100000:2.0:1789760737.437104:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 716924832672 time 716957552197 state 0x0 ref 1 00000100:00100000:2.0:1789760737.437119:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 716957812613 time 716957552197 state 0x1 ref 1 00000100:00100000:2.0:1789760737.437180:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 716957812613 time 717001479786 state 0x0 ref 1 00000100:00100000:2.0:1789760737.437194:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac28a1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512618880:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760737.437239:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24276992, mtime: 0x6aad94e0, atime: 0x0, ctime: 0x6aad94e0, blocks: 0xc007 00010000:00010000:2.0:1789760737.437248:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2e33b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.437262:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=717s v=(0 0) 00010000:00010000:2.0:1789760737.437267:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760737.437274:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2e33b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24276992->24281087] (req 24276992->24281087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.437289:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2e33b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24276992->24281087] (req 24276992->24281087) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760737.462694:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973246 is committed 00002000:00010000:3.0:1789760737.462948:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24276992 -> 24281088 00010000:00010000:3.0:1789760737.462964:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2e33b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24276992->24281087] (req 24276992->24281087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.462978:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2e33b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24276992->24281087] (req 24276992->24281087) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.462987:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2e33b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24276992->24281087] (req 24276992->24281087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.462994:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760737.462999:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2e33b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24276992->24281087] (req 24276992->24281087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.463008:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2e33b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24276992->24281087] (req 24276992->24281087) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.463035:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2e33b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24276992->24281087] (req 24276992->24281087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.463053:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2e33b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24276992->24281087] (req 24276992->24281087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760737.463186:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac28a1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512618880:12345-192.168.203.56@tcp:4:dd.0 Request processed in 25990us (26254us total) trans 4294973246 rc 0/0 00000100:00100000:3.0:1789760737.463196:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5957 00000100:00100000:3.0:1789760737.463206:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 716957812613 time 717001479786 state 0x2 ref 0 00000100:00100000:0.0:1789760737.466605:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760737.466872:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512619008 00000100:00100000:1.0:1789760737.466882:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512619008 00000100:00100000:1.0:1789760737.466938:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 716957812613 time 717001479786 state 0x0 ref 1 00000100:00100000:1.0:1789760737.466955:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 717001740202 time 717001479786 state 0x1 ref 1 00000100:00100000:1.0:1789760737.467016:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 717001740202 time 717031315207 state 0x0 ref 1 00000100:00100000:1.0:1789760737.467033:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febea650380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512619008:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760737.467091:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24281088, mtime: 0x6aad94e0, atime: 0x0, ctime: 0x6aad94e0, blocks: 0xc007 00010000:00010000:1.0:1789760737.467100:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2e342 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.467115:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=717s v=(0 0) 00010000:00010000:1.0:1789760737.467119:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760737.467124:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2e342 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24281088->24285183] (req 24281088->24285183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.467137:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2e342 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24281088->24285183] (req 24281088->24285183) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760737.502532:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973247 is committed 00002000:00010000:2.0:1789760737.502641:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24281088 -> 24285184 00010000:00010000:2.0:1789760737.502656:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2e342 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24281088->24285183] (req 24281088->24285183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.502670:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2e342 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24281088->24285183] (req 24281088->24285183) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.502679:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2e342 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24281088->24285183] (req 24281088->24285183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.502686:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760737.502691:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2e342 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24281088->24285183] (req 24281088->24285183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.502732:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2e342 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24281088->24285183] (req 24281088->24285183) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.502752:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2e342 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24281088->24285183] (req 24281088->24285183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.502873:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2e342 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24281088->24285183] (req 24281088->24285183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760737.503022:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febea650380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512619008:12345-192.168.203.56@tcp:4:dd.0 Request processed in 35986us (36420us total) trans 4294973247 rc 0/0 00000100:00100000:3.0:1789760737.503034:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5958 00000100:00100000:3.0:1789760737.505674:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 717001740202 time 717031315207 state 0x2 ref 0 00000100:00100000:0.0:1789760737.505909:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760737.505999:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512619136 00000100:00100000:2.0:1789760737.506008:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512619136 00000100:00100000:2.0:1789760737.506151:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 717001740202 time 717031315207 state 0x0 ref 1 00000100:00100000:2.0:1789760737.506166:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 717031575623 time 717031315207 state 0x1 ref 1 00000100:00100000:2.0:1789760737.506193:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 717031575623 time 717070493051 state 0x0 ref 1 00000100:00100000:2.0:1789760737.506206:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febea899180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512619136:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760737.506253:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24285184, mtime: 0x6aad94e0, atime: 0x0, ctime: 0x6aad94e0, blocks: 0xc007 00010000:00010000:2.0:1789760737.506262:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2e349 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.506277:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=717s v=(0 0) 00010000:00010000:2.0:1789760737.506282:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760737.506289:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2e349 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24285184->24289279] (req 24285184->24289279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.506303:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2e349 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24285184->24289279] (req 24285184->24289279) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760737.533998:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973248 is committed 00002000:00010000:3.0:1789760737.534093:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24285184 -> 24289280 00002000:00010000:3.0:1789760737.534098:0:6710:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760736 -> 1789760737 00002000:00010000:3.0:1789760737.534100:0:6710:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760736 -> 1789760737 00010000:00010000:3.0:1789760737.534113:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2e349 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24285184->24289279] (req 24285184->24289279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.534126:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2e349 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24285184->24289279] (req 24285184->24289279) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.534142:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2e349 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24285184->24289279] (req 24285184->24289279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.534149:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760737.534154:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2e349 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24285184->24289279] (req 24285184->24289279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.534164:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2e349 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24285184->24289279] (req 24285184->24289279) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.534183:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2e349 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24285184->24289279] (req 24285184->24289279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.534191:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2e349 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24285184->24289279] (req 24285184->24289279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760737.534279:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febea899180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512619136:12345-192.168.203.56@tcp:4:dd.0 Request processed in 28071us (28366us total) trans 4294973248 rc 0/0 00000100:00100000:3.0:1789760737.534289:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5959 00000100:00100000:3.0:1789760737.534299:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 717031575623 time 717070493051 state 0x2 ref 0 00000100:00100000:0.0:1789760737.537477:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760737.537559:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512619264 00000100:00100000:3.0:1789760737.537569:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512619264 00000100:00100000:3.0:1789760737.537618:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 717031575623 time 717070493051 state 0x0 ref 1 00000100:00100000:3.0:1789760737.537634:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 717070753467 time 717070493051 state 0x1 ref 1 00000100:00100000:3.0:1789760737.537664:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 717070753467 time 717101963779 state 0x0 ref 1 00000100:00100000:3.0:1789760737.537676:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febea89b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512619264:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760737.537746:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24289280, mtime: 0x6aad94e1, atime: 0x0, ctime: 0x6aad94e1, blocks: 0xc007 00010000:00010000:3.0:1789760737.537755:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2e350 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.537780:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=717s v=(0 0) 00010000:00010000:3.0:1789760737.537785:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760737.537791:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2e350 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24289280->24293375] (req 24289280->24293375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.537807:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2e350 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24289280->24293375] (req 24289280->24293375) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760737.570827:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24289280 -> 24293376 00010000:00010000:2.0:1789760737.570848:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2e350 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24289280->24293375] (req 24289280->24293375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.570861:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2e350 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24289280->24293375] (req 24289280->24293375) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.570871:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2e350 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24289280->24293375] (req 24289280->24293375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.570878:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760737.570883:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2e350 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24289280->24293375] (req 24289280->24293375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.570892:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2e350 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24289280->24293375] (req 24289280->24293375) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.570909:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2e350 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24289280->24293375] (req 24289280->24293375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.570917:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2e350 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24289280->24293375] (req 24289280->24293375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760737.571001:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febea89b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2358:x1876699512619264:12345-192.168.203.56@tcp:4:dd.0 Request processed in 33322us (33529us total) trans 4294973249 rc 0/0 00000100:00100000:2.0:1789760737.571012:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5960 00000100:00100000:2.0:1789760737.571022:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 717070753467 time 717101963779 state 0x2 ref 0 00000100:00100000:3.0:1789760737.573636:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760737.573734:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512619392 00000100:00100000:2.0:1789760737.573744:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512619392 00000100:00100000:2.0:1789760737.573807:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 717070753467 time 717101963779 state 0x0 ref 1 00000100:00100000:2.0:1789760737.573823:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 717102224195 time 717101963779 state 0x1 ref 1 00000100:00100000:2.0:1789760737.573844:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 717102224195 time 717138144128 state 0x0 ref 1 00000100:00100000:2.0:1789760737.573856:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac28a3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2359:x1876699512619392:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760737.573901:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24293376, mtime: 0x6aad94e1, atime: 0x0, ctime: 0x6aad94e1, blocks: 0xc007 00010000:00010000:2.0:1789760737.573909:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2e357 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.573924:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=717s v=(0 0) 00010000:00010000:2.0:1789760737.573928:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760737.573934:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2e357 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24293376->24297471] (req 24293376->24297471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.573946:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2e357 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24293376->24297471] (req 24293376->24297471) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760737.575352:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973249 is committed 00000001:00080000:3.0:1789760737.610305:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973250 is committed 00002000:00010000:2.0:1789760737.610412:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24293376 -> 24297472 00010000:00010000:2.0:1789760737.610427:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2e357 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24293376->24297471] (req 24293376->24297471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.610440:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2e357 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24293376->24297471] (req 24293376->24297471) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.610450:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2e357 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24293376->24297471] (req 24293376->24297471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.610459:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760737.610464:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2e357 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24293376->24297471] (req 24293376->24297471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.610473:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2e357 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24293376->24297471] (req 24293376->24297471) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.610491:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2e357 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24293376->24297471] (req 24293376->24297471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.610498:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2e357 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24293376->24297471] (req 24293376->24297471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760737.610586:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac28a3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512619392:12345-192.168.203.56@tcp:4:dd.0 Request processed in 36727us (36954us total) trans 4294973250 rc 0/0 00000100:00100000:2.0:1789760737.610595:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5961 00000100:00100000:2.0:1789760737.610605:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 717102224195 time 717138144128 state 0x2 ref 0 00000100:00100000:3.0:1789760737.614870:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760737.615294:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512619520 00000100:00100000:2.0:1789760737.615304:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512619520 00000100:00100000:2.0:1789760737.615354:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 717102224195 time 717138144128 state 0x0 ref 1 00000100:00100000:2.0:1789760737.615442:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 717138404544 time 717138144128 state 0x1 ref 1 00000100:00100000:2.0:1789760737.615470:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 717138404544 time 717179769877 state 0x0 ref 1 00000100:00100000:2.0:1789760737.615483:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febc7b33480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512619520:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760737.615533:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24297472, mtime: 0x6aad94e1, atime: 0x0, ctime: 0x6aad94e1, blocks: 0xc007 00010000:00010000:2.0:1789760737.615542:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2e35e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.615558:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=717s v=(0 0) 00010000:00010000:2.0:1789760737.615563:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760737.615569:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2e35e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24297472->24301567] (req 24297472->24301567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.615583:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2e35e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24297472->24301567] (req 24297472->24301567) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760737.649287:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973251 is committed 00002000:00010000:3.0:1789760737.649555:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24297472 -> 24301568 00010000:00010000:3.0:1789760737.649572:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2e35e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24297472->24301567] (req 24297472->24301567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.649584:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2e35e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24297472->24301567] (req 24297472->24301567) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.649593:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2e35e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24297472->24301567] (req 24297472->24301567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.649599:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760737.649605:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2e35e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24297472->24301567] (req 24297472->24301567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.649613:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2e35e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24297472->24301567] (req 24297472->24301567) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.649633:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2e35e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24297472->24301567] (req 24297472->24301567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.649640:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2e35e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24297472->24301567] (req 24297472->24301567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760737.650001:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febc7b33480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512619520:12345-192.168.203.56@tcp:4:dd.0 Request processed in 34515us (35136us total) trans 4294973251 rc 0/0 00000100:00100000:3.0:1789760737.650012:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5962 00000100:00100000:3.0:1789760737.650021:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 717138404544 time 717179769877 state 0x2 ref 0 00000100:00100000:3.0:1789760737.654856:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760737.655077:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512619648 00000100:00100000:2.0:1789760737.655087:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512619648 00000100:00100000:2.0:1789760737.655138:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 717138404544 time 717179769877 state 0x0 ref 1 00000100:00100000:2.0:1789760737.655153:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 717180030293 time 717179769877 state 0x1 ref 1 00000100:00100000:2.0:1789760737.655176:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 717180030293 time 717219476546 state 0x0 ref 1 00000100:00100000:2.0:1789760737.655190:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febc7b32680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512619648:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760737.655237:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24301568, mtime: 0x6aad94e1, atime: 0x0, ctime: 0x6aad94e1, blocks: 0xc007 00010000:00010000:2.0:1789760737.655246:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2e365 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.655262:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=717s v=(0 0) 00010000:00010000:2.0:1789760737.655266:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760737.655272:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2e365 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24301568->24305663] (req 24301568->24305663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.655286:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2e365 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24301568->24305663] (req 24301568->24305663) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.660257:0:7647:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:3.0:1789760737.706940:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24301568 -> 24305664 00010000:00010000:3.0:1789760737.706959:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2e365 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24301568->24305663] (req 24301568->24305663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.706972:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2e365 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24301568->24305663] (req 24301568->24305663) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.706981:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2e365 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24301568->24305663] (req 24301568->24305663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.706988:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760737.706992:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2e365 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24301568->24305663] (req 24301568->24305663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.707000:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2e365 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24301568->24305663] (req 24301568->24305663) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.707018:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2e365 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24301568->24305663] (req 24301568->24305663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.707025:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2e365 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24301568->24305663] (req 24301568->24305663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760737.707204:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febc7b32680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2360:x1876699512619648:12345-192.168.203.56@tcp:4:dd.0 Request processed in 52012us (52352us total) trans 4294973252 rc 0/0 00000100:00100000:3.0:1789760737.707216:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5963 00000100:00100000:3.0:1789760737.707225:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 717180030293 time 717219476546 state 0x2 ref 0 00000001:00080000:0.0:1789760737.707245:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973252 is committed 00000100:00100000:0.0:1789760737.711583:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760737.711915:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512619776 00000100:00100000:2.0:1789760737.711925:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512619776 00000100:00100000:2.0:1789760737.711979:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 717180030293 time 717219476546 state 0x0 ref 1 00000100:00100000:2.0:1789760737.711994:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 717219736962 time 717219476546 state 0x1 ref 1 00000100:00100000:2.0:1789760737.712051:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 717219736962 time 717276350883 state 0x0 ref 1 00000100:00100000:2.0:1789760737.712063:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febea89b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512619776:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760737.712110:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24305664, mtime: 0x6aad94e1, atime: 0x0, ctime: 0x6aad94e1, blocks: 0xc007 00010000:00010000:2.0:1789760737.712118:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2e36c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.712132:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=717s v=(0 0) 00010000:00010000:2.0:1789760737.712137:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760737.712144:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2e36c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24305664->24309759] (req 24305664->24309759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.712158:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2e36c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24305664->24309759] (req 24305664->24309759) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760737.737323:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24305664 -> 24309760 00010000:00010000:2.0:1789760737.737341:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2e36c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24305664->24309759] (req 24305664->24309759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.737353:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2e36c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24305664->24309759] (req 24305664->24309759) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.737372:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2e36c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24305664->24309759] (req 24305664->24309759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.737380:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760737.737385:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2e36c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24305664->24309759] (req 24305664->24309759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.737393:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2e36c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24305664->24309759] (req 24305664->24309759) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.737411:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2e36c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24305664->24309759] (req 24305664->24309759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.737420:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2e36c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24305664->24309759] (req 24305664->24309759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760737.737510:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febea89b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2359:x1876699512619776:12345-192.168.203.56@tcp:4:dd.0 Request processed in 25444us (25931us total) trans 4294973253 rc 0/0 00000100:00100000:2.0:1789760737.737520:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5964 00000100:00100000:2.0:1789760737.737530:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 717219736962 time 717276350883 state 0x2 ref 0 00000001:00080000:0.0:1789760737.737646:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973253 is committed 00000100:00100000:1.0:1789760737.747404:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760737.749291:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512619904 00000100:00100000:3.0:1789760737.749304:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512619904 00000100:00100000:3.0:1789760737.749361:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 717219736962 time 717276350883 state 0x0 ref 1 00000100:00100000:3.0:1789760737.749443:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 717276611299 time 717276350883 state 0x1 ref 1 00000100:00100000:3.0:1789760737.749474:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 717276611299 time 717313773698 state 0x0 ref 1 00000100:00100000:3.0:1789760737.749487:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2fbb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512619904:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760737.749530:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24309760, mtime: 0x6aad94e1, atime: 0x0, ctime: 0x6aad94e1, blocks: 0xc007 00010000:00010000:3.0:1789760737.749539:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2e373 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.749555:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=717s v=(0 0) 00010000:00010000:3.0:1789760737.749560:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760737.749566:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2e373 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24309760->24313855] (req 24309760->24313855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.749579:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2e373 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24309760->24313855] (req 24309760->24313855) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760737.793733:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973254 is committed 00002000:00010000:0.0:1789760737.793802:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24309760 -> 24313856 00010000:00010000:0.0:1789760737.793817:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2e373 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24309760->24313855] (req 24309760->24313855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.793831:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2e373 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24309760->24313855] (req 24309760->24313855) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.793841:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2e373 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24309760->24313855] (req 24309760->24313855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.793848:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760737.793853:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2e373 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24309760->24313855] (req 24309760->24313855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.793862:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2e373 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24309760->24313855] (req 24309760->24313855) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.793882:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2e373 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24309760->24313855] (req 24309760->24313855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.793896:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2e373 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24309760->24313855] (req 24309760->24313855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760737.793990:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2fbb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512619904:12345-192.168.203.56@tcp:4:dd.0 Request processed in 44501us (46591us total) trans 4294973254 rc 0/0 00000100:00100000:0.0:1789760737.794000:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5965 00000100:00100000:0.0:1789760737.794010:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 717276611299 time 717313773698 state 0x2 ref 0 00000100:00100000:3.0:1789760737.799319:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760737.801294:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512620032 00000100:00100000:2.0:1789760737.801311:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512620032 00000100:00100000:2.0:1789760737.804523:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 717276611299 time 717313773698 state 0x0 ref 1 00000100:00100000:2.0:1789760737.804549:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 717314034114 time 717313773698 state 0x1 ref 1 00000100:00100000:3.0:1789760737.804832:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 717314034114 time 717369129336 state 0x0 ref 1 00000100:00100000:3.0:1789760737.804858:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2747800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512620032:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760737.804913:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24313856, mtime: 0x6aad94e1, atime: 0x0, ctime: 0x6aad94e1, blocks: 0xc007 00010000:00010000:3.0:1789760737.804922:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2e37a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.804937:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=717s v=(0 0) 00010000:00010000:3.0:1789760737.804941:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760737.804948:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2e37a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24313856->24317951] (req 24313856->24317951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.804961:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2e37a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24313856->24317951] (req 24313856->24317951) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760737.851420:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973255 is committed 00002000:00010000:1.0:1789760737.851591:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24313856 -> 24317952 00010000:00010000:1.0:1789760737.851607:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2e37a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24313856->24317951] (req 24313856->24317951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.851619:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2e37a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24313856->24317951] (req 24313856->24317951) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.851628:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2e37a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24313856->24317951] (req 24313856->24317951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.851634:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760737.851640:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2e37a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24313856->24317951] (req 24313856->24317951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.851649:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2e37a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24313856->24317951] (req 24313856->24317951) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.851664:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2e37a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24313856->24317951] (req 24313856->24317951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.851670:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2e37a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24313856->24317951] (req 24313856->24317951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760737.851899:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2747800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512620032:12345-192.168.203.56@tcp:4:dd.0 Request processed in 47050us (52584us total) trans 4294973255 rc 0/0 00000100:00100000:1.0:1789760737.851909:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5966 00000100:00100000:1.0:1789760737.851918:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 717314034114 time 717369129336 state 0x2 ref 0 00000100:00100000:3.0:1789760737.857265:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760737.857763:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512620160 00000100:00100000:2.0:1789760737.857774:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512620160 00000100:00100000:2.0:1789760737.857840:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 717314034114 time 717369129336 state 0x0 ref 1 00000100:00100000:2.0:1789760737.857854:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 717369389752 time 717369129336 state 0x1 ref 1 00000100:00100000:2.0:1789760737.857884:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 717369389752 time 717422183958 state 0x0 ref 1 00000100:00100000:2.0:1789760737.857895:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2745500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512620160:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760737.857940:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24317952, mtime: 0x6aad94e1, atime: 0x0, ctime: 0x6aad94e1, blocks: 0xc007 00010000:00010000:2.0:1789760737.857949:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2e381 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.857964:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=717s v=(0 0) 00010000:00010000:2.0:1789760737.857969:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760737.857974:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2e381 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24317952->24322047] (req 24317952->24322047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.857989:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2e381 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24317952->24322047] (req 24317952->24322047) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760737.884816:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973256 is committed 00002000:00010000:0.0:1789760737.885087:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24317952 -> 24322048 00010000:00010000:0.0:1789760737.885103:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2e381 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24317952->24322047] (req 24317952->24322047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.885116:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2e381 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24317952->24322047] (req 24317952->24322047) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.885125:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2e381 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24317952->24322047] (req 24317952->24322047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.885132:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760737.885137:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2e381 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24317952->24322047] (req 24317952->24322047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.885145:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2e381 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24317952->24322047] (req 24317952->24322047) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.885170:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2e381 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24317952->24322047] (req 24317952->24322047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.885178:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2e381 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24317952->24322047] (req 24317952->24322047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760737.885305:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2745500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512620160:12345-192.168.203.56@tcp:4:dd.0 Request processed in 27407us (28044us total) trans 4294973256 rc 0/0 00000100:00100000:0.0:1789760737.885316:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5967 00000100:00100000:0.0:1789760737.885325:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 717369389752 time 717422183958 state 0x2 ref 0 00000100:00100000:1.0:1789760737.894324:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760737.894565:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512620288 00000100:00100000:0.0:1789760737.894575:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512620288 00000100:00100000:0.0:1789760737.894649:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 717369389752 time 717422183958 state 0x0 ref 1 00000100:00100000:0.0:1789760737.894664:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 717422444374 time 717422183958 state 0x1 ref 1 00000100:00100000:0.0:1789760737.894729:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 717422444374 time 717459028436 state 0x0 ref 1 00000100:00100000:0.0:1789760737.894742:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2fb8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512620288:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760737.894802:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24322048, mtime: 0x6aad94e1, atime: 0x0, ctime: 0x6aad94e1, blocks: 0xc007 00010000:00010000:0.0:1789760737.895273:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2e388 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.895289:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=717s v=(0 0) 00010000:00010000:0.0:1789760737.895294:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760737.895300:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2e388 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24322048->24326143] (req 24322048->24326143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.895315:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2e388 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24322048->24326143] (req 24322048->24326143) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760737.947893:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973257 is committed 00002000:00010000:1.0:1789760737.947992:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24322048 -> 24326144 00010000:00010000:1.0:1789760737.948190:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2e388 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24322048->24326143] (req 24322048->24326143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.948204:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2e388 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24322048->24326143] (req 24322048->24326143) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.948214:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2e388 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24322048->24326143] (req 24322048->24326143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.948226:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760737.948231:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2e388 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24322048->24326143] (req 24322048->24326143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.948240:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2e388 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24322048->24326143] (req 24322048->24326143) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.948259:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2e388 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24322048->24326143] (req 24322048->24326143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.948267:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2e388 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24322048->24326143] (req 24322048->24326143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760737.948409:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2fb8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512620288:12345-192.168.203.56@tcp:4:dd.0 Request processed in 53664us (54088us total) trans 4294973257 rc 0/0 00000100:00100000:1.0:1789760737.948420:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5968 00000100:00100000:1.0:1789760737.948430:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 717422444374 time 717459028436 state 0x2 ref 0 00000100:00100000:3.0:1789760737.951319:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760737.953525:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512620416 00000100:00100000:1.0:1789760737.953541:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512620416 00000100:00100000:2.0:1789760737.953848:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 717422444374 time 717459028436 state 0x0 ref 1 00000100:00100000:2.0:1789760737.953865:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 717459288852 time 717459028436 state 0x1 ref 1 00000100:00100000:2.0:1789760737.953897:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 717459288852 time 717518196383 state 0x0 ref 1 00000100:00100000:2.0:1789760737.953911:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2746a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512620416:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760737.953962:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24326144, mtime: 0x6aad94e1, atime: 0x0, ctime: 0x6aad94e1, blocks: 0xc007 00010000:00010000:2.0:1789760737.953972:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2e38f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.953987:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=717s v=(0 0) 00010000:00010000:2.0:1789760737.953992:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760737.953998:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2e38f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24326144->24330239] (req 24326144->24330239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.954025:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2e38f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24326144->24330239] (req 24326144->24330239) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760738.036295:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973258 is committed 00002000:00010000:1.0:1789760738.036480:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24326144 -> 24330240 00010000:00010000:1.0:1789760738.036496:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2e38f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24326144->24330239] (req 24326144->24330239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.036508:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2e38f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24326144->24330239] (req 24326144->24330239) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.036522:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2e38f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24326144->24330239] (req 24326144->24330239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.036529:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760738.036533:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2e38f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24326144->24330239] (req 24326144->24330239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.036542:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2e38f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24326144->24330239] (req 24326144->24330239) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.036569:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2e38f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24326144->24330239] (req 24326144->24330239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.036577:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2e38f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24326144->24330239] (req 24326144->24330239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760738.036681:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2746a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512620416:12345-192.168.203.56@tcp:4:dd.0 Request processed in 82768us (85365us total) trans 4294973258 rc 0/0 00000100:00100000:1.0:1789760738.036691:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5969 00000100:00100000:1.0:1789760738.036736:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 717459288852 time 717518196383 state 0x2 ref 0 00000100:00100000:0.0:1789760738.042494:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760738.051069:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512620544 00000100:00100000:2.0:1789760738.051084:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512620544 00000100:00100000:2.0:1789760738.051152:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 717459288852 time 717518196383 state 0x0 ref 1 00000100:00100000:2.0:1789760738.051168:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 717518456799 time 717518196383 state 0x1 ref 1 00000100:00100000:2.0:1789760738.051205:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 717518456799 time 717615504811 state 0x0 ref 1 00000100:00100000:2.0:1789760738.051220:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febea89aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512620544:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760738.051273:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24330240, mtime: 0x6aad94e1, atime: 0x0, ctime: 0x6aad94e1, blocks: 0xc007 00010000:00010000:2.0:1789760738.051283:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2e396 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.051299:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=717s v=(0 0) 00010000:00010000:2.0:1789760738.051303:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760738.051309:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2e396 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24330240->24334335] (req 24330240->24334335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.051323:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2e396 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24330240->24334335] (req 24330240->24334335) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760738.117740:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973259 is committed 00002000:00010000:1.0:1789760738.117896:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24330240 -> 24334336 00010000:00010000:1.0:1789760738.117911:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2e396 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24330240->24334335] (req 24330240->24334335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.117931:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2e396 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24330240->24334335] (req 24330240->24334335) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.117940:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2e396 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24330240->24334335] (req 24330240->24334335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.117946:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760738.117951:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2e396 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24330240->24334335] (req 24330240->24334335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.117960:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2e396 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24330240->24334335] (req 24330240->24334335) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.117981:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2e396 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24330240->24334335] (req 24330240->24334335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.117989:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2e396 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24330240->24334335] (req 24330240->24334335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760738.118229:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febea89aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512620544:12345-192.168.203.56@tcp:4:dd.0 Request processed in 67008us (75740us total) trans 4294973259 rc 0/0 00000100:00100000:1.0:1789760738.118237:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5970 00000100:00100000:1.0:1789760738.118245:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 717518456799 time 717615504811 state 0x2 ref 0 00000100:00100000:2.0:1789760738.125143:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760738.125657:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512620672 00000100:00100000:1.0:1789760738.125668:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512620672 00000100:00100000:1.0:1789760738.128514:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 717518456799 time 717615504811 state 0x0 ref 1 00000100:00100000:1.0:1789760738.128532:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 717615765227 time 717615504811 state 0x1 ref 1 00000100:00100000:1.0:1789760738.128753:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 717615765227 time 717692875564 state 0x0 ref 1 00000100:00100000:1.0:1789760738.128779:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febffb9f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512620672:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760738.128830:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24334336, mtime: 0x6aad94e1, atime: 0x0, ctime: 0x6aad94e1, blocks: 0xc007 00010000:00010000:1.0:1789760738.128839:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2e39d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.128854:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=717s v=(0 0) 00010000:00010000:1.0:1789760738.128859:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760738.128865:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2e39d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24334336->24338431] (req 24334336->24338431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.128877:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2e39d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24334336->24338431] (req 24334336->24338431) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760738.166747:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973260 is committed 00002000:00010000:1.0:1789760738.166875:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24334336 -> 24338432 00010000:00010000:1.0:1789760738.166892:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2e39d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24334336->24338431] (req 24334336->24338431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.166906:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2e39d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24334336->24338431] (req 24334336->24338431) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.166922:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2e39d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24334336->24338431] (req 24334336->24338431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.166930:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760738.166936:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2e39d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24334336->24338431] (req 24334336->24338431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.166945:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2e39d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24334336->24338431] (req 24334336->24338431) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.166965:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2e39d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24334336->24338431] (req 24334336->24338431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.166974:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2e39d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24334336->24338431] (req 24334336->24338431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760738.168288:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febffb9f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512620672:12345-192.168.203.56@tcp:4:dd.0 Request processed in 39513us (43145us total) trans 4294973260 rc 0/0 00000100:00100000:1.0:1789760738.168301:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5971 00000100:00100000:1.0:1789760738.168311:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 717615765227 time 717692875564 state 0x2 ref 0 00000100:00100000:2.0:1789760738.174937:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760738.175070:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512620800 00000100:00100000:1.0:1789760738.175081:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512620800 00000100:00100000:1.0:1789760738.175133:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 717615765227 time 717692875564 state 0x0 ref 1 00000100:00100000:1.0:1789760738.175148:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 717693135980 time 717692875564 state 0x1 ref 1 00000100:00100000:1.0:1789760738.175179:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 717693135980 time 717739479228 state 0x0 ref 1 00000100:00100000:1.0:1789760738.175191:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febffb9ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512620800:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760738.175240:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24338432, mtime: 0x6aad94e1, atime: 0x0, ctime: 0x6aad94e1, blocks: 0xc007 00010000:00010000:1.0:1789760738.175249:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2e3a4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.175265:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=717s v=(0 0) 00010000:00010000:1.0:1789760738.175269:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760738.175275:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2e3a4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24338432->24342527] (req 24338432->24342527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.175288:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2e3a4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24338432->24342527] (req 24338432->24342527) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760738.221394:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973261 is committed 00002000:00010000:1.0:1789760738.221542:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24338432 -> 24342528 00010000:00010000:1.0:1789760738.221560:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2e3a4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24338432->24342527] (req 24338432->24342527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.221573:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2e3a4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24338432->24342527] (req 24338432->24342527) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.221583:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2e3a4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24338432->24342527] (req 24338432->24342527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.221591:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760738.221597:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2e3a4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24338432->24342527] (req 24338432->24342527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.221607:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2e3a4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24338432->24342527] (req 24338432->24342527) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.221626:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2e3a4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24338432->24342527] (req 24338432->24342527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.221634:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2e3a4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24338432->24342527] (req 24338432->24342527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760738.221861:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febffb9ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512620800:12345-192.168.203.56@tcp:4:dd.0 Request processed in 46666us (46928us total) trans 4294973261 rc 0/0 00000100:00100000:1.0:1789760738.221872:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5972 00000100:00100000:1.0:1789760738.221882:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 717693135980 time 717739479228 state 0x2 ref 0 00000100:00100000:3.0:1789760738.225082:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760738.225157:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512620928 00000100:00100000:1.0:1789760738.225166:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512620928 00000100:00100000:1.0:1789760738.225213:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 717693135980 time 717739479228 state 0x0 ref 1 00000100:00100000:1.0:1789760738.225228:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 717739739644 time 717739479228 state 0x1 ref 1 00000100:00100000:1.0:1789760738.225257:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 717739739644 time 717789557132 state 0x0 ref 1 00000100:00100000:1.0:1789760738.225267:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2745880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512620928:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760738.225313:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24342528, mtime: 0x6aad94e1, atime: 0x0, ctime: 0x6aad94e1, blocks: 0xc007 00010000:00010000:1.0:1789760738.225322:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2e3ab lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.225339:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=717s v=(0 0) 00010000:00010000:1.0:1789760738.225343:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760738.225348:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2e3ab lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24342528->24346623] (req 24342528->24346623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.225372:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2e3ab lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24342528->24346623] (req 24342528->24346623) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760738.273630:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973262 is committed 00002000:00010000:1.0:1789760738.273933:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24342528 -> 24346624 00010000:00010000:1.0:1789760738.273959:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2e3ab lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24342528->24346623] (req 24342528->24346623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.273975:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2e3ab lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24342528->24346623] (req 24342528->24346623) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.273985:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2e3ab lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24342528->24346623] (req 24342528->24346623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.273993:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760738.273998:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2e3ab lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24342528->24346623] (req 24342528->24346623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.274008:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2e3ab lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24342528->24346623] (req 24342528->24346623) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.274042:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2e3ab lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24342528->24346623] (req 24342528->24346623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.274052:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2e3ab lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24342528->24346623] (req 24342528->24346623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760738.274253:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2745880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512620928:12345-192.168.203.56@tcp:4:dd.0 Request processed in 48981us (49177us total) trans 4294973262 rc 0/0 00000100:00100000:1.0:1789760738.274264:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5973 00000100:00100000:1.0:1789760738.274274:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 717739739644 time 717789557132 state 0x2 ref 0 00000100:00100000:2.0:1789760738.278084:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760738.278196:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512621056 00000100:00100000:1.0:1789760738.278206:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512621056 00000100:00100000:1.0:1789760738.278260:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 717739739644 time 717789557132 state 0x0 ref 1 00000100:00100000:1.0:1789760738.278277:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 717789817548 time 717789557132 state 0x1 ref 1 00000100:00100000:1.0:1789760738.278310:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 717789817548 time 717842609648 state 0x0 ref 1 00000100:00100000:1.0:1789760738.278321:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febffb9c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512621056:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760738.278382:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24346624, mtime: 0x6aad94e1, atime: 0x0, ctime: 0x6aad94e1, blocks: 0xc007 00010000:00010000:1.0:1789760738.278391:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2e3b2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.278406:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=717s v=(0 0) 00010000:00010000:1.0:1789760738.278411:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760738.278417:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2e3b2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24346624->24350719] (req 24346624->24350719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.278429:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2e3b2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24346624->24350719] (req 24346624->24350719) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760738.320836:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973263 is committed 00002000:00010000:0.0:1789760738.320995:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24346624 -> 24350720 00010000:00010000:0.0:1789760738.321011:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2e3b2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24346624->24350719] (req 24346624->24350719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.321023:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2e3b2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24346624->24350719] (req 24346624->24350719) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.321097:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2e3b2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24346624->24350719] (req 24346624->24350719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.321104:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760738.321109:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2e3b2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24346624->24350719] (req 24346624->24350719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.321117:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2e3b2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24346624->24350719] (req 24346624->24350719) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.321137:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2e3b2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24346624->24350719] (req 24346624->24350719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.321145:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2e3b2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24346624->24350719] (req 24346624->24350719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760738.321441:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febffb9c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512621056:12345-192.168.203.56@tcp:4:dd.0 Request processed in 43115us (43359us total) trans 4294973263 rc 0/0 00000100:00100000:0.0:1789760738.321452:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5974 00000100:00100000:0.0:1789760738.321461:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 717789817548 time 717842609648 state 0x2 ref 0 00000100:00100000:2.0:1789760738.324912:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760738.325026:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512621184 00000100:00100000:1.0:1789760738.325099:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512621184 00000100:00100000:1.0:1789760738.325156:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 717789817548 time 717842609648 state 0x0 ref 1 00000100:00100000:1.0:1789760738.325172:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 717842870064 time 717842609648 state 0x1 ref 1 00000100:00100000:1.0:1789760738.325243:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 717842870064 time 717889541236 state 0x0 ref 1 00000100:00100000:1.0:1789760738.325257:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febffb9c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512621184:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760738.325305:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24350720, mtime: 0x6aad94e1, atime: 0x0, ctime: 0x6aad94e1, blocks: 0xc007 00010000:00010000:1.0:1789760738.325314:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2e3b9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.325329:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=717s v=(0 0) 00010000:00010000:1.0:1789760738.325333:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760738.325339:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2e3b9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24350720->24354815] (req 24350720->24354815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.325351:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2e3b9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24350720->24354815] (req 24350720->24354815) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760738.366344:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973264 is committed 00002000:00010000:1.0:1789760738.366491:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24350720 -> 24354816 00010000:00010000:1.0:1789760738.366506:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2e3b9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24350720->24354815] (req 24350720->24354815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.366519:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2e3b9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24350720->24354815] (req 24350720->24354815) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.366528:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2e3b9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24350720->24354815] (req 24350720->24354815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.366534:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760738.366539:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2e3b9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24350720->24354815] (req 24350720->24354815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.366547:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2e3b9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24350720->24354815] (req 24350720->24354815) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.366566:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2e3b9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24350720->24354815] (req 24350720->24354815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.366574:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2e3b9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24350720->24354815] (req 24350720->24354815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760738.366665:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febffb9c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512621184:12345-192.168.203.56@tcp:4:dd.0 Request processed in 41405us (41757us total) trans 4294973264 rc 0/0 00000100:00100000:1.0:1789760738.366679:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5975 00000100:00100000:1.0:1789760738.366688:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 717842870064 time 717889541236 state 0x2 ref 0 00000100:00100000:2.0:1789760738.370606:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760738.370725:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512621312 00000100:00100000:1.0:1789760738.370735:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512621312 00000100:00100000:1.0:1789760738.370890:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 717842870064 time 717889541236 state 0x0 ref 1 00000100:00100000:1.0:1789760738.370906:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 717889801652 time 717889541236 state 0x1 ref 1 00000100:00100000:1.0:1789760738.370964:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 717889801652 time 717935263127 state 0x0 ref 1 00000100:00100000:1.0:1789760738.370977:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febffb9f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512621312:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760738.371023:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24354816, mtime: 0x6aad94e1, atime: 0x0, ctime: 0x6aad94e1, blocks: 0xc007 00010000:00010000:1.0:1789760738.371041:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2e3c0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.371058:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=717s v=(0 0) 00010000:00010000:1.0:1789760738.371063:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760738.371069:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2e3c0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24354816->24358911] (req 24354816->24358911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.371083:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2e3c0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24354816->24358911] (req 24354816->24358911) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760738.397568:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24354816 -> 24358912 00010000:00010000:1.0:1789760738.397588:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2e3c0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24354816->24358911] (req 24354816->24358911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.397603:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2e3c0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24354816->24358911] (req 24354816->24358911) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.397613:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2e3c0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24354816->24358911] (req 24354816->24358911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.397621:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760738.397626:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2e3c0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24354816->24358911] (req 24354816->24358911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.397636:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2e3c0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24354816->24358911] (req 24354816->24358911) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.397655:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2e3c0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24354816->24358911] (req 24354816->24358911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.397663:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2e3c0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24354816->24358911] (req 24354816->24358911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760738.397822:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febffb9f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2358:x1876699512621312:12345-192.168.203.56@tcp:4:dd.0 Request processed in 26841us (27219us total) trans 4294973265 rc 0/0 00000100:00100000:1.0:1789760738.397833:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5976 00000100:00100000:1.0:1789760738.397843:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 717889801652 time 717935263127 state 0x2 ref 0 00000001:00080000:2.0:1789760738.397976:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973265 is committed 00000100:00100000:2.0:1789760738.401312:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760738.401396:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512621440 00000100:00100000:0.0:1789760738.401406:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512621440 00000100:00100000:0.0:1789760738.401455:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 717889801652 time 717935263127 state 0x0 ref 1 00000100:00100000:0.0:1789760738.401470:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 717935523543 time 717935263127 state 0x1 ref 1 00000100:00100000:0.0:1789760738.401499:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 717935523543 time 717965798707 state 0x0 ref 1 00000100:00100000:0.0:1789760738.401512:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4132d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512621440:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760738.401557:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24358912, mtime: 0x6aad94e1, atime: 0x0, ctime: 0x6aad94e1, blocks: 0xc007 00010000:00010000:0.0:1789760738.401566:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2e3c7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.401583:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=717s v=(0 0) 00010000:00010000:0.0:1789760738.401587:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760738.401594:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2e3c7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24358912->24363007] (req 24358912->24363007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.401615:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2e3c7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24358912->24363007] (req 24358912->24363007) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760738.426950:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973266 is committed 00002000:00010000:0.0:1789760738.427112:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24358912 -> 24363008 00010000:00010000:0.0:1789760738.427126:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2e3c7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24358912->24363007] (req 24358912->24363007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.427140:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2e3c7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24358912->24363007] (req 24358912->24363007) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.427149:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2e3c7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24358912->24363007] (req 24358912->24363007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.427157:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760738.427162:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2e3c7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24358912->24363007] (req 24358912->24363007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.427172:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2e3c7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24358912->24363007] (req 24358912->24363007) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.427192:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2e3c7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24358912->24363007] (req 24358912->24363007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.427200:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2e3c7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24358912->24363007] (req 24358912->24363007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760738.427332:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4132d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512621440:12345-192.168.203.56@tcp:4:dd.0 Request processed in 25817us (26026us total) trans 4294973266 rc 0/0 00000100:00100000:0.0:1789760738.427344:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5977 00000100:00100000:0.0:1789760738.427353:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 717935523543 time 717965798707 state 0x2 ref 0 00000100:00100000:2.0:1789760738.430302:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760738.430415:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512621568 00000100:00100000:1.0:1789760738.430425:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512621568 00000100:00100000:1.0:1789760738.430476:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 717935523543 time 717965798707 state 0x0 ref 1 00000100:00100000:1.0:1789760738.430491:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 717966059123 time 717965798707 state 0x1 ref 1 00000100:00100000:1.0:1789760738.430553:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 717966059123 time 717994851797 state 0x0 ref 1 00000100:00100000:1.0:1789760738.430567:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4132300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512621568:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760738.430614:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24363008, mtime: 0x6aad94e1, atime: 0x0, ctime: 0x6aad94e1, blocks: 0xc007 00010000:00010000:1.0:1789760738.430623:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2e3ce lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.430640:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=717s v=(0 0) 00010000:00010000:1.0:1789760738.430644:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760738.430652:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2e3ce lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24363008->24367103] (req 24363008->24367103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.430666:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2e3ce lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24363008->24367103] (req 24363008->24367103) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760738.460101:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24363008 -> 24367104 00010000:00010000:3.0:1789760738.460124:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2e3ce lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24363008->24367103] (req 24363008->24367103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.460138:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2e3ce lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24363008->24367103] (req 24363008->24367103) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.460148:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2e3ce lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24363008->24367103] (req 24363008->24367103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.460156:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760738.460162:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2e3ce lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24363008->24367103] (req 24363008->24367103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.460172:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2e3ce lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24363008->24367103] (req 24363008->24367103) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.460195:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2e3ce lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24363008->24367103] (req 24363008->24367103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.460204:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2e3ce lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24363008->24367103] (req 24363008->24367103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760738.460329:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4132300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2358:x1876699512621568:12345-192.168.203.56@tcp:4:dd.0 Request processed in 29760us (30031us total) trans 4294973267 rc 0/0 00000100:00100000:3.0:1789760738.460339:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5978 00000100:00100000:3.0:1789760738.460348:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 717966059123 time 717994851797 state 0x2 ref 0 00000001:00080000:0.0:1789760738.460487:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973267 is committed 00000100:00100000:2.0:1789760738.463485:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760738.470513:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512621696 00000100:00100000:1.0:1789760738.470530:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512621696 00000100:00100000:1.0:1789760738.470597:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 717966059123 time 717994851797 state 0x0 ref 1 00000100:00100000:1.0:1789760738.470617:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 717995112213 time 717994851797 state 0x1 ref 1 00000100:00100000:1.0:1789760738.470651:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 717995112213 time 718034949946 state 0x0 ref 1 00000100:00100000:1.0:1789760738.470669:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4131f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512621696:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760738.470733:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24367104, mtime: 0x6aad94e1, atime: 0x0, ctime: 0x6aad94e1, blocks: 0xc007 00010000:00010000:1.0:1789760738.470743:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2e3d5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.470758:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=718s v=(0 0) 00010000:00010000:1.0:1789760738.470763:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760738.470769:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2e3d5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24367104->24371199] (req 24367104->24371199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.470783:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2e3d5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24367104->24371199] (req 24367104->24371199) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760738.496582:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24367104 -> 24371200 00010000:00010000:2.0:1789760738.496601:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2e3d5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24367104->24371199] (req 24367104->24371199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.496615:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2e3d5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24367104->24371199] (req 24367104->24371199) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.496625:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2e3d5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24367104->24371199] (req 24367104->24371199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.496633:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760738.496638:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2e3d5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24367104->24371199] (req 24367104->24371199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.496647:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2e3d5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24367104->24371199] (req 24367104->24371199) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.496666:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2e3d5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24367104->24371199] (req 24367104->24371199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.496680:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2e3d5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24367104->24371199] (req 24367104->24371199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760738.496785:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4131f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2359:x1876699512621696:12345-192.168.203.56@tcp:4:dd.0 Request processed in 26116us (33306us total) trans 4294973268 rc 0/0 00000100:00100000:2.0:1789760738.496795:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5979 00000100:00100000:2.0:1789760738.496805:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 717995112213 time 718034949946 state 0x2 ref 0 00000001:00080000:3.0:1789760738.497329:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973268 is committed 00000100:00100000:2.0:1789760738.500347:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760738.500442:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512621824 00000100:00100000:0.0:1789760738.500452:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512621824 00000100:00100000:0.0:1789760738.500513:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 717995112213 time 718034949946 state 0x0 ref 1 00000100:00100000:0.0:1789760738.500528:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 718035210362 time 718034949946 state 0x1 ref 1 00000100:00100000:0.0:1789760738.500558:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 718035210362 time 718064858039 state 0x0 ref 1 00000100:00100000:0.0:1789760738.500571:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4130380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512621824:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760738.500618:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24371200, mtime: 0x6aad94e1, atime: 0x0, ctime: 0x6aad94e1, blocks: 0xc007 00010000:00010000:0.0:1789760738.500627:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2e3dc lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.500643:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=718s v=(0 0) 00010000:00010000:0.0:1789760738.500647:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760738.500653:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2e3dc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24371200->24375295] (req 24371200->24375295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.500667:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2e3dc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24371200->24375295] (req 24371200->24375295) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760738.540242:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973269 is committed 00002000:00010000:3.0:1789760738.540481:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24371200 -> 24375296 00002000:00010000:3.0:1789760738.540486:0:6710:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760737 -> 1789760738 00002000:00010000:3.0:1789760738.540489:0:6710:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760737 -> 1789760738 00010000:00010000:3.0:1789760738.540503:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2e3dc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24371200->24375295] (req 24371200->24375295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.540516:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2e3dc lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24371200->24375295] (req 24371200->24375295) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.540525:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2e3dc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24371200->24375295] (req 24371200->24375295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.540532:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760738.540538:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2e3dc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24371200->24375295] (req 24371200->24375295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.540547:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2e3dc lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24371200->24375295] (req 24371200->24375295) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.540565:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2e3dc lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24371200->24375295] (req 24371200->24375295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.540572:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2e3dc lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24371200->24375295] (req 24371200->24375295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760738.540659:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4130380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512621824:12345-192.168.203.56@tcp:4:dd.0 Request processed in 40085us (40316us total) trans 4294973269 rc 0/0 00000100:00100000:3.0:1789760738.540668:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5980 00000100:00100000:3.0:1789760738.540677:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 718035210362 time 718064858039 state 0x2 ref 0 00000100:00100000:0.0:1789760738.543523:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760738.543727:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512621952 00000100:00100000:3.0:1789760738.543736:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512621952 00000100:00100000:3.0:1789760738.543791:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 718035210362 time 718064858039 state 0x0 ref 1 00000100:00100000:3.0:1789760738.543807:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 718065118455 time 718064858039 state 0x1 ref 1 00000100:00100000:3.0:1789760738.543828:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 718065118455 time 718108128542 state 0x0 ref 1 00000100:00100000:3.0:1789760738.543841:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febff932a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512621952:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760738.543889:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24375296, mtime: 0x6aad94e2, atime: 0x0, ctime: 0x6aad94e2, blocks: 0xc007 00010000:00010000:3.0:1789760738.543898:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2e3e3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.543913:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=718s v=(0 0) 00010000:00010000:3.0:1789760738.543918:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760738.543924:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2e3e3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24375296->24379391] (req 24375296->24379391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.543937:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2e3e3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24375296->24379391] (req 24375296->24379391) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760738.587691:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973270 is committed 00002000:00010000:3.0:1789760738.589908:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24375296 -> 24379392 00010000:00010000:3.0:1789760738.589928:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2e3e3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24375296->24379391] (req 24375296->24379391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.589940:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2e3e3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24375296->24379391] (req 24375296->24379391) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.589949:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2e3e3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24375296->24379391] (req 24375296->24379391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.589955:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760738.589960:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2e3e3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24375296->24379391] (req 24375296->24379391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.589968:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2e3e3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24375296->24379391] (req 24375296->24379391) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.589993:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2e3e3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24375296->24379391] (req 24375296->24379391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.590001:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2e3e3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24375296->24379391] (req 24375296->24379391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760738.590203:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febff932a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512621952:12345-192.168.203.56@tcp:4:dd.0 Request processed in 46359us (46683us total) trans 4294973270 rc 0/0 00000100:00100000:3.0:1789760738.590214:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5981 00000100:00100000:3.0:1789760738.590223:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 718065118455 time 718108128542 state 0x2 ref 0 00000100:00100000:2.0:1789760738.595182:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760738.595305:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512622080 00000100:00100000:1.0:1789760738.595314:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512622080 00000100:00100000:1.0:1789760738.595383:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 718065118455 time 718108128542 state 0x0 ref 1 00000100:00100000:1.0:1789760738.595398:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 718108388958 time 718108128542 state 0x1 ref 1 00000100:00100000:1.0:1789760738.595459:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 718108388958 time 718159757381 state 0x0 ref 1 00000100:00100000:1.0:1789760738.595472:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4133b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512622080:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760738.595518:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24379392, mtime: 0x6aad94e2, atime: 0x0, ctime: 0x6aad94e2, blocks: 0xc007 00010000:00010000:1.0:1789760738.595527:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2e3ea lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.595548:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=718s v=(0 0) 00010000:00010000:1.0:1789760738.595553:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760738.595559:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2e3ea lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24379392->24383487] (req 24379392->24383487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.595574:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2e3ea lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24379392->24383487] (req 24379392->24383487) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760738.627533:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973271 is committed 00002000:00010000:3.0:1789760738.627650:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24379392 -> 24383488 00010000:00010000:3.0:1789760738.627666:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2e3ea lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24379392->24383487] (req 24379392->24383487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.627680:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2e3ea lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24379392->24383487] (req 24379392->24383487) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.627689:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2e3ea lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24379392->24383487] (req 24379392->24383487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.627721:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760738.627726:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2e3ea lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24379392->24383487] (req 24379392->24383487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.627758:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2e3ea lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24379392->24383487] (req 24379392->24383487) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.627777:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2e3ea lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24379392->24383487] (req 24379392->24383487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.627784:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2e3ea lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24379392->24383487] (req 24379392->24383487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760738.627870:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4133b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512622080:12345-192.168.203.56@tcp:4:dd.0 Request processed in 32394us (32750us total) trans 4294973271 rc 0/0 00000100:00100000:3.0:1789760738.627880:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5982 00000100:00100000:3.0:1789760738.627889:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 718108388958 time 718159757381 state 0x2 ref 0 00000100:00100000:2.0:1789760738.635242:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760738.635309:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512622208 00000100:00100000:3.0:1789760738.635319:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512622208 00000100:00100000:3.0:1789760738.635383:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 718108388958 time 718159757381 state 0x0 ref 1 00000100:00100000:3.0:1789760738.635397:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 718160017797 time 718159757381 state 0x1 ref 1 00000100:00100000:3.0:1789760738.635424:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 718160017797 time 718199724114 state 0x0 ref 1 00000100:00100000:3.0:1789760738.635438:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4133800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512622208:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760738.635489:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24383488, mtime: 0x6aad94e2, atime: 0x0, ctime: 0x6aad94e2, blocks: 0xc007 00010000:00010000:3.0:1789760738.635498:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2e3f1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.635513:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=718s v=(0 0) 00010000:00010000:3.0:1789760738.635518:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760738.635525:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2e3f1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24383488->24387583] (req 24383488->24387583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.635539:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2e3f1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24383488->24387583] (req 24383488->24387583) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760738.667274:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973272 is committed 00002000:00010000:3.0:1789760738.667405:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24383488 -> 24387584 00010000:00010000:3.0:1789760738.667422:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2e3f1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24383488->24387583] (req 24383488->24387583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.667434:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2e3f1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24383488->24387583] (req 24383488->24387583) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.667443:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2e3f1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24383488->24387583] (req 24383488->24387583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.667450:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760738.667455:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2e3f1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24383488->24387583] (req 24383488->24387583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.667464:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2e3f1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24383488->24387583] (req 24383488->24387583) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.667484:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2e3f1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24383488->24387583] (req 24383488->24387583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.667491:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2e3f1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24383488->24387583] (req 24383488->24387583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760738.667585:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4133800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512622208:12345-192.168.203.56@tcp:4:dd.0 Request processed in 32146us (32347us total) trans 4294973272 rc 0/0 00000100:00100000:3.0:1789760738.667596:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5983 00000100:00100000:3.0:1789760738.667605:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 718160017797 time 718199724114 state 0x2 ref 0 00000100:00100000:2.0:1789760738.670427:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760738.670509:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512622336 00000100:00100000:3.0:1789760738.670519:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512622336 00000100:00100000:3.0:1789760738.670576:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 718160017797 time 718199724114 state 0x0 ref 1 00000100:00100000:3.0:1789760738.670592:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 718199984530 time 718199724114 state 0x1 ref 1 00000100:00100000:3.0:1789760738.670628:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 718199984530 time 718234927910 state 0x0 ref 1 00000100:00100000:3.0:1789760738.670646:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4130000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512622336:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760738.670724:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24387584, mtime: 0x6aad94e2, atime: 0x0, ctime: 0x6aad94e2, blocks: 0xc007 00010000:00010000:3.0:1789760738.670734:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2e3f8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.670757:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=718s v=(0 0) 00010000:00010000:3.0:1789760738.670762:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760738.670768:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2e3f8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24387584->24391679] (req 24387584->24391679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.670787:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2e3f8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24387584->24391679] (req 24387584->24391679) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.679458:0:5836:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:2.0:1789760738.721453:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973273 is committed 00002000:00010000:3.0:1789760738.721633:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24387584 -> 24391680 00010000:00010000:3.0:1789760738.721648:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2e3f8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24387584->24391679] (req 24387584->24391679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.721661:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2e3f8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24387584->24391679] (req 24387584->24391679) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.721671:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2e3f8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24387584->24391679] (req 24387584->24391679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.721679:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760738.721684:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2e3f8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24387584->24391679] (req 24387584->24391679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.721693:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2e3f8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24387584->24391679] (req 24387584->24391679) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.721736:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2e3f8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24387584->24391679] (req 24387584->24391679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.721744:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2e3f8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24387584->24391679] (req 24387584->24391679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760738.721854:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4130000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512622336:12345-192.168.203.56@tcp:4:dd.0 Request processed in 51207us (51430us total) trans 4294973273 rc 0/0 00000100:00100000:3.0:1789760738.721865:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5984 00000100:00100000:3.0:1789760738.721875:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 718199984530 time 718234927910 state 0x2 ref 0 00000100:00100000:2.0:1789760738.727644:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760738.727734:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512622464 00000100:00100000:3.0:1789760738.727826:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512622464 00000100:00100000:2.0:1789760738.728944:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 718199984530 time 718234927910 state 0x0 ref 1 00000100:00100000:2.0:1789760738.728961:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 718235188326 time 718234927910 state 0x1 ref 1 00000100:00100000:2.0:1789760738.729047:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 718235188326 time 718293346691 state 0x0 ref 1 00000100:00100000:2.0:1789760738.729061:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4133480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512622464:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760738.729112:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24391680, mtime: 0x6aad94e2, atime: 0x0, ctime: 0x6aad94e2, blocks: 0xc007 00010000:00010000:2.0:1789760738.729121:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2e3ff lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.729136:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=718s v=(0 0) 00010000:00010000:2.0:1789760738.729140:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760738.729146:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2e3ff lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24391680->24395775] (req 24391680->24395775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.729161:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2e3ff lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24391680->24395775] (req 24391680->24395775) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760738.774755:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973274 is committed 00002000:00010000:2.0:1789760738.774816:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24391680 -> 24395776 00010000:00010000:2.0:1789760738.774831:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2e3ff lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24391680->24395775] (req 24391680->24395775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.774844:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2e3ff lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24391680->24395775] (req 24391680->24395775) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.774853:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2e3ff lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24391680->24395775] (req 24391680->24395775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.774861:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760738.774865:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2e3ff lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24391680->24395775] (req 24391680->24395775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.774879:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2e3ff lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24391680->24395775] (req 24391680->24395775) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.774899:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2e3ff lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24391680->24395775] (req 24391680->24395775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.774907:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2e3ff lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24391680->24395775] (req 24391680->24395775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760738.775003:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4133480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512622464:12345-192.168.203.56@tcp:4:dd.0 Request processed in 45939us (47362us total) trans 4294973274 rc 0/0 00000100:00100000:2.0:1789760738.775014:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5985 00000100:00100000:2.0:1789760738.775033:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 718235188326 time 718293346691 state 0x2 ref 0 00000100:00100000:2.0:1789760738.782053:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760738.782121:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512622592 00000100:00100000:3.0:1789760738.782131:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512622592 00000100:00100000:3.0:1789760738.782189:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 718235188326 time 718293346691 state 0x0 ref 1 00000100:00100000:3.0:1789760738.782204:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 718293607107 time 718293346691 state 0x1 ref 1 00000100:00100000:3.0:1789760738.782295:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 718293607107 time 718346593756 state 0x0 ref 1 00000100:00100000:3.0:1789760738.782310:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4130700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512622592:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760738.782372:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24395776, mtime: 0x6aad94e2, atime: 0x0, ctime: 0x6aad94e2, blocks: 0xc007 00010000:00010000:3.0:1789760738.782382:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2e406 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.782397:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=718s v=(0 0) 00010000:00010000:3.0:1789760738.782401:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760738.782408:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2e406 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24395776->24399871] (req 24395776->24399871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.782422:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2e406 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24395776->24399871] (req 24395776->24399871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760738.811729:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973275 is committed 00002000:00010000:2.0:1789760738.811881:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24395776 -> 24399872 00010000:00010000:2.0:1789760738.811896:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2e406 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24395776->24399871] (req 24395776->24399871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.811909:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2e406 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24395776->24399871] (req 24395776->24399871) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.811919:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2e406 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24395776->24399871] (req 24395776->24399871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.811932:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760738.811937:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2e406 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24395776->24399871] (req 24395776->24399871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.811947:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2e406 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24395776->24399871] (req 24395776->24399871) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.811966:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2e406 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24395776->24399871] (req 24395776->24399871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.811975:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2e406 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24395776->24399871] (req 24395776->24399871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760738.812119:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4130700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512622592:12345-192.168.203.56@tcp:4:dd.0 Request processed in 29807us (30071us total) trans 4294973275 rc 0/0 00000100:00100000:2.0:1789760738.812130:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5986 00000100:00100000:2.0:1789760738.812140:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 718293607107 time 718346593756 state 0x2 ref 0 00000100:00100000:1.0:1789760738.818910:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760738.819032:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512622720 00000100:00100000:3.0:1789760738.819042:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512622720 00000100:00100000:3.0:1789760738.819114:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 718293607107 time 718346593756 state 0x0 ref 1 00000100:00100000:3.0:1789760738.819130:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 718346854172 time 718346593756 state 0x1 ref 1 00000100:00100000:3.0:1789760738.819161:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 718346854172 time 718383460681 state 0x0 ref 1 00000100:00100000:3.0:1789760738.819174:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2fb9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512622720:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760738.819225:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24399872, mtime: 0x6aad94e2, atime: 0x0, ctime: 0x6aad94e2, blocks: 0xc007 00010000:00010000:3.0:1789760738.819245:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2e40d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.819261:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=718s v=(0 0) 00010000:00010000:3.0:1789760738.819265:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760738.819271:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2e40d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24399872->24403967] (req 24399872->24403967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.819286:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2e40d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24399872->24403967] (req 24399872->24403967) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760738.849371:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973276 is committed 00002000:00010000:0.0:1789760738.849511:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24399872 -> 24403968 00010000:00010000:0.0:1789760738.849527:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2e40d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24399872->24403967] (req 24399872->24403967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.849540:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2e40d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24399872->24403967] (req 24399872->24403967) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.849549:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2e40d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24399872->24403967] (req 24399872->24403967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.849557:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760738.849562:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2e40d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24399872->24403967] (req 24399872->24403967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.849572:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2e40d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24399872->24403967] (req 24399872->24403967) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.849591:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2e40d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24399872->24403967] (req 24399872->24403967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.849598:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2e40d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24399872->24403967] (req 24399872->24403967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760738.849735:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2fb9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512622720:12345-192.168.203.56@tcp:4:dd.0 Request processed in 30521us (30791us total) trans 4294973276 rc 0/0 00000100:00100000:0.0:1789760738.850231:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5987 00000100:00100000:0.0:1789760738.850244:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 718346854172 time 718383460681 state 0x2 ref 0 00000100:00100000:2.0:1789760738.854330:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760738.854430:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512622848 00000100:00100000:0.0:1789760738.854440:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512622848 00000100:00100000:0.0:1789760738.854494:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 718346854172 time 718383460681 state 0x0 ref 1 00000100:00100000:0.0:1789760738.854509:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 718383721097 time 718383460681 state 0x1 ref 1 00000100:00100000:0.0:1789760738.854579:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 718383721097 time 718418877421 state 0x0 ref 1 00000100:00100000:0.0:1789760738.854592:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4131180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512622848:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760738.854639:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24403968, mtime: 0x6aad94e2, atime: 0x0, ctime: 0x6aad94e2, blocks: 0xc007 00010000:00010000:0.0:1789760738.854649:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2e414 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.854665:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=718s v=(0 0) 00010000:00010000:0.0:1789760738.854670:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760738.854675:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2e414 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24403968->24408063] (req 24403968->24408063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.854687:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2e414 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24403968->24408063] (req 24403968->24408063) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760738.888535:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973277 is committed 00002000:00010000:0.0:1789760738.888637:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24403968 -> 24408064 00010000:00010000:0.0:1789760738.888653:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2e414 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24403968->24408063] (req 24403968->24408063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.888665:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2e414 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24403968->24408063] (req 24403968->24408063) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.888674:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2e414 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24403968->24408063] (req 24403968->24408063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.888681:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760738.888686:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2e414 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24403968->24408063] (req 24403968->24408063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.888715:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2e414 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24403968->24408063] (req 24403968->24408063) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.888737:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2e414 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24403968->24408063] (req 24403968->24408063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.888822:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2e414 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24403968->24408063] (req 24403968->24408063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760738.888914:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4131180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512622848:12345-192.168.203.56@tcp:4:dd.0 Request processed in 34319us (34588us total) trans 4294973277 rc 0/0 00000100:00100000:0.0:1789760738.888928:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5988 00000100:00100000:0.0:1789760738.888937:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 718383721097 time 718418877421 state 0x2 ref 0 00000100:00100000:2.0:1789760738.892518:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760738.892616:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512622976 00000100:00100000:3.0:1789760738.892626:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512622976 00000100:00100000:3.0:1789760738.892682:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 718383721097 time 718418877421 state 0x0 ref 1 00000100:00100000:3.0:1789760738.892725:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 718419137837 time 718418877421 state 0x1 ref 1 00000100:00100000:3.0:1789760738.892904:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 718419137837 time 718457050717 state 0x0 ref 1 00000100:00100000:3.0:1789760738.892919:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4130a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512622976:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760738.892967:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24408064, mtime: 0x6aad94e2, atime: 0x0, ctime: 0x6aad94e2, blocks: 0xc007 00010000:00010000:3.0:1789760738.892976:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2e41b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.892992:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=718s v=(0 0) 00010000:00010000:3.0:1789760738.892997:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760738.893004:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2e41b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24408064->24412159] (req 24408064->24412159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.893018:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2e41b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24408064->24412159] (req 24408064->24412159) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760738.931497:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973278 is committed 00002000:00010000:1.0:1789760738.932853:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24408064 -> 24412160 00010000:00010000:1.0:1789760738.932944:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2e41b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24408064->24412159] (req 24408064->24412159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.932957:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2e41b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24408064->24412159] (req 24408064->24412159) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.932966:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2e41b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24408064->24412159] (req 24408064->24412159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.932974:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760738.932979:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2e41b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24408064->24412159] (req 24408064->24412159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.932987:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2e41b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24408064->24412159] (req 24408064->24412159) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.933008:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2e41b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24408064->24412159] (req 24408064->24412159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.933016:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2e41b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24408064->24412159] (req 24408064->24412159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760738.933109:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4130a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512622976:12345-192.168.203.56@tcp:4:dd.0 Request processed in 40188us (40595us total) trans 4294973278 rc 0/0 00000100:00100000:1.0:1789760738.933120:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5989 00000100:00100000:1.0:1789760738.933129:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 718419137837 time 718457050717 state 0x2 ref 0 00000100:00100000:1.0:1789760738.936650:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760738.936772:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512623104 00000100:00100000:0.0:1789760738.936784:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512623104 00000100:00100000:0.0:1789760738.936845:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 718419137837 time 718457050717 state 0x0 ref 1 00000100:00100000:0.0:1789760738.936862:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 718457311133 time 718457050717 state 0x1 ref 1 00000100:00100000:0.0:1789760738.936939:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 718457311133 time 718501238137 state 0x0 ref 1 00000100:00100000:0.0:1789760738.936954:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2fbaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512623104:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760738.937014:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24412160, mtime: 0x6aad94e2, atime: 0x0, ctime: 0x6aad94e2, blocks: 0xc007 00010000:00010000:0.0:1789760738.937023:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2e422 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.937051:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=718s v=(0 0) 00010000:00010000:0.0:1789760738.937056:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760738.937062:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2e422 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24412160->24416255] (req 24412160->24416255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.937076:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2e422 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24412160->24416255] (req 24412160->24416255) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760738.977724:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973279 is committed 00002000:00010000:0.0:1789760738.977832:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24412160 -> 24416256 00010000:00010000:0.0:1789760738.977847:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2e422 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24412160->24416255] (req 24412160->24416255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.977861:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2e422 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24412160->24416255] (req 24412160->24416255) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.977871:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2e422 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24412160->24416255] (req 24412160->24416255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.977879:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760738.977884:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2e422 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24412160->24416255] (req 24412160->24416255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.977893:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2e422 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24412160->24416255] (req 24412160->24416255) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.977913:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2e422 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24412160->24416255] (req 24412160->24416255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.977921:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2e422 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24412160->24416255] (req 24412160->24416255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760738.978009:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2fbaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512623104:12345-192.168.203.56@tcp:4:dd.0 Request processed in 41053us (41363us total) trans 4294973279 rc 0/0 00000100:00100000:0.0:1789760738.978020:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5990 00000100:00100000:0.0:1789760738.978036:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 718457311133 time 718501238137 state 0x2 ref 0 00000100:00100000:2.0:1789760738.982930:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760738.983043:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512623232 00000100:00100000:0.0:1789760738.983054:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512623232 00000100:00100000:0.0:1789760738.983109:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 718457311133 time 718501238137 state 0x0 ref 1 00000100:00100000:0.0:1789760738.983126:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 718501498553 time 718501238137 state 0x1 ref 1 00000100:00100000:0.0:1789760738.983160:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 718501498553 time 718547459641 state 0x0 ref 1 00000100:00100000:0.0:1789760738.983172:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac28f0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512623232:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760738.983234:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24416256, mtime: 0x6aad94e2, atime: 0x0, ctime: 0x6aad94e2, blocks: 0xc007 00010000:00010000:0.0:1789760738.983244:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2e429 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.983259:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=718s v=(0 0) 00010000:00010000:0.0:1789760738.983263:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760738.983270:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2e429 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24416256->24420351] (req 24416256->24420351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.983282:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2e429 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24416256->24420351] (req 24416256->24420351) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760739.041368:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973280 is committed 00002000:00010000:2.0:1789760739.041473:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24416256 -> 24420352 00010000:00010000:2.0:1789760739.041489:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2e429 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24416256->24420351] (req 24416256->24420351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.041502:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2e429 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24416256->24420351] (req 24416256->24420351) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.041512:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2e429 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24416256->24420351] (req 24416256->24420351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.041520:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760739.041524:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2e429 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24416256->24420351] (req 24416256->24420351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.041533:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2e429 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24416256->24420351] (req 24416256->24420351) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.041553:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2e429 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24416256->24420351] (req 24416256->24420351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.041561:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2e429 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24416256->24420351] (req 24416256->24420351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760739.041660:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac28f0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512623232:12345-192.168.203.56@tcp:4:dd.0 Request processed in 58485us (58735us total) trans 4294973280 rc 0/0 00000100:00100000:2.0:1789760739.041671:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5991 00000100:00100000:2.0:1789760739.041681:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 718501498553 time 718547459641 state 0x2 ref 0 00000100:00100000:2.0:1789760739.044906:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760739.044968:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512623360 00000100:00100000:1.0:1789760739.044978:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512623360 00000100:00100000:1.0:1789760739.045082:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 718501498553 time 718547459641 state 0x0 ref 1 00000100:00100000:1.0:1789760739.045098:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 718547720057 time 718547459641 state 0x1 ref 1 00000100:00100000:1.0:1789760739.045128:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 718547720057 time 718609426923 state 0x0 ref 1 00000100:00100000:1.0:1789760739.045151:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac38b6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512623360:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760739.045200:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24420352, mtime: 0x6aad94e2, atime: 0x0, ctime: 0x6aad94e2, blocks: 0xc007 00010000:00010000:1.0:1789760739.045209:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2e430 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.045229:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=718s v=(0 0) 00010000:00010000:1.0:1789760739.045233:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760739.045238:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2e430 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24420352->24424447] (req 24420352->24424447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.045251:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2e430 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24420352->24424447] (req 24420352->24424447) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1789760739.046221:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:3.0:1789760739.110899:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973281 is committed 00002000:00010000:2.0:1789760739.110965:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24420352 -> 24424448 00010000:00010000:2.0:1789760739.110982:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2e430 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24420352->24424447] (req 24420352->24424447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.110997:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2e430 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24420352->24424447] (req 24420352->24424447) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.111007:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2e430 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24420352->24424447] (req 24420352->24424447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.111014:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760739.111019:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2e430 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24420352->24424447] (req 24420352->24424447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.111028:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2e430 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24420352->24424447] (req 24420352->24424447) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.111048:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2e430 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24420352->24424447] (req 24420352->24424447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.111057:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2e430 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24420352->24424447] (req 24420352->24424447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760739.111157:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac38b6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512623360:12345-192.168.203.56@tcp:4:dd.0 Request processed in 66013us (66256us total) trans 4294973281 rc 0/0 00000100:00100000:2.0:1789760739.111167:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5992 00000100:00100000:2.0:1789760739.111177:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 718547720057 time 718609426923 state 0x2 ref 0 00000100:00100000:3.0:1789760739.116067:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760739.116241:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512623488 00000100:00100000:0.0:1789760739.116251:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512623488 00000100:00100000:0.0:1789760739.116303:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 718547720057 time 718609426923 state 0x0 ref 1 00000100:00100000:0.0:1789760739.116318:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 718609687339 time 718609426923 state 0x1 ref 1 00000100:00100000:0.0:1789760739.116349:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 718609687339 time 718680648380 state 0x0 ref 1 00000100:00100000:0.0:1789760739.116375:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2746300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512623488:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760739.116422:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24424448, mtime: 0x6aad94e2, atime: 0x0, ctime: 0x6aad94e2, blocks: 0xc007 00010000:00010000:0.0:1789760739.116430:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2e437 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.116446:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=718s v=(0 0) 00010000:00010000:0.0:1789760739.116451:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760739.116458:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2e437 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24424448->24428543] (req 24424448->24428543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.116472:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2e437 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24424448->24428543] (req 24424448->24428543) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760739.198195:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973282 is committed 00002000:00010000:2.0:1789760739.198362:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24424448 -> 24428544 00010000:00010000:2.0:1789760739.198379:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2e437 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24424448->24428543] (req 24424448->24428543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.198399:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2e437 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24424448->24428543] (req 24424448->24428543) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.198408:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2e437 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24424448->24428543] (req 24424448->24428543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.198416:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760739.198421:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2e437 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24424448->24428543] (req 24424448->24428543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.198429:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2e437 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24424448->24428543] (req 24424448->24428543) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.198503:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2e437 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24424448->24428543] (req 24424448->24428543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.198512:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2e437 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24424448->24428543] (req 24424448->24428543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760739.198597:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2746300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512623488:12345-192.168.203.56@tcp:4:dd.0 Request processed in 82221us (82535us total) trans 4294973282 rc 0/0 00000100:00100000:2.0:1789760739.198608:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5993 00000100:00100000:2.0:1789760739.198619:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 718609687339 time 718680648380 state 0x2 ref 0 00000100:00100000:3.0:1789760739.203210:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760739.203269:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512623616 00000100:00100000:2.0:1789760739.203279:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512623616 00000100:00100000:2.0:1789760739.203335:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 718609687339 time 718680648380 state 0x0 ref 1 00000100:00100000:2.0:1789760739.203361:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 718680908796 time 718680648380 state 0x1 ref 1 00000100:00100000:2.0:1789760739.203397:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 718680908796 time 718767696679 state 0x0 ref 1 00000100:00100000:2.0:1789760739.203409:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2745180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512623616:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760739.203458:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24428544, mtime: 0x6aad94e2, atime: 0x0, ctime: 0x6aad94e2, blocks: 0xc007 00010000:00010000:2.0:1789760739.203467:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2e43e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.203483:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=718s v=(0 0) 00010000:00010000:2.0:1789760739.203488:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760739.203495:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2e43e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24428544->24432639] (req 24428544->24432639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.203509:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2e43e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24428544->24432639] (req 24428544->24432639) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760739.238420:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973283 is committed 00002000:00010000:0.0:1789760739.238530:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24428544 -> 24432640 00010000:00010000:0.0:1789760739.238545:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2e43e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24428544->24432639] (req 24428544->24432639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.238558:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2e43e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24428544->24432639] (req 24428544->24432639) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.238566:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2e43e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24428544->24432639] (req 24428544->24432639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.238573:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760739.238578:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2e43e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24428544->24432639] (req 24428544->24432639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.238588:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2e43e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24428544->24432639] (req 24428544->24432639) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.238607:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2e43e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24428544->24432639] (req 24428544->24432639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.238614:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2e43e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24428544->24432639] (req 24428544->24432639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760739.238783:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2745180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512623616:12345-192.168.203.56@tcp:4:dd.0 Request processed in 35371us (35579us total) trans 4294973283 rc 0/0 00000100:00100000:0.0:1789760739.238794:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5994 00000100:00100000:0.0:1789760739.238804:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 718680908796 time 718767696679 state 0x2 ref 0 00000100:00100000:2.0:1789760739.242848:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760739.242940:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512623744 00000100:00100000:1.0:1789760739.242950:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512623744 00000100:00100000:1.0:1789760739.243000:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 718680908796 time 718767696679 state 0x0 ref 1 00000100:00100000:1.0:1789760739.243015:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 718767957095 time 718767696679 state 0x1 ref 1 00000100:00100000:1.0:1789760739.243037:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 718767957095 time 718807337475 state 0x0 ref 1 00000100:00100000:1.0:1789760739.243048:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac531ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512623744:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760739.243094:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24432640, mtime: 0x6aad94e2, atime: 0x0, ctime: 0x6aad94e2, blocks: 0xc007 00010000:00010000:1.0:1789760739.243103:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2e445 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.243117:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=718s v=(0 0) 00010000:00010000:1.0:1789760739.243121:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760739.243127:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2e445 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24432640->24436735] (req 24432640->24436735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.243146:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2e445 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24432640->24436735] (req 24432640->24436735) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760739.281325:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973284 is committed 00002000:00010000:3.0:1789760739.281455:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24432640 -> 24436736 00010000:00010000:3.0:1789760739.281471:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2e445 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24432640->24436735] (req 24432640->24436735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.281489:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2e445 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24432640->24436735] (req 24432640->24436735) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.281499:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2e445 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24432640->24436735] (req 24432640->24436735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.281507:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760739.281512:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2e445 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24432640->24436735] (req 24432640->24436735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.281522:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2e445 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24432640->24436735] (req 24432640->24436735) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.281544:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2e445 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24432640->24436735] (req 24432640->24436735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.281552:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2e445 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24432640->24436735] (req 24432640->24436735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760739.281740:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac531ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512623744:12345-192.168.203.56@tcp:4:dd.0 Request processed in 38641us (38849us total) trans 4294973284 rc 0/0 00000100:00100000:3.0:1789760739.281752:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5995 00000100:00100000:3.0:1789760739.281764:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 718767957095 time 718807337475 state 0x2 ref 0 00000100:00100000:3.0:1789760739.286348:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760739.286760:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512623872 00000100:00100000:2.0:1789760739.286779:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512623872 00000100:00100000:2.0:1789760739.286843:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 718767957095 time 718807337475 state 0x0 ref 1 00000100:00100000:2.0:1789760739.286860:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 718807597891 time 718807337475 state 0x1 ref 1 00000100:00100000:2.0:1789760739.286889:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 718807597891 time 718851188755 state 0x0 ref 1 00000100:00100000:2.0:1789760739.286901:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2744a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512623872:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760739.286944:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24436736, mtime: 0x6aad94e2, atime: 0x0, ctime: 0x6aad94e2, blocks: 0xc007 00010000:00010000:2.0:1789760739.286953:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2e44c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.286969:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=718s v=(0 0) 00010000:00010000:2.0:1789760739.286973:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760739.286979:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2e44c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24436736->24440831] (req 24436736->24440831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.286993:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2e44c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24436736->24440831] (req 24436736->24440831) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760739.347296:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24436736 -> 24440832 00010000:00010000:0.0:1789760739.347316:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2e44c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24436736->24440831] (req 24436736->24440831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.347329:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2e44c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24436736->24440831] (req 24436736->24440831) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.347337:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2e44c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24436736->24440831] (req 24436736->24440831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.347348:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760739.347363:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2e44c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24436736->24440831] (req 24436736->24440831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.347372:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2e44c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24436736->24440831] (req 24436736->24440831) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.347398:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2e44c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24436736->24440831] (req 24436736->24440831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.347406:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2e44c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24436736->24440831] (req 24436736->24440831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760739.347503:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2744a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2361:x1876699512623872:12345-192.168.203.56@tcp:4:dd.0 Request processed in 60600us (61161us total) trans 4294973285 rc 0/0 00000100:00100000:0.0:1789760739.347513:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5996 00000100:00100000:0.0:1789760739.347523:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 718807597891 time 718851188755 state 0x2 ref 0 00000001:00080000:3.0:1789760739.347865:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973285 is committed 00000100:00100000:2.0:1789760739.351333:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760739.351574:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512624000 00000100:00100000:0.0:1789760739.351585:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512624000 00000100:00100000:0.0:1789760739.351637:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 718807597891 time 718851188755 state 0x0 ref 1 00000100:00100000:0.0:1789760739.351651:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 718851449171 time 718851188755 state 0x1 ref 1 00000100:00100000:0.0:1789760739.351682:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 718851449171 time 718915982712 state 0x0 ref 1 00000100:00100000:0.0:1789760739.351740:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac531e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512624000:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760739.351787:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24440832, mtime: 0x6aad94e2, atime: 0x0, ctime: 0x6aad94e2, blocks: 0xc007 00010000:00010000:0.0:1789760739.351795:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2e453 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.351812:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=718s v=(0 0) 00010000:00010000:0.0:1789760739.351816:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760739.351822:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2e453 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24440832->24444927] (req 24440832->24444927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.351834:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2e453 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24440832->24444927] (req 24440832->24444927) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760739.406381:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973286 is committed 00002000:00010000:0.0:1789760739.406572:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24440832 -> 24444928 00010000:00010000:0.0:1789760739.406587:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2e453 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24440832->24444927] (req 24440832->24444927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.406600:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2e453 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24440832->24444927] (req 24440832->24444927) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.406610:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2e453 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24440832->24444927] (req 24440832->24444927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.406617:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760739.406623:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2e453 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24440832->24444927] (req 24440832->24444927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.406632:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2e453 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24440832->24444927] (req 24440832->24444927) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.406651:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2e453 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24440832->24444927] (req 24440832->24444927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.406659:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2e453 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24440832->24444927] (req 24440832->24444927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760739.407333:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac531e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512624000:12345-192.168.203.56@tcp:4:dd.0 Request processed in 55631us (56001us total) trans 4294973286 rc 0/0 00000100:00100000:3.0:1789760739.407344:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5997 00000100:00100000:3.0:1789760739.407363:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 718851449171 time 718915982712 state 0x2 ref 0 00000100:00100000:1.0:1789760739.414401:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760739.414531:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512624128 00000100:00100000:3.0:1789760739.414542:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512624128 00000100:00100000:3.0:1789760739.414596:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 718851449171 time 718915982712 state 0x0 ref 1 00000100:00100000:3.0:1789760739.414612:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 718916243128 time 718915982712 state 0x1 ref 1 00000100:00100000:3.0:1789760739.414644:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 718916243128 time 718978943639 state 0x0 ref 1 00000100:00100000:3.0:1789760739.414658:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2fb8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512624128:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760739.414948:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24444928, mtime: 0x6aad94e2, atime: 0x0, ctime: 0x6aad94e2, blocks: 0xc007 00010000:00010000:2.0:1789760739.414960:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2e45a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.414976:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=718s v=(0 0) 00010000:00010000:2.0:1789760739.414980:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760739.414988:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2e45a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24444928->24449023] (req 24444928->24449023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.415009:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2e45a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24444928->24449023] (req 24444928->24449023) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760739.467863:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24444928 -> 24449024 00010000:00010000:1.0:1789760739.467883:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2e45a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24444928->24449023] (req 24444928->24449023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.467896:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2e45a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24444928->24449023] (req 24444928->24449023) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.467905:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2e45a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24444928->24449023] (req 24444928->24449023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.467912:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760739.467917:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2e45a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24444928->24449023] (req 24444928->24449023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.467926:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2e45a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24444928->24449023] (req 24444928->24449023) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.467947:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2e45a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24444928->24449023] (req 24444928->24449023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.467954:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2e45a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24444928->24449023] (req 24444928->24449023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760739.468051:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2fb8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2358:x1876699512624128:12345-192.168.203.56@tcp:4:dd.0 Request processed in 53391us (53653us total) trans 4294973287 rc 0/0 00000100:00100000:1.0:1789760739.468061:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5998 00000100:00100000:1.0:1789760739.468069:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 718916243128 time 718978943639 state 0x2 ref 0 00000001:00080000:3.0:1789760739.468209:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973287 is committed 00000100:00100000:0.0:1789760739.471178:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760739.471277:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512624256 00000100:00100000:2.0:1789760739.471287:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512624256 00000100:00100000:2.0:1789760739.471340:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 718916243128 time 718978943639 state 0x0 ref 1 00000100:00100000:2.0:1789760739.471364:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 718979204055 time 718978943639 state 0x1 ref 1 00000100:00100000:2.0:1789760739.471430:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 718979204055 time 719035729781 state 0x0 ref 1 00000100:00100000:2.0:1789760739.471444:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febff930e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512624256:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760739.471491:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24449024, mtime: 0x6aad94e2, atime: 0x0, ctime: 0x6aad94e2, blocks: 0xc007 00010000:00010000:2.0:1789760739.471500:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2e461 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.471515:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=719s v=(0 0) 00010000:00010000:2.0:1789760739.471520:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760739.471526:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2e461 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24449024->24453119] (req 24449024->24453119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.471541:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2e461 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24449024->24453119] (req 24449024->24453119) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760739.504861:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973288 is committed 00002000:00010000:2.0:1789760739.505250:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24449024 -> 24453120 00010000:00010000:2.0:1789760739.505267:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2e461 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24449024->24453119] (req 24449024->24453119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.505280:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2e461 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24449024->24453119] (req 24449024->24453119) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.505289:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2e461 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24449024->24453119] (req 24449024->24453119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.505296:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760739.505302:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2e461 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24449024->24453119] (req 24449024->24453119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.505311:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2e461 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24449024->24453119] (req 24449024->24453119) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.505335:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2e461 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24449024->24453119] (req 24449024->24453119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.505343:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2e461 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24449024->24453119] (req 24449024->24453119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760739.505517:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febff930e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512624256:12345-192.168.203.56@tcp:4:dd.0 Request processed in 34071us (34345us total) trans 4294973288 rc 0/0 00000100:00100000:2.0:1789760739.505529:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 5999 00000100:00100000:2.0:1789760739.505540:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 718979204055 time 719035729781 state 0x2 ref 0 00000100:00100000:2.0:1789760739.508565:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760739.508634:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512624384 00000100:00100000:3.0:1789760739.508644:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512624384 00000100:00100000:3.0:1789760739.508842:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 718979204055 time 719035729781 state 0x0 ref 1 00000100:00100000:3.0:1789760739.508861:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 719035990197 time 719035729781 state 0x1 ref 1 00000100:00100000:3.0:1789760739.508926:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 719035990197 time 719073225534 state 0x0 ref 1 00000100:00100000:3.0:1789760739.508938:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac531dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512624384:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760739.508987:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24453120, mtime: 0x6aad94e2, atime: 0x0, ctime: 0x6aad94e2, blocks: 0xc007 00010000:00010000:3.0:1789760739.508996:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2e468 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.509011:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=719s v=(0 0) 00010000:00010000:3.0:1789760739.509016:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760739.509022:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2e468 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24453120->24457215] (req 24453120->24457215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.509036:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2e468 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24453120->24457215] (req 24453120->24457215) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760739.543539:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973289 is committed 00002000:00010000:0.0:1789760739.543782:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24453120 -> 24457216 00002000:00010000:0.0:1789760739.543787:0:13062:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760738 -> 1789760739 00002000:00010000:0.0:1789760739.543790:0:13062:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760738 -> 1789760739 00010000:00010000:0.0:1789760739.543804:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2e468 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24453120->24457215] (req 24453120->24457215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.543817:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2e468 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24453120->24457215] (req 24453120->24457215) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.543824:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2e468 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24453120->24457215] (req 24453120->24457215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.543831:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760739.543835:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2e468 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24453120->24457215] (req 24453120->24457215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.543848:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2e468 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24453120->24457215] (req 24453120->24457215) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.543866:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2e468 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24453120->24457215] (req 24453120->24457215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.543874:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2e468 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24453120->24457215] (req 24453120->24457215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760739.544004:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac531dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512624384:12345-192.168.203.56@tcp:4:dd.0 Request processed in 35063us (35443us total) trans 4294973289 rc 0/0 00000100:00100000:0.0:1789760739.544014:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6000 00000100:00100000:0.0:1789760739.544032:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 719035990197 time 719073225534 state 0x2 ref 0 00000100:00100000:1.0:1789760739.546783:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760739.546871:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512624512 00000100:00100000:2.0:1789760739.546880:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512624512 00000100:00100000:2.0:1789760739.546929:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 719035990197 time 719073225534 state 0x0 ref 1 00000100:00100000:2.0:1789760739.546944:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 719073485950 time 719073225534 state 0x1 ref 1 00000100:00100000:2.0:1789760739.546972:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 719073485950 time 719111271581 state 0x0 ref 1 00000100:00100000:2.0:1789760739.546984:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2fba680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512624512:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760739.547134:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24457216, mtime: 0x6aad94e3, atime: 0x0, ctime: 0x6aad94e3, blocks: 0xc007 00010000:00010000:2.0:1789760739.547144:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2e46f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.547159:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=719s v=(0 0) 00010000:00010000:2.0:1789760739.547164:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760739.547171:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2e46f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24457216->24461311] (req 24457216->24461311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.547185:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2e46f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24457216->24461311] (req 24457216->24461311) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760739.589809:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24457216 -> 24461312 00000001:00080000:0.0:1789760739.589810:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973290 is committed 00010000:00010000:2.0:1789760739.589833:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2e46f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24457216->24461311] (req 24457216->24461311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.589852:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2e46f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24457216->24461311] (req 24457216->24461311) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.589861:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2e46f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24457216->24461311] (req 24457216->24461311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.589867:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760739.589872:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2e46f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24457216->24461311] (req 24457216->24461311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.589881:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2e46f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24457216->24461311] (req 24457216->24461311) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.589901:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2e46f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24457216->24461311] (req 24457216->24461311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.589909:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2e46f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24457216->24461311] (req 24457216->24461311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760739.590074:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2fba680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512624512:12345-192.168.203.56@tcp:4:dd.0 Request processed in 43086us (43294us total) trans 4294973290 rc 0/0 00000100:00100000:2.0:1789760739.590086:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6001 00000100:00100000:2.0:1789760739.590095:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 719073485950 time 719111271581 state 0x2 ref 0 00000100:00100000:3.0:1789760739.591471:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760739.591555:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760739.591592:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760739.591622:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512624896 00000100:00100000:0.0:1789760739.591632:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512624896 00000100:00100000:0.0:1789760739.591693:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512624768 00000100:00100000:0.0:1789760739.591712:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512624768 00000100:00100000:0.0:1789760739.591742:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 469 00000100:00100000:0.0:1789760739.591752:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2746d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:d2c40150-e97b-40da-952b-cb9f18bf7e96+5:2360:x1876699512624896:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:1.0:1789760739.591754:0:5848:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 470 00000100:00100000:1.0:1789760739.591780:0:5848:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2744380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+8:2360:x1876699512624768:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:0.0:1789760739.591815:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2746d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:d2c40150-e97b-40da-952b-cb9f18bf7e96+5:2360:x1876699512624896:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 64us (260us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760739.591822:0:5848:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2744380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+8:2360:x1876699512624768:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 53us (226us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760739.591828:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 469 00000100:00100000:1.0:1789760739.591837:0:5848:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 470 00000100:00100000:2.0:1789760739.592132:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512624640 00000100:00100000:2.0:1789760739.592138:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512624640 00000100:00100000:2.0:1789760739.592167:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 396 00000100:00100000:2.0:1789760739.592174:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febe0503450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:e6cf7267-9b19-462a-a263-14115df15dab+13:2360:x1876699512624640:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:2.0:1789760739.592230:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febe0503450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:e6cf7267-9b19-462a-a263-14115df15dab+13:2360:x1876699512624640:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 56us (766us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760739.592237:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 396 00000100:00080000:3.0:1789760739.592684:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789760739.593164:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760739.593452:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512625024 00000100:00100000:2.0:1789760739.593459:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512625024 00000100:00100000:2.0:1789760739.593499:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 719073485950 time 719111271581 state 0x0 ref 1 00000100:00100000:2.0:1789760739.593512:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 719111531997 time 719111271581 state 0x1 ref 1 00000100:00100000:2.0:1789760739.593538:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 719111531997 time 719157839296 state 0x0 ref 1 00000100:00100000:2.0:1789760739.593547:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2746680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512625024:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760739.593581:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24461312, mtime: 0x6aad94e3, atime: 0x0, ctime: 0x6aad94e3, blocks: 0xc007 00010000:00010000:2.0:1789760739.593587:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2e476 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.593600:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=719s v=(0 0) 00010000:00010000:2.0:1789760739.593604:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760739.593608:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2e476 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24461312->24465407] (req 24461312->24465407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.593618:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2e476 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24461312->24465407] (req 24461312->24465407) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760739.621146:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973291 is committed 00002000:00010000:3.0:1789760739.621232:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24461312 -> 24465408 00010000:00010000:3.0:1789760739.621248:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2e476 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24461312->24465407] (req 24461312->24465407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.621260:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2e476 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24461312->24465407] (req 24461312->24465407) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.621269:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2e476 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24461312->24465407] (req 24461312->24465407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.621281:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760739.621286:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2e476 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24461312->24465407] (req 24461312->24465407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.621296:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2e476 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24461312->24465407] (req 24461312->24465407) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.621314:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2e476 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24461312->24465407] (req 24461312->24465407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.621328:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2e476 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24461312->24465407] (req 24461312->24465407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760739.621600:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2746680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512625024:12345-192.168.203.56@tcp:4:dd.0 Request processed in 28047us (28433us total) trans 4294973291 rc 0/0 00000100:00100000:3.0:1789760739.621611:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6002 00000100:00100000:3.0:1789760739.621620:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 719111531997 time 719157839296 state 0x2 ref 0 00000100:00100000:1.0:1789760739.626312:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760739.626397:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512625152 00000100:00100000:0.0:1789760739.626409:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512625152 00000100:00100000:0.0:1789760739.626463:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 719111531997 time 719157839296 state 0x0 ref 1 00000100:00100000:0.0:1789760739.626484:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 719158099712 time 719157839296 state 0x1 ref 1 00000100:00100000:0.0:1789760739.626516:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 719158099712 time 719190815637 state 0x0 ref 1 00000100:00100000:0.0:1789760739.626529:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2fb8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512625152:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760739.626575:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24465408, mtime: 0x6aad94e3, atime: 0x0, ctime: 0x6aad94e3, blocks: 0xc007 00010000:00010000:0.0:1789760739.626585:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2e47d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.626601:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=719s v=(0 0) 00010000:00010000:0.0:1789760739.626605:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760739.626612:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2e47d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24465408->24469503] (req 24465408->24469503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.626627:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2e47d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24465408->24469503] (req 24465408->24469503) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760739.655753:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24465408 -> 24469504 00010000:00010000:0.0:1789760739.655776:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2e47d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24465408->24469503] (req 24465408->24469503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.655789:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2e47d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24465408->24469503] (req 24465408->24469503) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.655803:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2e47d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24465408->24469503] (req 24465408->24469503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.655811:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760739.655816:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2e47d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24465408->24469503] (req 24465408->24469503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.655825:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2e47d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24465408->24469503] (req 24465408->24469503) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.655842:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2e47d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24465408->24469503] (req 24465408->24469503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.655850:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2e47d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24465408->24469503] (req 24465408->24469503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760739.655936:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2fb8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2359:x1876699512625152:12345-192.168.203.56@tcp:4:dd.0 Request processed in 29405us (29628us total) trans 4294973292 rc 0/0 00000100:00100000:0.0:1789760739.655947:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6003 00000100:00100000:0.0:1789760739.655958:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 719158099712 time 719190815637 state 0x2 ref 0 00000001:00080000:3.0:1789760739.656970:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973292 is committed 00000100:00100000:0.0:1789760739.663687:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760739.663830:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512625280 00000100:00100000:2.0:1789760739.663840:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512625280 00000100:00100000:2.0:1789760739.663892:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 719158099712 time 719190815637 state 0x0 ref 1 00000100:00100000:2.0:1789760739.663907:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 719191076053 time 719190815637 state 0x1 ref 1 00000100:00100000:2.0:1789760739.664103:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 719191076053 time 719228402080 state 0x0 ref 1 00000100:00100000:2.0:1789760739.664118:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febfb374380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512625280:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760739.664167:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24469504, mtime: 0x6aad94e3, atime: 0x0, ctime: 0x6aad94e3, blocks: 0xc007 00010000:00010000:2.0:1789760739.664177:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2e484 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.664192:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=719s v=(0 0) 00010000:00010000:2.0:1789760739.664196:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760739.664203:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2e484 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24469504->24473599] (req 24469504->24473599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.664217:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2e484 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24469504->24473599] (req 24469504->24473599) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.698907:0:5835:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:2.0:1789760739.703746:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973293 is committed 00002000:00010000:0.0:1789760739.703979:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24469504 -> 24473600 00010000:00010000:0.0:1789760739.703996:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2e484 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24469504->24473599] (req 24469504->24473599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.704014:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2e484 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24469504->24473599] (req 24469504->24473599) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.704028:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2e484 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24469504->24473599] (req 24469504->24473599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.704036:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760739.704040:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2e484 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24469504->24473599] (req 24469504->24473599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.704050:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2e484 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24469504->24473599] (req 24469504->24473599) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.704069:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2e484 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24469504->24473599] (req 24469504->24473599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.704076:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2e484 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24469504->24473599] (req 24469504->24473599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760739.704167:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febfb374380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512625280:12345-192.168.203.56@tcp:4:dd.0 Request processed in 40046us (40484us total) trans 4294973293 rc 0/0 00000100:00100000:0.0:1789760739.704177:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6004 00000100:00100000:0.0:1789760739.704187:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 719191076053 time 719228402080 state 0x2 ref 0 00000100:00100000:1.0:1789760739.707365:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760739.707826:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512625408 00000100:00100000:3.0:1789760739.707837:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512625408 00000100:00100000:3.0:1789760739.707890:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 719191076053 time 719228402080 state 0x0 ref 1 00000100:00100000:3.0:1789760739.707906:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 719228662496 time 719228402080 state 0x1 ref 1 00000100:00100000:3.0:1789760739.707972:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 719228662496 time 719272271062 state 0x0 ref 1 00000100:00100000:3.0:1789760739.707985:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febfbddc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512625408:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760739.708043:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24473600, mtime: 0x6aad94e3, atime: 0x0, ctime: 0x6aad94e3, blocks: 0xc007 00010000:00010000:3.0:1789760739.708052:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2e48b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.708067:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=719s v=(0 0) 00010000:00010000:3.0:1789760739.708072:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760739.708078:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2e48b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24473600->24477695] (req 24473600->24477695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.708092:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2e48b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24473600->24477695] (req 24473600->24477695) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760739.743412:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24473600 -> 24477696 00000001:00080000:0.0:1789760739.743417:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973294 is committed 00010000:00010000:3.0:1789760739.743433:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2e48b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24473600->24477695] (req 24473600->24477695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.743446:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2e48b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24473600->24477695] (req 24473600->24477695) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.743459:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2e48b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24473600->24477695] (req 24473600->24477695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.743466:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760739.743472:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2e48b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24473600->24477695] (req 24473600->24477695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.743481:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2e48b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24473600->24477695] (req 24473600->24477695) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.743506:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2e48b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24473600->24477695] (req 24473600->24477695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.743514:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2e48b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24473600->24477695] (req 24473600->24477695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760739.743599:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febfbddc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512625408:12345-192.168.203.56@tcp:4:dd.0 Request processed in 35612us (36250us total) trans 4294973294 rc 0/0 00000100:00100000:3.0:1789760739.743610:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6005 00000100:00100000:3.0:1789760739.743620:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 719228662496 time 719272271062 state 0x2 ref 0 00000100:00100000:2.0:1789760739.746467:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760739.746566:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512625536 00000100:00100000:0.0:1789760739.746576:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512625536 00000100:00100000:0.0:1789760739.746629:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 719228662496 time 719272271062 state 0x0 ref 1 00000100:00100000:0.0:1789760739.746644:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 719272531478 time 719272271062 state 0x1 ref 1 00000100:00100000:0.0:1789760739.746675:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 719272531478 time 719310975100 state 0x0 ref 1 00000100:00100000:0.0:1789760739.746688:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac531ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512625536:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760739.746883:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24477696, mtime: 0x6aad94e3, atime: 0x0, ctime: 0x6aad94e3, blocks: 0xc007 00010000:00010000:0.0:1789760739.746891:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2e492 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.746906:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=719s v=(0 0) 00010000:00010000:0.0:1789760739.746912:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760739.746918:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2e492 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24477696->24481791] (req 24477696->24481791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.746932:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2e492 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24477696->24481791] (req 24477696->24481791) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760739.773602:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973295 is committed 00002000:00010000:1.0:1789760739.773743:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24477696 -> 24481792 00010000:00010000:1.0:1789760739.773761:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2e492 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24477696->24481791] (req 24477696->24481791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.773774:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2e492 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24477696->24481791] (req 24477696->24481791) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.773783:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2e492 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24477696->24481791] (req 24477696->24481791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.773790:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760739.773795:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2e492 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24477696->24481791] (req 24477696->24481791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.773804:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2e492 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24477696->24481791] (req 24477696->24481791) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.773825:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2e492 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24477696->24481791] (req 24477696->24481791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.773833:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2e492 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24477696->24481791] (req 24477696->24481791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760739.773925:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac531ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512625536:12345-192.168.203.56@tcp:4:dd.0 Request processed in 27233us (27461us total) trans 4294973295 rc 0/0 00000100:00100000:1.0:1789760739.774002:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6006 00000100:00100000:1.0:1789760739.774011:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 719272531478 time 719310975100 state 0x2 ref 0 00000100:00100000:0.0:1789760739.776417:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760739.776868:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512625664 00000100:00100000:2.0:1789760739.776878:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512625664 00000100:00100000:2.0:1789760739.776933:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 719272531478 time 719310975100 state 0x0 ref 1 00000100:00100000:2.0:1789760739.776949:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 719311235516 time 719310975100 state 0x1 ref 1 00000100:00100000:2.0:1789760739.777014:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 719311235516 time 719341312658 state 0x0 ref 1 00000100:00100000:2.0:1789760739.777038:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4058380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512625664:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760739.777091:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24481792, mtime: 0x6aad94e3, atime: 0x0, ctime: 0x6aad94e3, blocks: 0xc007 00010000:00010000:2.0:1789760739.777100:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2e499 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.777116:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=719s v=(0 0) 00010000:00010000:2.0:1789760739.777120:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760739.777127:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2e499 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24481792->24485887] (req 24481792->24485887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.777141:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2e499 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24481792->24485887] (req 24481792->24485887) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760739.811470:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973296 is committed 00002000:00010000:0.0:1789760739.811587:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24481792 -> 24485888 00010000:00010000:0.0:1789760739.811603:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2e499 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24481792->24485887] (req 24481792->24485887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.811615:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2e499 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24481792->24485887] (req 24481792->24485887) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.811625:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2e499 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24481792->24485887] (req 24481792->24485887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.811631:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760739.811636:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2e499 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24481792->24485887] (req 24481792->24485887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.811646:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2e499 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24481792->24485887] (req 24481792->24485887) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.811663:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2e499 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24481792->24485887] (req 24481792->24485887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.811670:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2e499 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24481792->24485887] (req 24481792->24485887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760739.811949:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4058380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512625664:12345-192.168.203.56@tcp:4:dd.0 Request processed in 34908us (35534us total) trans 4294973296 rc 0/0 00000100:00100000:0.0:1789760739.811961:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6007 00000100:00100000:0.0:1789760739.811971:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 719311235516 time 719341312658 state 0x2 ref 0 00000100:00100000:3.0:1789760739.816685:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760739.818153:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512625792 00000100:00100000:0.0:1789760739.818166:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512625792 00000100:00100000:0.0:1789760739.818226:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 719311235516 time 719341312658 state 0x0 ref 1 00000100:00100000:0.0:1789760739.818242:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 719341573074 time 719341312658 state 0x1 ref 1 00000100:00100000:0.0:1789760739.818324:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 719341573074 time 719382622194 state 0x0 ref 1 00000100:00100000:0.0:1789760739.818338:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2747b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512625792:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760739.818396:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24485888, mtime: 0x6aad94e3, atime: 0x0, ctime: 0x6aad94e3, blocks: 0xc007 00010000:00010000:0.0:1789760739.818404:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2e4a0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.818418:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=719s v=(0 0) 00010000:00010000:0.0:1789760739.818423:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760739.818429:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2e4a0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24485888->24489983] (req 24485888->24489983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.818442:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2e4a0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24485888->24489983] (req 24485888->24489983) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760739.849612:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24485888 -> 24489984 00010000:00010000:0.0:1789760739.849631:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2e4a0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24485888->24489983] (req 24485888->24489983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.849643:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2e4a0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24485888->24489983] (req 24485888->24489983) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.849662:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2e4a0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24485888->24489983] (req 24485888->24489983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.849669:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760739.849674:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2e4a0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24485888->24489983] (req 24485888->24489983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.849683:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2e4a0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24485888->24489983] (req 24485888->24489983) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.849719:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2e4a0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24485888->24489983] (req 24485888->24489983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.849727:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2e4a0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24485888->24489983] (req 24485888->24489983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760739.849822:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2747b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2359:x1876699512625792:12345-192.168.203.56@tcp:4:dd.0 Request processed in 31482us (33145us total) trans 4294973297 rc 0/0 00000100:00100000:0.0:1789760739.849832:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6008 00000100:00100000:0.0:1789760739.849841:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 719341573074 time 719382622194 state 0x2 ref 0 00000001:00080000:1.0:1789760739.850644:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973297 is committed 00000100:00100000:3.0:1789760739.855411:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760739.855501:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512625920 00000100:00100000:1.0:1789760739.855512:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512625920 00000100:00100000:1.0:1789760739.855572:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 719341573074 time 719382622194 state 0x0 ref 1 00000100:00100000:1.0:1789760739.855586:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 719382882610 time 719382622194 state 0x1 ref 1 00000100:00100000:1.0:1789760739.855617:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 719382882610 time 719419917261 state 0x0 ref 1 00000100:00100000:1.0:1789760739.855629:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febfb89ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512625920:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760739.855681:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24489984, mtime: 0x6aad94e3, atime: 0x0, ctime: 0x6aad94e3, blocks: 0xc007 00010000:00010000:1.0:1789760739.855690:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2e4a7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.855737:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=719s v=(0 0) 00010000:00010000:1.0:1789760739.855741:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760739.855747:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2e4a7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24489984->24494079] (req 24489984->24494079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.855761:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2e4a7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24489984->24494079] (req 24489984->24494079) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760739.888302:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973298 is committed 00002000:00010000:1.0:1789760739.888429:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24489984 -> 24494080 00010000:00010000:1.0:1789760739.888447:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2e4a7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24489984->24494079] (req 24489984->24494079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.888460:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2e4a7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24489984->24494079] (req 24489984->24494079) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.888469:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2e4a7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24489984->24494079] (req 24489984->24494079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.888477:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760739.888482:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2e4a7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24489984->24494079] (req 24489984->24494079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.888491:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2e4a7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24489984->24494079] (req 24489984->24494079) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.888510:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2e4a7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24489984->24494079] (req 24489984->24494079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.888517:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2e4a7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24489984->24494079] (req 24489984->24494079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760739.888613:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febfb89ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512625920:12345-192.168.203.56@tcp:4:dd.0 Request processed in 32980us (33206us total) trans 4294973298 rc 0/0 00000100:00100000:1.0:1789760739.888623:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6009 00000100:00100000:1.0:1789760739.888634:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 719382882610 time 719419917261 state 0x2 ref 0 00000100:00100000:3.0:1789760739.892627:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760739.893877:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512626048 00000100:00100000:0.0:1789760739.893889:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512626048 00000100:00100000:1.0:1789760739.894011:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 719382882610 time 719419917261 state 0x0 ref 1 00000100:00100000:1.0:1789760739.894027:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 719420177677 time 719419917261 state 0x1 ref 1 00000100:00100000:1.0:1789760739.894063:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 719420177677 time 719458362294 state 0x0 ref 1 00000100:00100000:1.0:1789760739.894076:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febfb89fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512626048:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760739.894127:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24494080, mtime: 0x6aad94e3, atime: 0x0, ctime: 0x6aad94e3, blocks: 0xc007 00010000:00010000:1.0:1789760739.894136:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2e4ae lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.894151:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=719s v=(0 0) 00010000:00010000:1.0:1789760739.894155:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760739.894163:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2e4ae lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24494080->24498175] (req 24494080->24498175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.894176:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2e4ae lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24494080->24498175] (req 24494080->24498175) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760739.922514:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973299 is committed 00002000:00010000:1.0:1789760739.922778:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24494080 -> 24498176 00010000:00010000:1.0:1789760739.922795:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2e4ae lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24494080->24498175] (req 24494080->24498175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.922809:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2e4ae lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24494080->24498175] (req 24494080->24498175) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.922818:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2e4ae lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24494080->24498175] (req 24494080->24498175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.922826:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760739.922832:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2e4ae lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24494080->24498175] (req 24494080->24498175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.922842:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2e4ae lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24494080->24498175] (req 24494080->24498175) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.922861:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2e4ae lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24494080->24498175] (req 24494080->24498175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.922869:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2e4ae lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24494080->24498175] (req 24494080->24498175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760739.923037:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febfb89fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512626048:12345-192.168.203.56@tcp:4:dd.0 Request processed in 28959us (30414us total) trans 4294973299 rc 0/0 00000100:00100000:1.0:1789760739.923048:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6010 00000100:00100000:1.0:1789760739.923058:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 719420177677 time 719458362294 state 0x2 ref 0 00000100:00100000:3.0:1789760739.926944:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760739.927480:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512626176 00000100:00100000:0.0:1789760739.927491:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512626176 00000100:00100000:0.0:1789760739.927551:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 719420177677 time 719458362294 state 0x0 ref 1 00000100:00100000:0.0:1789760739.927568:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 719458622710 time 719458362294 state 0x1 ref 1 00000100:00100000:0.0:1789760739.927602:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 719458622710 time 719491901506 state 0x0 ref 1 00000100:00100000:0.0:1789760739.927615:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febea4b6850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512626176:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760739.927663:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24498176, mtime: 0x6aad94e3, atime: 0x0, ctime: 0x6aad94e3, blocks: 0xc007 00010000:00010000:0.0:1789760739.927672:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2e4b5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.927687:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=719s v=(0 0) 00010000:00010000:0.0:1789760739.927691:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760739.927735:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2e4b5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24498176->24502271] (req 24498176->24502271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.928117:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2e4b5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24498176->24502271] (req 24498176->24502271) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760739.959355:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973300 is committed 00002000:00010000:2.0:1789760739.959491:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24498176 -> 24502272 00010000:00010000:2.0:1789760739.959510:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2e4b5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24498176->24502271] (req 24498176->24502271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.959523:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2e4b5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24498176->24502271] (req 24498176->24502271) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.959539:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2e4b5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24498176->24502271] (req 24498176->24502271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.959547:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760739.959552:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2e4b5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24498176->24502271] (req 24498176->24502271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.959561:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2e4b5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24498176->24502271] (req 24498176->24502271) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.959583:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2e4b5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24498176->24502271] (req 24498176->24502271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.959591:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2e4b5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24498176->24502271] (req 24498176->24502271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760739.959817:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febea4b6850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512626176:12345-192.168.203.56@tcp:4:dd.0 Request processed in 32198us (32877us total) trans 4294973300 rc 0/0 00000100:00100000:2.0:1789760739.959830:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6011 00000100:00100000:2.0:1789760739.959842:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 719458622710 time 719491901506 state 0x2 ref 0 00000100:00100000:2.0:1789760739.965931:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760739.966035:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512626304 00000100:00100000:1.0:1789760739.966046:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512626304 00000100:00100000:1.0:1789760739.966102:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 719458622710 time 719491901506 state 0x0 ref 1 00000100:00100000:1.0:1789760739.966118:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 719492161922 time 719491901506 state 0x1 ref 1 00000100:00100000:1.0:1789760739.966144:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 719492161922 time 719530443618 state 0x0 ref 1 00000100:00100000:1.0:1789760739.966156:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac531ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512626304:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760739.966205:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24502272, mtime: 0x6aad94e3, atime: 0x0, ctime: 0x6aad94e3, blocks: 0xc007 00010000:00010000:1.0:1789760739.966214:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2e4bc lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.966230:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=719s v=(0 0) 00010000:00010000:1.0:1789760739.966234:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760739.966240:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2e4bc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24502272->24506367] (req 24502272->24506367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.966254:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2e4bc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24502272->24506367] (req 24502272->24506367) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760739.988379:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973301 is committed 00002000:00010000:0.0:1789760739.988555:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24502272 -> 24506368 00010000:00010000:0.0:1789760739.988572:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2e4bc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24502272->24506367] (req 24502272->24506367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.988585:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2e4bc lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24502272->24506367] (req 24502272->24506367) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.988594:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2e4bc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24502272->24506367] (req 24502272->24506367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.988603:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760739.988608:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2e4bc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24502272->24506367] (req 24502272->24506367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.988618:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2e4bc lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24502272->24506367] (req 24502272->24506367) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.988637:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2e4bc lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24502272->24506367] (req 24502272->24506367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.988645:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2e4bc lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24502272->24506367] (req 24502272->24506367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760739.988774:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac531ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512626304:12345-192.168.203.56@tcp:4:dd.0 Request processed in 22615us (22847us total) trans 4294973301 rc 0/0 00000100:00100000:0.0:1789760739.988785:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6012 00000100:00100000:0.0:1789760739.988794:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 719492161922 time 719530443618 state 0x2 ref 0 00000100:00100000:3.0:1789760739.991161:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760739.991237:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512626432 00000100:00100000:1.0:1789760739.991247:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512626432 00000100:00100000:1.0:1789760739.991298:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 719492161922 time 719530443618 state 0x0 ref 1 00000100:00100000:1.0:1789760739.991313:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 719530704034 time 719530443618 state 0x1 ref 1 00000100:00100000:1.0:1789760739.991338:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 719530704034 time 719555638170 state 0x0 ref 1 00000100:00100000:1.0:1789760739.991362:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febfb89ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512626432:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760739.991408:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24506368, mtime: 0x6aad94e3, atime: 0x0, ctime: 0x6aad94e3, blocks: 0xc007 00010000:00010000:1.0:1789760739.991417:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2e4c3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.991432:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=719s v=(0 0) 00010000:00010000:1.0:1789760739.991436:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760739.991442:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2e4c3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24506368->24510463] (req 24506368->24510463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.991456:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2e4c3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24506368->24510463] (req 24506368->24510463) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760740.029422:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973302 is committed 00002000:00010000:0.0:1789760740.029559:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24506368 -> 24510464 00010000:00010000:0.0:1789760740.029640:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2e4c3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24506368->24510463] (req 24506368->24510463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.029653:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2e4c3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24506368->24510463] (req 24506368->24510463) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.029661:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2e4c3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24506368->24510463] (req 24506368->24510463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.029668:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760740.029673:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2e4c3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24506368->24510463] (req 24506368->24510463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.029682:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2e4c3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24506368->24510463] (req 24506368->24510463) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.029774:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2e4c3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24506368->24510463] (req 24506368->24510463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.029783:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2e4c3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24506368->24510463] (req 24506368->24510463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760740.029916:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febfb89ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512626432:12345-192.168.203.56@tcp:4:dd.0 Request processed in 38562us (38760us total) trans 4294973302 rc 0/0 00000100:00100000:0.0:1789760740.029927:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6013 00000100:00100000:0.0:1789760740.029937:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 719530704034 time 719555638170 state 0x2 ref 0 00000100:00100000:3.0:1789760740.033965:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760740.034143:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512626560 00000100:00100000:1.0:1789760740.034153:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512626560 00000100:00100000:1.0:1789760740.034205:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 719530704034 time 719555638170 state 0x0 ref 1 00000100:00100000:1.0:1789760740.034221:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 719555898586 time 719555638170 state 0x1 ref 1 00000100:00100000:1.0:1789760740.034287:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 719555898586 time 719598586427 state 0x0 ref 1 00000100:00100000:1.0:1789760740.034300:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febfb89fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512626560:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760740.034358:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24510464, mtime: 0x6aad94e3, atime: 0x0, ctime: 0x6aad94e3, blocks: 0xc007 00010000:00010000:1.0:1789760740.034368:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2e4ca lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760740.034384:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=719s v=(0 0) 00010000:00010000:1.0:1789760740.034389:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760740.034404:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2e4ca lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24510464->24514559] (req 24510464->24514559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760740.034419:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2e4ca lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24510464->24514559] (req 24510464->24514559) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760740.071107:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24510464 -> 24514560 00010000:00010000:0.0:1789760740.071131:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2e4ca lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24510464->24514559] (req 24510464->24514559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.071145:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2e4ca lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24510464->24514559] (req 24510464->24514559) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.071155:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2e4ca lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24510464->24514559] (req 24510464->24514559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760740.071163:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973303 is committed 00010000:00010000:0.0:1789760740.071167:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760740.071173:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2e4ca lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24510464->24514559] (req 24510464->24514559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.071182:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2e4ca lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24510464->24514559] (req 24510464->24514559) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.071201:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2e4ca lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24510464->24514559] (req 24510464->24514559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.071210:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2e4ca lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24510464->24514559] (req 24510464->24514559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760740.071491:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febfb89fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512626560:12345-192.168.203.56@tcp:4:dd.0 Request processed in 36995us (37606us total) trans 4294973303 rc 0/0 00000100:00100000:0.0:1789760740.071501:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6014 00000100:00100000:0.0:1789760740.071511:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 719555898586 time 719598586427 state 0x2 ref 0 00000100:00100000:2.0:1789760740.077301:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760740.077383:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512626688 00000100:00100000:0.0:1789760740.077393:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512626688 00000100:00100000:0.0:1789760740.077455:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 719555898586 time 719598586427 state 0x0 ref 1 00000100:00100000:0.0:1789760740.077478:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 719598846843 time 719598586427 state 0x1 ref 1 00000100:00100000:0.0:1789760740.077538:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 719598846843 time 719641837373 state 0x0 ref 1 00000100:00100000:0.0:1789760740.077551:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac531e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512626688:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760740.077658:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24514560, mtime: 0x6aad94e3, atime: 0x0, ctime: 0x6aad94e3, blocks: 0xc007 00010000:00010000:0.0:1789760740.077668:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2e4d1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.077685:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=719s v=(0 0) 00010000:00010000:0.0:1789760740.077689:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760740.077729:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2e4d1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24514560->24518655] (req 24514560->24518655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.077744:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2e4d1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24514560->24518655] (req 24514560->24518655) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760740.112625:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973304 is committed 00002000:00010000:2.0:1789760740.112749:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24514560 -> 24518656 00010000:00010000:2.0:1789760740.112764:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2e4d1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24514560->24518655] (req 24514560->24518655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.112776:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2e4d1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24514560->24518655] (req 24514560->24518655) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.112786:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2e4d1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24514560->24518655] (req 24514560->24518655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.112793:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760740.112797:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2e4d1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24514560->24518655] (req 24514560->24518655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.112805:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2e4d1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24514560->24518655] (req 24514560->24518655) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.112824:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2e4d1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24514560->24518655] (req 24514560->24518655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.112831:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2e4d1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24514560->24518655] (req 24514560->24518655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760740.112922:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac531e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512626688:12345-192.168.203.56@tcp:4:dd.0 Request processed in 35368us (35623us total) trans 4294973304 rc 0/0 00000100:00100000:2.0:1789760740.112932:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6015 00000100:00100000:2.0:1789760740.112942:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 719598846843 time 719641837373 state 0x2 ref 0 00000100:00100000:3.0:1789760740.117459:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760740.117673:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512626816 00000100:00100000:0.0:1789760740.117683:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512626816 00000100:00100000:0.0:1789760740.117780:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 719598846843 time 719641837373 state 0x0 ref 1 00000100:00100000:0.0:1789760740.117794:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 719642097789 time 719641837373 state 0x1 ref 1 00000100:00100000:0.0:1789760740.117831:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 719642097789 time 719682130852 state 0x0 ref 1 00000100:00100000:0.0:1789760740.117844:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5305880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512626816:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760740.117889:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24518656, mtime: 0x6aad94e3, atime: 0x0, ctime: 0x6aad94e3, blocks: 0xc007 00010000:00010000:0.0:1789760740.117898:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2e4d8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.117930:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=719s v=(0 0) 00010000:00010000:0.0:1789760740.117935:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760740.117941:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2e4d8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24518656->24522751] (req 24518656->24522751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.117954:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2e4d8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24518656->24522751] (req 24518656->24522751) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760740.151743:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24518656 -> 24522752 00010000:00010000:1.0:1789760740.151763:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2e4d8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24518656->24522751] (req 24518656->24522751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760740.151776:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2e4d8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24518656->24522751] (req 24518656->24522751) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760740.151785:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2e4d8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24518656->24522751] (req 24518656->24522751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760740.151797:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760740.151802:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2e4d8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24518656->24522751] (req 24518656->24522751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760740.151820:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2e4d8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24518656->24522751] (req 24518656->24522751) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760740.151840:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2e4d8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24518656->24522751] (req 24518656->24522751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760740.151846:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2e4d8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24518656->24522751] (req 24518656->24522751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760740.151938:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5305880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2358:x1876699512626816:12345-192.168.203.56@tcp:4:dd.0 Request processed in 34091us (34485us total) trans 4294973305 rc 0/0 00000100:00100000:1.0:1789760740.151949:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6016 00000100:00100000:1.0:1789760740.151958:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 719642097789 time 719682130852 state 0x2 ref 0 00000001:00080000:1.0:1789760740.153289:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973305 is committed 00000100:00100000:3.0:1789760740.159215:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760740.159654:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512626944 00000100:00100000:0.0:1789760740.159665:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512626944 00000100:00100000:0.0:1789760740.159751:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 719642097789 time 719682130852 state 0x0 ref 1 00000100:00100000:0.0:1789760740.159766:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 719682391268 time 719682130852 state 0x1 ref 1 00000100:00100000:0.0:1789760740.159796:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 719682391268 time 719724095306 state 0x0 ref 1 00000100:00100000:0.0:1789760740.159932:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5304a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512626944:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760740.159980:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24522752, mtime: 0x6aad94e3, atime: 0x0, ctime: 0x6aad94e3, blocks: 0xc007 00010000:00010000:0.0:1789760740.159988:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2e4df lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.160003:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=719s v=(0 0) 00010000:00010000:0.0:1789760740.160007:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760740.160013:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2e4df lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24522752->24526847] (req 24522752->24526847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.160033:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2e4df lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24522752->24526847] (req 24522752->24526847) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760740.196057:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973306 is committed 00002000:00010000:0.0:1789760740.196231:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24522752 -> 24526848 00010000:00010000:0.0:1789760740.196248:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2e4df lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24522752->24526847] (req 24522752->24526847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.196261:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2e4df lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24522752->24526847] (req 24522752->24526847) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.196271:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2e4df lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24522752->24526847] (req 24522752->24526847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.196278:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760740.196283:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2e4df lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24522752->24526847] (req 24522752->24526847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.196291:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2e4df lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24522752->24526847] (req 24522752->24526847) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.196310:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2e4df lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24522752->24526847] (req 24522752->24526847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.196318:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2e4df lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24522752->24526847] (req 24522752->24526847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760740.196604:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5304a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512626944:12345-192.168.203.56@tcp:4:dd.0 Request processed in 36792us (37398us total) trans 4294973306 rc 0/0 00000100:00100000:0.0:1789760740.196616:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6017 00000100:00100000:0.0:1789760740.196625:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 719682391268 time 719724095306 state 0x2 ref 0 00000100:00100000:3.0:1789760740.200309:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760740.200843:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512627072 00000100:00100000:0.0:1789760740.200853:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512627072 00000100:00100000:0.0:1789760740.200904:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 719682391268 time 719724095306 state 0x0 ref 1 00000100:00100000:0.0:1789760740.200919:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 719724355722 time 719724095306 state 0x1 ref 1 00000100:00100000:0.0:1789760740.200949:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 719724355722 time 719765248606 state 0x0 ref 1 00000100:00100000:0.0:1789760740.200962:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5306d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512627072:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760740.201010:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24526848, mtime: 0x6aad94e3, atime: 0x0, ctime: 0x6aad94e3, blocks: 0xc007 00010000:00010000:0.0:1789760740.201020:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2e4e6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.201044:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=719s v=(0 0) 00010000:00010000:0.0:1789760740.201049:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760740.201055:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2e4e6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24526848->24530943] (req 24526848->24530943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.201071:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2e4e6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24526848->24530943] (req 24526848->24530943) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760740.238852:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973307 is committed 00002000:00010000:1.0:1789760740.239536:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24526848 -> 24530944 00010000:00010000:1.0:1789760740.239553:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2e4e6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24526848->24530943] (req 24526848->24530943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760740.239565:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2e4e6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24526848->24530943] (req 24526848->24530943) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760740.239579:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2e4e6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24526848->24530943] (req 24526848->24530943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760740.239586:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760740.239591:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2e4e6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24526848->24530943] (req 24526848->24530943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760740.239600:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2e4e6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24526848->24530943] (req 24526848->24530943) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760740.239618:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2e4e6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24526848->24530943] (req 24526848->24530943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760740.239625:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2e4e6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24526848->24530943] (req 24526848->24530943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760740.239923:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5306d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512627072:12345-192.168.203.56@tcp:4:dd.0 Request processed in 38958us (39617us total) trans 4294973307 rc 0/0 00000100:00100000:1.0:1789760740.239933:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6018 00000100:00100000:1.0:1789760740.239942:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 719724355722 time 719765248606 state 0x2 ref 0 00000100:00100000:3.0:1789760740.245199:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760740.245654:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512627200 00000100:00100000:1.0:1789760740.246316:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512627200 00000100:00100000:1.0:1789760740.246381:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 719724355722 time 719765248606 state 0x0 ref 1 00000100:00100000:1.0:1789760740.246397:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 719765509022 time 719765248606 state 0x1 ref 1 00000100:00100000:1.0:1789760740.246432:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 719765509022 time 719810731483 state 0x0 ref 1 00000100:00100000:1.0:1789760740.246445:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5306300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512627200:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760740.247053:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24530944, mtime: 0x6aad94e3, atime: 0x0, ctime: 0x6aad94e3, blocks: 0xc007 00010000:00010000:1.0:1789760740.247064:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2e4ed lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760740.247081:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=719s v=(0 0) 00010000:00010000:1.0:1789760740.247085:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760740.247093:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2e4ed lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24530944->24535039] (req 24530944->24535039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760740.247107:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2e4ed lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24530944->24535039] (req 24530944->24535039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760740.315562:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973308 is committed 00002000:00010000:3.0:1789760740.315662:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24530944 -> 24535040 00010000:00010000:3.0:1789760740.315677:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2e4ed lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24530944->24535039] (req 24530944->24535039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.315785:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2e4ed lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24530944->24535039] (req 24530944->24535039) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.315798:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2e4ed lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24530944->24535039] (req 24530944->24535039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.315893:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760740.315900:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2e4ed lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24530944->24535039] (req 24530944->24535039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.315911:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2e4ed lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24530944->24535039] (req 24530944->24535039) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.315931:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2e4ed lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24530944->24535039] (req 24530944->24535039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.315940:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2e4ed lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24530944->24535039] (req 24530944->24535039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760740.316331:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5306300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512627200:12345-192.168.203.56@tcp:4:dd.0 Request processed in 69884us (71137us total) trans 4294973308 rc 0/0 00000100:00100000:3.0:1789760740.316356:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6019 00000100:00100000:3.0:1789760740.316374:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 719765509022 time 719810731483 state 0x2 ref 0 00000100:00100000:3.0:1789760740.318923:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760740.319180:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512627328 00000100:00100000:0.0:1789760740.319189:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512627328 00000100:00100000:0.0:1789760740.319240:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 719765509022 time 719810731483 state 0x0 ref 1 00000100:00100000:0.0:1789760740.319256:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 719810991899 time 719810731483 state 0x1 ref 1 00000100:00100000:0.0:1789760740.319286:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 719810991899 time 719883585936 state 0x0 ref 1 00000100:00100000:0.0:1789760740.319297:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5307800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512627328:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760740.319354:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24535040, mtime: 0x6aad94e3, atime: 0x0, ctime: 0x6aad94e3, blocks: 0xc007 00010000:00010000:0.0:1789760740.319369:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2e4f4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.319385:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=719s v=(0 0) 00010000:00010000:0.0:1789760740.319390:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760740.319396:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2e4f4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24535040->24539135] (req 24535040->24539135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.319410:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2e4f4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24535040->24539135] (req 24535040->24539135) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760740.344552:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973309 is committed 00002000:00010000:3.0:1789760740.344946:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24535040 -> 24539136 00010000:00010000:3.0:1789760740.344961:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2e4f4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24535040->24539135] (req 24535040->24539135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.344972:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2e4f4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24535040->24539135] (req 24535040->24539135) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.344980:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2e4f4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24535040->24539135] (req 24535040->24539135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.344986:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760740.344991:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2e4f4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24535040->24539135] (req 24535040->24539135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.344999:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2e4f4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24535040->24539135] (req 24535040->24539135) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.345025:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2e4f4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24535040->24539135] (req 24535040->24539135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.345034:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2e4f4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24535040->24539135] (req 24535040->24539135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760740.345277:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5307800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512627328:12345-192.168.203.56@tcp:4:dd.0 Request processed in 25975us (26358us total) trans 4294973309 rc 0/0 00000100:00100000:3.0:1789760740.345287:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6020 00000100:00100000:3.0:1789760740.345296:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 719810991899 time 719883585936 state 0x2 ref 0 00000100:00100000:0.0:1789760740.348536:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760740.349234:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512627456 00000100:00100000:1.0:1789760740.349243:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512627456 00000100:00100000:0.0:1789760740.349375:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 719810991899 time 719883585936 state 0x0 ref 1 00000100:00100000:0.0:1789760740.349394:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 719883846352 time 719883585936 state 0x1 ref 1 00000100:00100000:0.0:1789760740.349452:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 719883846352 time 719913751296 state 0x0 ref 1 00000100:00100000:0.0:1789760740.349464:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4059180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512627456:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760740.349513:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24539136, mtime: 0x6aad94e3, atime: 0x0, ctime: 0x6aad94e3, blocks: 0xc007 00010000:00010000:0.0:1789760740.349522:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2e4fb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.349537:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=719s v=(0 0) 00010000:00010000:0.0:1789760740.349542:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760740.349547:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2e4fb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24539136->24543231] (req 24539136->24543231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.349561:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2e4fb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24539136->24543231] (req 24539136->24543231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760740.384248:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973310 is committed 00002000:00010000:0.0:1789760740.384364:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24539136 -> 24543232 00010000:00010000:0.0:1789760740.384378:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2e4fb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24539136->24543231] (req 24539136->24543231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.384391:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2e4fb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24539136->24543231] (req 24539136->24543231) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.384400:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2e4fb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24539136->24543231] (req 24539136->24543231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.384413:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760740.384418:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2e4fb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24539136->24543231] (req 24539136->24543231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.384427:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2e4fb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24539136->24543231] (req 24539136->24543231) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.384446:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2e4fb lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24539136->24543231] (req 24539136->24543231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.384454:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2e4fb lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24539136->24543231] (req 24539136->24543231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760740.384580:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4059180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512627456:12345-192.168.203.56@tcp:4:dd.0 Request processed in 35113us (36047us total) trans 4294973310 rc 0/0 00000100:00100000:0.0:1789760740.384592:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6021 00000100:00100000:0.0:1789760740.384604:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 719883846352 time 719913751296 state 0x2 ref 0 00000100:00100000:0.0:1789760740.387981:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760740.388092:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512627584 00000100:00100000:3.0:1789760740.388109:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512627584 00000100:00100000:3.0:1789760740.388163:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 719883846352 time 719913751296 state 0x0 ref 1 00000100:00100000:3.0:1789760740.388178:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 719914011712 time 719913751296 state 0x1 ref 1 00000100:00100000:3.0:1789760740.388208:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 719914011712 time 719952507847 state 0x0 ref 1 00000100:00100000:3.0:1789760740.388221:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4059500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512627584:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760740.388266:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24543232, mtime: 0x6aad94e3, atime: 0x0, ctime: 0x6aad94e3, blocks: 0xc007 00010000:00010000:3.0:1789760740.388274:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2e502 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.388290:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=719s v=(0 0) 00010000:00010000:3.0:1789760740.388294:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760740.388300:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2e502 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24543232->24547327] (req 24543232->24547327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.388312:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2e502 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24543232->24547327] (req 24543232->24547327) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760740.423749:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24543232 -> 24547328 00010000:00010000:3.0:1789760740.423768:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2e502 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24543232->24547327] (req 24543232->24547327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.423781:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2e502 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24543232->24547327] (req 24543232->24547327) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.423790:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2e502 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24543232->24547327] (req 24543232->24547327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.423798:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760740.423802:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2e502 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24543232->24547327] (req 24543232->24547327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.423810:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2e502 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24543232->24547327] (req 24543232->24547327) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.423828:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2e502 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24543232->24547327] (req 24543232->24547327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.423836:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2e502 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24543232->24547327] (req 24543232->24547327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760740.423927:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4059500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2361:x1876699512627584:12345-192.168.203.56@tcp:4:dd.0 Request processed in 35703us (35950us total) trans 4294973311 rc 0/0 00000100:00100000:3.0:1789760740.423941:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6022 00000100:00100000:3.0:1789760740.423950:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 719914011712 time 719952507847 state 0x2 ref 0 00000001:00080000:3.0:1789760740.423985:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973311 is committed 00000100:00100000:3.0:1789760740.430480:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760740.430646:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512627712 00000100:00100000:0.0:1789760740.430656:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512627712 00000100:00100000:0.0:1789760740.430796:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 719914011712 time 719952507847 state 0x0 ref 1 00000100:00100000:0.0:1789760740.430811:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 719952768263 time 719952507847 state 0x1 ref 1 00000100:00100000:0.0:1789760740.430875:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 719952768263 time 719995173942 state 0x0 ref 1 00000100:00100000:0.0:1789760740.430889:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5304000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512627712:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760740.430938:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24547328, mtime: 0x6aad94e3, atime: 0x0, ctime: 0x6aad94e3, blocks: 0xc007 00010000:00010000:0.0:1789760740.430947:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2e509 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.430963:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=719s v=(0 0) 00010000:00010000:0.0:1789760740.430968:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760740.430975:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2e509 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24547328->24551423] (req 24547328->24551423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.430989:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2e509 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24547328->24551423] (req 24547328->24551423) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760740.457125:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24547328 -> 24551424 00010000:00010000:0.0:1789760740.457143:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2e509 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24547328->24551423] (req 24547328->24551423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.457156:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2e509 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24547328->24551423] (req 24547328->24551423) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.457165:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2e509 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24547328->24551423] (req 24547328->24551423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.457172:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760740.457177:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2e509 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24547328->24551423] (req 24547328->24551423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.457187:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2e509 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24547328->24551423] (req 24547328->24551423) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.457208:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2e509 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24547328->24551423] (req 24547328->24551423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.457215:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2e509 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24547328->24551423] (req 24547328->24551423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760740.457301:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5304000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2360:x1876699512627712:12345-192.168.203.56@tcp:4:dd.0 Request processed in 26409us (26826us total) trans 4294973312 rc 0/0 00000100:00100000:0.0:1789760740.457310:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6023 00000100:00100000:0.0:1789760740.457319:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 719952768263 time 719995173942 state 0x2 ref 0 00000001:00080000:1.0:1789760740.457447:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973312 is committed 00000100:00100000:2.0:1789760740.459594:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760740.459912:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512627840 00000100:00100000:0.0:1789760740.459922:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512627840 00000100:00100000:0.0:1789760740.459971:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 719952768263 time 719995173942 state 0x0 ref 1 00000100:00100000:0.0:1789760740.459986:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 719995434358 time 719995173942 state 0x1 ref 1 00000100:00100000:0.0:1789760740.460100:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 719995434358 time 720024398594 state 0x0 ref 1 00000100:00100000:0.0:1789760740.460114:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac531dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512627840:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760740.460160:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24551424, mtime: 0x6aad94e3, atime: 0x0, ctime: 0x6aad94e3, blocks: 0xc007 00010000:00010000:0.0:1789760740.460168:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2e510 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.460185:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=720s v=(0 0) 00010000:00010000:0.0:1789760740.460190:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760740.460196:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2e510 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24551424->24555519] (req 24551424->24555519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.460211:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2e510 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24551424->24555519] (req 24551424->24555519) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760740.490860:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24551424 -> 24555520 00010000:00010000:0.0:1789760740.490881:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2e510 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24551424->24555519] (req 24551424->24555519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.490894:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2e510 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24551424->24555519] (req 24551424->24555519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.490904:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2e510 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24551424->24555519] (req 24551424->24555519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.490912:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760740.490917:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2e510 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24551424->24555519] (req 24551424->24555519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.490926:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2e510 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24551424->24555519] (req 24551424->24555519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.490945:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2e510 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24551424->24555519] (req 24551424->24555519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.490954:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2e510 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24551424->24555519] (req 24551424->24555519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760740.491054:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac531dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2361:x1876699512627840:12345-192.168.203.56@tcp:4:dd.0 Request processed in 30939us (31464us total) trans 4294973313 rc 0/0 00000100:00100000:0.0:1789760740.491065:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6024 00000100:00100000:0.0:1789760740.491076:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 719995434358 time 720024398594 state 0x2 ref 0 00000001:00080000:1.0:1789760740.491164:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973313 is committed 00000100:00100000:2.0:1789760740.495880:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760740.496108:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512627968 00000100:00100000:3.0:1789760740.496118:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512627968 00000100:00100000:3.0:1789760740.496175:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 719995434358 time 720024398594 state 0x0 ref 1 00000100:00100000:3.0:1789760740.496191:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 720024659010 time 720024398594 state 0x1 ref 1 00000100:00100000:3.0:1789760740.496234:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 720024659010 time 720060532859 state 0x0 ref 1 00000100:00100000:3.0:1789760740.496251:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac531ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512627968:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760740.496302:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24555520, mtime: 0x6aad94e3, atime: 0x0, ctime: 0x6aad94e3, blocks: 0xc007 00010000:00010000:3.0:1789760740.496312:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2e517 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.496327:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=720s v=(0 0) 00010000:00010000:3.0:1789760740.496332:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760740.496339:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2e517 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24555520->24559615] (req 24555520->24559615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.496367:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2e517 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24555520->24559615] (req 24555520->24559615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760740.524975:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973314 is committed 00002000:00010000:0.0:1789760740.525143:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24555520 -> 24559616 00002000:00010000:0.0:1789760740.525148:0:6710:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760739 -> 1789760740 00002000:00010000:0.0:1789760740.525151:0:6710:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760739 -> 1789760740 00010000:00010000:0.0:1789760740.525164:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2e517 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24555520->24559615] (req 24555520->24559615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.525178:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2e517 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24555520->24559615] (req 24555520->24559615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.525187:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2e517 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24555520->24559615] (req 24555520->24559615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.525194:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760740.525200:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2e517 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24555520->24559615] (req 24555520->24559615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.525210:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2e517 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24555520->24559615] (req 24555520->24559615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.525231:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2e517 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24555520->24559615] (req 24555520->24559615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.525238:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2e517 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24555520->24559615] (req 24555520->24559615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760740.525381:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac531ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512627968:12345-192.168.203.56@tcp:4:dd.0 Request processed in 29130us (29504us total) trans 4294973314 rc 0/0 00000100:00100000:0.0:1789760740.525392:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6025 00000100:00100000:0.0:1789760740.525402:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 720024659010 time 720060532859 state 0x2 ref 0 00000100:00100000:2.0:1789760740.528158:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760740.528577:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512628096 00000100:00100000:0.0:1789760740.528586:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512628096 00000100:00100000:0.0:1789760740.528638:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 720024659010 time 720060532859 state 0x0 ref 1 00000100:00100000:0.0:1789760740.528651:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 720060793275 time 720060532859 state 0x1 ref 1 00000100:00100000:0.0:1789760740.528773:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 720060793275 time 720093071426 state 0x0 ref 1 00000100:00100000:0.0:1789760740.528786:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac531f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512628096:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760740.528832:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24559616, mtime: 0x6aad94e4, atime: 0x0, ctime: 0x6aad94e4, blocks: 0xc007 00010000:00010000:0.0:1789760740.528842:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2e51e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.528857:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=720s v=(0 0) 00010000:00010000:0.0:1789760740.528861:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760740.528867:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2e51e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24559616->24563711] (req 24559616->24563711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.528880:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2e51e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24559616->24563711] (req 24559616->24563711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760740.556006:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24559616 -> 24563712 00010000:00010000:1.0:1789760740.556029:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2e51e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24559616->24563711] (req 24559616->24563711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760740.556031:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973315 is committed 00010000:00010000:1.0:1789760740.556041:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2e51e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24559616->24563711] (req 24559616->24563711) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760740.556056:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2e51e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24559616->24563711] (req 24559616->24563711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760740.556064:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760740.556069:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2e51e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24559616->24563711] (req 24559616->24563711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760740.556078:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2e51e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24559616->24563711] (req 24559616->24563711) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760740.556098:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2e51e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24559616->24563711] (req 24559616->24563711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760740.556106:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2e51e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24559616->24563711] (req 24559616->24563711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760740.556244:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac531f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512628096:12345-192.168.203.56@tcp:4:dd.0 Request processed in 27455us (28090us total) trans 4294973315 rc 0/0 00000100:00100000:1.0:1789760740.556254:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6026 00000100:00100000:1.0:1789760740.556264:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 720060793275 time 720093071426 state 0x2 ref 0 00000100:00100000:2.0:1789760740.559835:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760740.559966:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512628224 00000100:00100000:0.0:1789760740.559975:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512628224 00000100:00100000:0.0:1789760740.560032:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 720060793275 time 720093071426 state 0x0 ref 1 00000100:00100000:0.0:1789760740.560048:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 720093331842 time 720093071426 state 0x1 ref 1 00000100:00100000:0.0:1789760740.560083:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 720093331842 time 720124382419 state 0x0 ref 1 00000100:00100000:0.0:1789760740.560095:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac531df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512628224:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760740.560148:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24563712, mtime: 0x6aad94e4, atime: 0x0, ctime: 0x6aad94e4, blocks: 0xc007 00010000:00010000:0.0:1789760740.560157:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2e525 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.560172:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=720s v=(0 0) 00010000:00010000:0.0:1789760740.560177:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760740.560183:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2e525 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24563712->24567807] (req 24563712->24567807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.560197:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2e525 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24563712->24567807] (req 24563712->24567807) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760740.597457:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24563712 -> 24567808 00010000:00010000:1.0:1789760740.597478:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2e525 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24563712->24567807] (req 24563712->24567807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760740.597498:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2e525 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24563712->24567807] (req 24563712->24567807) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760740.597506:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2e525 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24563712->24567807] (req 24563712->24567807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760740.597514:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760740.597519:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2e525 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24563712->24567807] (req 24563712->24567807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760740.597529:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2e525 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24563712->24567807] (req 24563712->24567807) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760740.597546:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2e525 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24563712->24567807] (req 24563712->24567807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760740.597555:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2e525 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24563712->24567807] (req 24563712->24567807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760740.597652:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac531df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2360:x1876699512628224:12345-192.168.203.56@tcp:4:dd.0 Request processed in 37554us (37822us total) trans 4294973316 rc 0/0 00000100:00100000:1.0:1789760740.597662:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6027 00000100:00100000:1.0:1789760740.597673:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 720093331842 time 720124382419 state 0x2 ref 0 00000001:00080000:2.0:1789760740.598953:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973316 is committed 00000100:00100000:2.0:1789760740.605620:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760740.605910:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512628352 00000100:00100000:0.0:1789760740.605920:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512628352 00000100:00100000:0.0:1789760740.605972:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 720093331842 time 720124382419 state 0x0 ref 1 00000100:00100000:0.0:1789760740.605987:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 720124642835 time 720124382419 state 0x1 ref 1 00000100:00100000:0.0:1789760740.606069:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 720124642835 time 720170368035 state 0x0 ref 1 00000100:00100000:0.0:1789760740.606081:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac531c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512628352:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760740.606126:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24567808, mtime: 0x6aad94e4, atime: 0x0, ctime: 0x6aad94e4, blocks: 0xc007 00010000:00010000:0.0:1789760740.606135:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2e52c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.606150:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=720s v=(0 0) 00010000:00010000:0.0:1789760740.606154:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760740.606161:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2e52c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24567808->24571903] (req 24567808->24571903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.606174:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2e52c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24567808->24571903] (req 24567808->24571903) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760740.642767:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24567808 -> 24571904 00000001:00080000:2.0:1789760740.642768:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973317 is committed 00010000:00010000:1.0:1789760740.642836:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2e52c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24567808->24571903] (req 24567808->24571903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760740.642846:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2e52c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24567808->24571903] (req 24567808->24571903) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760740.642852:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2e52c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24567808->24571903] (req 24567808->24571903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760740.642857:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760740.642860:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2e52c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24567808->24571903] (req 24567808->24571903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760740.642866:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2e52c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24567808->24571903] (req 24567808->24571903) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760740.642881:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2e52c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24567808->24571903] (req 24567808->24571903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760740.642886:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2e52c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24567808->24571903] (req 24567808->24571903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760740.643043:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac531c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512628352:12345-192.168.203.56@tcp:4:dd.0 Request processed in 36958us (37428us total) trans 4294973317 rc 0/0 00000100:00100000:1.0:1789760740.643055:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6028 00000100:00100000:1.0:1789760740.643066:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 720124642835 time 720170368035 state 0x2 ref 0 00000100:00100000:3.0:1789760740.646272:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760740.646561:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512628480 00000100:00100000:1.0:1789760740.646571:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512628480 00000100:00100000:1.0:1789760740.646622:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 720124642835 time 720170368035 state 0x0 ref 1 00000100:00100000:1.0:1789760740.646638:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 720170628451 time 720170368035 state 0x1 ref 1 00000100:00100000:1.0:1789760740.646668:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 720170628451 time 720210967858 state 0x0 ref 1 00000100:00100000:1.0:1789760740.646681:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5304380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512628480:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760740.646759:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24571904, mtime: 0x6aad94e4, atime: 0x0, ctime: 0x6aad94e4, blocks: 0xc007 00010000:00010000:1.0:1789760740.646768:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2e533 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760740.646785:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=720s v=(0 0) 00010000:00010000:1.0:1789760740.646789:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760740.646796:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2e533 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24571904->24575999] (req 24571904->24575999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760740.646810:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2e533 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24571904->24575999] (req 24571904->24575999) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760740.678520:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24571904 -> 24576000 00010000:00010000:2.0:1789760740.678541:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2e533 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24571904->24575999] (req 24571904->24575999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.678555:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2e533 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24571904->24575999] (req 24571904->24575999) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.678564:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2e533 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24571904->24575999] (req 24571904->24575999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.678571:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760740.678576:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2e533 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24571904->24575999] (req 24571904->24575999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.678585:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2e533 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24571904->24575999] (req 24571904->24575999) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.678610:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2e533 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24571904->24575999] (req 24571904->24575999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.678618:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2e533 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24571904->24575999] (req 24571904->24575999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760740.678744:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5304380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2360:x1876699512628480:12345-192.168.203.56@tcp:4:dd.0 Request processed in 32060us (32476us total) trans 4294973318 rc 0/0 00000100:00100000:2.0:1789760740.678754:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6029 00000100:00100000:2.0:1789760740.678763:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 720170628451 time 720210967858 state 0x2 ref 0 00000001:00080000:2.0:1789760740.678801:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973318 is committed 00000100:00100000:2.0:1789760740.682352:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760740.683239:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512628608 00000100:00100000:1.0:1789760740.683250:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512628608 00000100:00100000:1.0:1789760740.683307:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 720170628451 time 720210967858 state 0x0 ref 1 00000100:00100000:1.0:1789760740.683323:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 720211228274 time 720210967858 state 0x1 ref 1 00000100:00100000:1.0:1789760740.683402:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 720211228274 time 720247700868 state 0x0 ref 1 00000100:00100000:1.0:1789760740.683416:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac531fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512628608:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760740.683473:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24576000, mtime: 0x6aad94e4, atime: 0x0, ctime: 0x6aad94e4, blocks: 0xc007 00010000:00010000:1.0:1789760740.683482:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2e53a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760740.683499:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=720s v=(0 0) 00010000:00010000:1.0:1789760740.683503:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760740.683510:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2e53a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24576000->24580095] (req 24576000->24580095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760740.683525:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2e53a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24576000->24580095] (req 24576000->24580095) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760740.716579:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24576000 -> 24580096 00010000:00010000:2.0:1789760740.716601:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2e53a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24576000->24580095] (req 24576000->24580095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.716615:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2e53a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24576000->24580095] (req 24576000->24580095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.716625:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2e53a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24576000->24580095] (req 24576000->24580095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.716632:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760740.716637:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2e53a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24576000->24580095] (req 24576000->24580095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.716646:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2e53a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24576000->24580095] (req 24576000->24580095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760740.716647:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973319 is committed 00010000:00010000:2.0:1789760740.716664:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2e53a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24576000->24580095] (req 24576000->24580095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.716673:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2e53a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24576000->24580095] (req 24576000->24580095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760740.716973:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac531fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512628608:12345-192.168.203.56@tcp:4:dd.0 Request processed in 33556us (34636us total) trans 4294973319 rc 0/0 00000100:00100000:2.0:1789760740.716985:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6030 00000100:00100000:2.0:1789760740.716995:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 720211228274 time 720247700868 state 0x2 ref 0 00000100:00100000:0.0:1789760740.720309:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760740.720778:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512628736 00000100:00100000:1.0:1789760740.720789:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512628736 00000100:00100000:1.0:1789760740.720841:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 720211228274 time 720247700868 state 0x0 ref 1 00000100:00100000:1.0:1789760740.720857:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 720247961284 time 720247700868 state 0x1 ref 1 00000100:00100000:1.0:1789760740.720915:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 720247961284 time 720285213535 state 0x0 ref 1 00000100:00100000:1.0:1789760740.720927:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4058a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512628736:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760740.720974:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24580096, mtime: 0x6aad94e4, atime: 0x0, ctime: 0x6aad94e4, blocks: 0xc007 00010000:00010000:1.0:1789760740.720983:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2e541 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760740.720998:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=720s v=(0 0) 00010000:00010000:1.0:1789760740.721002:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760740.721008:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2e541 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24580096->24584191] (req 24580096->24584191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760740.721030:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2e541 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24580096->24584191] (req 24580096->24584191) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760740.726653:0:7647:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:3.0:1789760740.754567:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973320 is committed 00002000:00010000:2.0:1789760740.754653:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24580096 -> 24584192 00010000:00010000:2.0:1789760740.754923:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2e541 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24580096->24584191] (req 24580096->24584191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.754936:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2e541 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24580096->24584191] (req 24580096->24584191) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.754945:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2e541 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24580096->24584191] (req 24580096->24584191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.754952:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760740.754957:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2e541 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24580096->24584191] (req 24580096->24584191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.754966:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2e541 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24580096->24584191] (req 24580096->24584191) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.754984:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2e541 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24580096->24584191] (req 24580096->24584191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.754991:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2e541 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24580096->24584191] (req 24580096->24584191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760740.755231:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4058a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512628736:12345-192.168.203.56@tcp:4:dd.0 Request processed in 34302us (35010us total) trans 4294973320 rc 0/0 00000100:00100000:2.0:1789760740.755242:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6031 00000100:00100000:2.0:1789760740.755251:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 720247961284 time 720285213535 state 0x2 ref 0 00000100:00100000:3.0:1789760740.759904:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760740.760144:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512628864 00000100:00100000:2.0:1789760740.760154:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512628864 00000100:00100000:2.0:1789760740.760204:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 720247961284 time 720285213535 state 0x0 ref 1 00000100:00100000:2.0:1789760740.760219:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 720285473951 time 720285213535 state 0x1 ref 1 00000100:00100000:2.0:1789760740.760246:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 720285473951 time 720324545585 state 0x0 ref 1 00000100:00100000:2.0:1789760740.760262:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5306680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512628864:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760740.760310:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24584192, mtime: 0x6aad94e4, atime: 0x0, ctime: 0x6aad94e4, blocks: 0xc007 00010000:00010000:2.0:1789760740.760319:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2e548 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.760334:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=720s v=(0 0) 00010000:00010000:2.0:1789760740.760351:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760740.760358:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2e548 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24584192->24588287] (req 24584192->24588287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.760372:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2e548 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24584192->24588287] (req 24584192->24588287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760740.794661:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973321 is committed 00002000:00010000:2.0:1789760740.794820:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24584192 -> 24588288 00010000:00010000:2.0:1789760740.794835:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2e548 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24584192->24588287] (req 24584192->24588287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.794848:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2e548 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24584192->24588287] (req 24584192->24588287) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.795074:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2e548 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24584192->24588287] (req 24584192->24588287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.795084:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760740.795089:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2e548 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24584192->24588287] (req 24584192->24588287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.795099:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2e548 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24584192->24588287] (req 24584192->24588287) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.795124:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2e548 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24584192->24588287] (req 24584192->24588287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.795131:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2e548 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24584192->24588287] (req 24584192->24588287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760740.795255:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5306680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512628864:12345-192.168.203.56@tcp:4:dd.0 Request processed in 34994us (35360us total) trans 4294973321 rc 0/0 00000100:00100000:2.0:1789760740.795266:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6032 00000100:00100000:2.0:1789760740.795275:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 720285473951 time 720324545585 state 0x2 ref 0 00000100:00100000:0.0:1789760740.799179:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760740.799571:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512628992 00000100:00100000:2.0:1789760740.799581:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512628992 00000100:00100000:2.0:1789760740.799636:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 720285473951 time 720324545585 state 0x0 ref 1 00000100:00100000:2.0:1789760740.799651:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 720324806001 time 720324545585 state 0x1 ref 1 00000100:00100000:2.0:1789760740.799899:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 720324806001 time 720364197381 state 0x0 ref 1 00000100:00100000:2.0:1789760740.799912:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac405ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512628992:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760740.799965:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24588288, mtime: 0x6aad94e4, atime: 0x0, ctime: 0x6aad94e4, blocks: 0xc007 00010000:00010000:2.0:1789760740.799974:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2e54f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.799989:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=720s v=(0 0) 00010000:00010000:2.0:1789760740.799994:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760740.800001:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2e54f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24588288->24592383] (req 24588288->24592383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.800025:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2e54f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24588288->24592383] (req 24588288->24592383) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760740.840223:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973322 is committed 00002000:00010000:2.0:1789760740.840396:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24588288 -> 24592384 00010000:00010000:2.0:1789760740.840412:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2e54f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24588288->24592383] (req 24588288->24592383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.840423:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2e54f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24588288->24592383] (req 24588288->24592383) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.840441:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2e54f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24588288->24592383] (req 24588288->24592383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.840448:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760740.840453:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2e54f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24588288->24592383] (req 24588288->24592383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.840461:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2e54f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24588288->24592383] (req 24588288->24592383) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.840479:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2e54f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24588288->24592383] (req 24588288->24592383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.840492:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2e54f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24588288->24592383] (req 24588288->24592383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760740.840617:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac405ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512628992:12345-192.168.203.56@tcp:4:dd.0 Request processed in 40702us (41444us total) trans 4294973322 rc 0/0 00000100:00100000:2.0:1789760740.840629:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6033 00000100:00100000:2.0:1789760740.840638:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 720324806001 time 720364197381 state 0x2 ref 0 00000100:00100000:0.0:1789760740.844500:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760740.844565:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512629120 00000100:00100000:2.0:1789760740.844575:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512629120 00000100:00100000:2.0:1789760740.844625:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 720324806001 time 720364197381 state 0x0 ref 1 00000100:00100000:2.0:1789760740.844639:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 720364457797 time 720364197381 state 0x1 ref 1 00000100:00100000:2.0:1789760740.844736:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 720364457797 time 720409035235 state 0x0 ref 1 00000100:00100000:2.0:1789760740.844750:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4058e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512629120:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760740.844796:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24592384, mtime: 0x6aad94e4, atime: 0x0, ctime: 0x6aad94e4, blocks: 0xc007 00010000:00010000:2.0:1789760740.844805:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2e556 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.844820:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=720s v=(0 0) 00010000:00010000:2.0:1789760740.844825:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760740.844831:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2e556 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24592384->24596479] (req 24592384->24596479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.844849:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2e556 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24592384->24596479] (req 24592384->24596479) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760740.877325:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973323 is committed 00002000:00010000:3.0:1789760740.877422:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24592384 -> 24596480 00010000:00010000:3.0:1789760740.877438:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2e556 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24592384->24596479] (req 24592384->24596479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.877519:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2e556 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24592384->24596479] (req 24592384->24596479) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.877528:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2e556 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24592384->24596479] (req 24592384->24596479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.877535:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760740.877539:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2e556 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24592384->24596479] (req 24592384->24596479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.877548:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2e556 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24592384->24596479] (req 24592384->24596479) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.877567:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2e556 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24592384->24596479] (req 24592384->24596479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.877574:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2e556 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24592384->24596479] (req 24592384->24596479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760740.877740:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4058e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512629120:12345-192.168.203.56@tcp:4:dd.0 Request processed in 32948us (33204us total) trans 4294973323 rc 0/0 00000100:00100000:3.0:1789760740.877750:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6034 00000100:00100000:3.0:1789760740.877760:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 720364457797 time 720409035235 state 0x2 ref 0 00000100:00100000:0.0:1789760740.882303:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760740.882400:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512629248 00000100:00100000:2.0:1789760740.882410:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512629248 00000100:00100000:2.0:1789760740.882477:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 720364457797 time 720409035235 state 0x0 ref 1 00000100:00100000:2.0:1789760740.882493:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 720409295651 time 720409035235 state 0x1 ref 1 00000100:00100000:2.0:1789760740.882553:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 720409295651 time 720446851911 state 0x0 ref 1 00000100:00100000:2.0:1789760740.882566:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4058380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512629248:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760740.882614:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24596480, mtime: 0x6aad94e4, atime: 0x0, ctime: 0x6aad94e4, blocks: 0xc007 00010000:00010000:2.0:1789760740.882623:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2e55d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.882639:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=720s v=(0 0) 00010000:00010000:2.0:1789760740.882643:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760740.882650:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2e55d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24596480->24600575] (req 24596480->24600575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.882664:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2e55d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24596480->24600575] (req 24596480->24600575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760740.908028:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24596480 -> 24600576 00010000:00010000:2.0:1789760740.908049:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2e55d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24596480->24600575] (req 24596480->24600575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.908061:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2e55d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24596480->24600575] (req 24596480->24600575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.908070:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2e55d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24596480->24600575] (req 24596480->24600575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.908077:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760740.908081:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2e55d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24596480->24600575] (req 24596480->24600575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.908091:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2e55d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24596480->24600575] (req 24596480->24600575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.908108:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2e55d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24596480->24600575] (req 24596480->24600575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.908115:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2e55d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24596480->24600575] (req 24596480->24600575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760740.908194:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4058380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2359:x1876699512629248:12345-192.168.203.56@tcp:4:dd.0 Request processed in 25626us (25896us total) trans 4294973324 rc 0/0 00000100:00100000:2.0:1789760740.908204:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6035 00000100:00100000:2.0:1789760740.908218:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 720409295651 time 720446851911 state 0x2 ref 0 00000001:00080000:3.0:1789760740.908444:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973324 is committed 00000100:00100000:0.0:1789760740.911256:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760740.911329:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512629376 00000100:00100000:2.0:1789760740.911349:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512629376 00000100:00100000:2.0:1789760740.911400:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 720409295651 time 720446851911 state 0x0 ref 1 00000100:00100000:2.0:1789760740.911415:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 720447112327 time 720446851911 state 0x1 ref 1 00000100:00100000:2.0:1789760740.911451:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 720447112327 time 720475751528 state 0x0 ref 1 00000100:00100000:2.0:1789760740.911470:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4217100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512629376:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760740.911516:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24600576, mtime: 0x6aad94e4, atime: 0x0, ctime: 0x6aad94e4, blocks: 0xc007 00010000:00010000:2.0:1789760740.911525:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2e564 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.911541:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=720s v=(0 0) 00010000:00010000:2.0:1789760740.911545:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760740.911551:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2e564 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24600576->24604671] (req 24600576->24604671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.911564:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2e564 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24600576->24604671] (req 24600576->24604671) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760740.940444:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973325 is committed 00002000:00010000:2.0:1789760740.940662:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24600576 -> 24604672 00010000:00010000:2.0:1789760740.940680:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2e564 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24600576->24604671] (req 24600576->24604671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.940692:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2e564 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24600576->24604671] (req 24600576->24604671) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.940736:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2e564 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24600576->24604671] (req 24600576->24604671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.940743:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760740.940748:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2e564 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24600576->24604671] (req 24600576->24604671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.940756:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2e564 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24600576->24604671] (req 24600576->24604671) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.940781:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2e564 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24600576->24604671] (req 24600576->24604671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.940789:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2e564 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24600576->24604671] (req 24600576->24604671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760740.941056:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4217100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512629376:12345-192.168.203.56@tcp:4:dd.0 Request processed in 29589us (29800us total) trans 4294973325 rc 0/0 00000100:00100000:2.0:1789760740.941067:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6036 00000100:00100000:2.0:1789760740.941076:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 720447112327 time 720475751528 state 0x2 ref 0 00000100:00100000:0.0:1789760740.943791:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760740.944018:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512629504 00000100:00100000:3.0:1789760740.944028:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512629504 00000100:00100000:3.0:1789760740.944081:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 720447112327 time 720475751528 state 0x0 ref 1 00000100:00100000:3.0:1789760740.944150:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 720476011944 time 720475751528 state 0x1 ref 1 00000100:00100000:3.0:1789760740.944195:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 720476011944 time 720508484052 state 0x0 ref 1 00000100:00100000:3.0:1789760740.944209:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac3302680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512629504:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760740.944261:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24604672, mtime: 0x6aad94e4, atime: 0x0, ctime: 0x6aad94e4, blocks: 0xc007 00010000:00010000:3.0:1789760740.944270:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2e56b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.944286:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=720s v=(0 0) 00010000:00010000:3.0:1789760740.944290:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760740.944296:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2e56b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24604672->24608767] (req 24604672->24608767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.944315:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2e56b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24604672->24608767] (req 24604672->24608767) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760740.989950:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973326 is committed 00002000:00010000:2.0:1789760740.990078:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24604672 -> 24608768 00010000:00010000:2.0:1789760740.990094:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2e56b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24604672->24608767] (req 24604672->24608767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.990282:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2e56b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24604672->24608767] (req 24604672->24608767) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.990291:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2e56b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24604672->24608767] (req 24604672->24608767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.990297:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760740.990302:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2e56b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24604672->24608767] (req 24604672->24608767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.990312:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2e56b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24604672->24608767] (req 24604672->24608767) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.990334:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2e56b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24604672->24608767] (req 24604672->24608767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.990352:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2e56b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24604672->24608767] (req 24604672->24608767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760740.990455:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac3302680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512629504:12345-192.168.203.56@tcp:4:dd.0 Request processed in 46243us (46665us total) trans 4294973326 rc 0/0 00000100:00100000:2.0:1789760740.990466:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6037 00000100:00100000:2.0:1789760740.990476:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 720476011944 time 720508484052 state 0x2 ref 0 00000100:00100000:0.0:1789760740.994090:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760740.996993:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512629632 00000100:00100000:2.0:1789760740.997008:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512629632 00000100:00100000:2.0:1789760740.997073:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 720476011944 time 720508484052 state 0x0 ref 1 00000100:00100000:2.0:1789760740.997091:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 720508744468 time 720508484052 state 0x1 ref 1 00000100:00100000:2.0:1789760740.997179:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 720508744468 time 720561477371 state 0x0 ref 1 00000100:00100000:2.0:1789760740.997193:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac3300e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512629632:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760740.997240:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24608768, mtime: 0x6aad94e4, atime: 0x0, ctime: 0x6aad94e4, blocks: 0xc007 00010000:00010000:2.0:1789760740.997249:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2e572 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.997265:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=720s v=(0 0) 00010000:00010000:2.0:1789760740.997270:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760740.997277:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2e572 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24608768->24612863] (req 24608768->24612863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.997290:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2e572 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24608768->24612863] (req 24608768->24612863) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760741.030720:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24608768 -> 24612864 00010000:00010000:2.0:1789760741.030741:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2e572 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24608768->24612863] (req 24608768->24612863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760741.030747:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973327 is committed 00010000:00010000:2.0:1789760741.030753:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2e572 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24608768->24612863] (req 24608768->24612863) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.030768:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2e572 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24608768->24612863] (req 24608768->24612863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.030776:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760741.030781:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2e572 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24608768->24612863] (req 24608768->24612863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.030790:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2e572 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24608768->24612863] (req 24608768->24612863) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.030810:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2e572 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24608768->24612863] (req 24608768->24612863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.030818:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2e572 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24608768->24612863] (req 24608768->24612863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760741.030944:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac3300e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512629632:12345-192.168.203.56@tcp:4:dd.0 Request processed in 33749us (36861us total) trans 4294973327 rc 0/0 00000100:00100000:2.0:1789760741.030954:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6038 00000100:00100000:2.0:1789760741.030964:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 720508744468 time 720561477371 state 0x2 ref 0 00000100:00100000:0.0:1789760741.034005:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760741.034156:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512629760 00000100:00100000:2.0:1789760741.034168:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512629760 00000100:00100000:2.0:1789760741.034237:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 720508744468 time 720561477371 state 0x0 ref 1 00000100:00100000:2.0:1789760741.034253:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 720561737787 time 720561477371 state 0x1 ref 1 00000100:00100000:2.0:1789760741.034319:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 720561737787 time 720598618274 state 0x0 ref 1 00000100:00100000:2.0:1789760741.034333:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac3301c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512629760:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760741.034390:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24612864, mtime: 0x6aad94e4, atime: 0x0, ctime: 0x6aad94e4, blocks: 0xc007 00010000:00010000:2.0:1789760741.034398:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2e579 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.034414:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=720s v=(0 0) 00010000:00010000:2.0:1789760741.034418:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760741.034424:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2e579 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24612864->24616959] (req 24612864->24616959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.034448:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2e579 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24612864->24616959] (req 24612864->24616959) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1789760741.035204:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:2.0:1789760741.074824:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973328 is committed 00002000:00010000:0.0:1789760741.075006:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24612864 -> 24616960 00010000:00010000:0.0:1789760741.075031:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2e579 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24612864->24616959] (req 24612864->24616959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.075044:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2e579 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24612864->24616959] (req 24612864->24616959) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.075053:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2e579 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24612864->24616959] (req 24612864->24616959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.075061:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760741.075066:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2e579 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24612864->24616959] (req 24612864->24616959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.075074:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2e579 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24612864->24616959] (req 24612864->24616959) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.075095:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2e579 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24612864->24616959] (req 24612864->24616959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.075108:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2e579 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24612864->24616959] (req 24612864->24616959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760741.075212:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac3301c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512629760:12345-192.168.203.56@tcp:4:dd.0 Request processed in 40874us (41210us total) trans 4294973328 rc 0/0 00000100:00100000:0.0:1789760741.075222:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6039 00000100:00100000:0.0:1789760741.075232:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 720561737787 time 720598618274 state 0x2 ref 0 00000100:00100000:0.0:1789760741.077832:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760741.078255:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512629888 00000100:00100000:2.0:1789760741.078266:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512629888 00000100:00100000:2.0:1789760741.078317:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 720561737787 time 720598618274 state 0x0 ref 1 00000100:00100000:2.0:1789760741.078332:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 720598878690 time 720598618274 state 0x1 ref 1 00000100:00100000:2.0:1789760741.078370:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 720598878690 time 720642669792 state 0x0 ref 1 00000100:00100000:2.0:1789760741.078383:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac38b3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512629888:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760741.078447:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24616960, mtime: 0x6aad94e4, atime: 0x0, ctime: 0x6aad94e4, blocks: 0xc007 00010000:00010000:2.0:1789760741.078459:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2e580 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.078474:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=720s v=(0 0) 00010000:00010000:2.0:1789760741.078479:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760741.078486:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2e580 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24616960->24621055] (req 24616960->24621055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.078499:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2e580 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24616960->24621055] (req 24616960->24621055) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760741.132595:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973329 is committed 00002000:00010000:2.0:1789760741.132689:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24616960 -> 24621056 00010000:00010000:2.0:1789760741.132778:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2e580 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24616960->24621055] (req 24616960->24621055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.132791:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2e580 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24616960->24621055] (req 24616960->24621055) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.132800:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2e580 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24616960->24621055] (req 24616960->24621055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.132808:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760741.132813:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2e580 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24616960->24621055] (req 24616960->24621055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.132823:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2e580 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24616960->24621055] (req 24616960->24621055) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.132848:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2e580 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24616960->24621055] (req 24616960->24621055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.132855:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2e580 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24616960->24621055] (req 24616960->24621055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760741.133907:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac38b3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512629888:12345-192.168.203.56@tcp:4:dd.0 Request processed in 54623us (55178us total) trans 4294973329 rc 0/0 00000100:00100000:3.0:1789760741.133924:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6040 00000100:00100000:3.0:1789760741.133936:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 720598878690 time 720642669792 state 0x2 ref 0 00000100:00100000:3.0:1789760741.136581:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760741.136733:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512630016 00000100:00100000:2.0:1789760741.136744:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512630016 00000100:00100000:2.0:1789760741.136795:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 720598878690 time 720642669792 state 0x0 ref 1 00000100:00100000:2.0:1789760741.136811:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 720642930208 time 720642669792 state 0x1 ref 1 00000100:00100000:2.0:1789760741.136832:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 720642930208 time 720701132267 state 0x0 ref 1 00000100:00100000:2.0:1789760741.136844:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5307b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512630016:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760741.136895:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24621056, mtime: 0x6aad94e4, atime: 0x0, ctime: 0x6aad94e4, blocks: 0xc007 00010000:00010000:2.0:1789760741.136905:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2e587 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.136919:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=720s v=(0 0) 00010000:00010000:2.0:1789760741.136924:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760741.136930:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2e587 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24621056->24625151] (req 24621056->24625151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.136944:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2e587 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24621056->24625151] (req 24621056->24625151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760741.182460:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24621056 -> 24625152 00010000:00010000:3.0:1789760741.182482:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2e587 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24621056->24625151] (req 24621056->24625151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.182495:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2e587 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24621056->24625151] (req 24621056->24625151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.182504:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2e587 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24621056->24625151] (req 24621056->24625151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.182513:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760741.182518:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2e587 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24621056->24625151] (req 24621056->24625151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.182531:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2e587 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24621056->24625151] (req 24621056->24625151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.182553:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2e587 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24621056->24625151] (req 24621056->24625151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.182560:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2e587 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24621056->24625151] (req 24621056->24625151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760741.182660:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5307b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2358:x1876699512630016:12345-192.168.203.56@tcp:4:dd.0 Request processed in 45813us (46082us total) trans 4294973330 rc 0/0 00000100:00100000:3.0:1789760741.182671:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6041 00000100:00100000:3.0:1789760741.182681:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 720642930208 time 720701132267 state 0x2 ref 0 00000001:00080000:2.0:1789760741.183950:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973330 is committed 00000100:00100000:0.0:1789760741.185327:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760741.185432:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512630144 00000100:00100000:2.0:1789760741.185441:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512630144 00000100:00100000:2.0:1789760741.185496:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 720642930208 time 720701132267 state 0x0 ref 1 00000100:00100000:2.0:1789760741.185513:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 720701392683 time 720701132267 state 0x1 ref 1 00000100:00100000:2.0:1789760741.185544:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 720701392683 time 720749843524 state 0x0 ref 1 00000100:00100000:2.0:1789760741.185557:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac38b2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512630144:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760741.185606:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24625152, mtime: 0x6aad94e4, atime: 0x0, ctime: 0x6aad94e4, blocks: 0xc007 00010000:00010000:2.0:1789760741.185614:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2e58e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.185630:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=720s v=(0 0) 00010000:00010000:2.0:1789760741.185634:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760741.185640:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2e58e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24625152->24629247] (req 24625152->24629247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.185652:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2e58e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24625152->24629247] (req 24625152->24629247) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760741.232257:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973331 is committed 00002000:00010000:0.0:1789760741.232443:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24625152 -> 24629248 00010000:00010000:0.0:1789760741.232460:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2e58e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24625152->24629247] (req 24625152->24629247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.232474:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2e58e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24625152->24629247] (req 24625152->24629247) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.232484:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2e58e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24625152->24629247] (req 24625152->24629247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.232492:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760741.232498:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2e58e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24625152->24629247] (req 24625152->24629247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.232506:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2e58e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24625152->24629247] (req 24625152->24629247) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.232526:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2e58e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24625152->24629247] (req 24625152->24629247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.232535:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2e58e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24625152->24629247] (req 24625152->24629247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760741.232632:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac38b2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512630144:12345-192.168.203.56@tcp:4:dd.0 Request processed in 47072us (47308us total) trans 4294973331 rc 0/0 00000100:00100000:0.0:1789760741.232645:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6042 00000100:00100000:0.0:1789760741.232656:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 720701392683 time 720749843524 state 0x2 ref 0 00000100:00100000:0.0:1789760741.237573:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760741.238844:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512630272 00000100:00100000:3.0:1789760741.238857:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512630272 00000100:00100000:2.0:1789760741.238973:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 720701392683 time 720749843524 state 0x0 ref 1 00000100:00100000:2.0:1789760741.238990:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 720750103940 time 720749843524 state 0x1 ref 1 00000100:00100000:2.0:1789760741.239025:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 720750103940 time 720803324796 state 0x0 ref 1 00000100:00100000:2.0:1789760741.239038:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac38b1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512630272:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760741.239085:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24629248, mtime: 0x6aad94e4, atime: 0x0, ctime: 0x6aad94e4, blocks: 0xc007 00010000:00010000:2.0:1789760741.239095:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2e595 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.239110:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=720s v=(0 0) 00010000:00010000:2.0:1789760741.239114:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760741.239121:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2e595 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24629248->24633343] (req 24629248->24633343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.239134:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2e595 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24629248->24633343] (req 24629248->24633343) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760741.300232:0:3310:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8febfbddc000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3310:1876699540223616:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1789760741.300308:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760741.300355:0:6707:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540223616 00000100:00100000:2.0:1789760741.300365:0:6707:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540223616 00000100:00100000:1.0:1789760741.300381:0:3310:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8feac437f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3310:1876699540224128:0@lo:400:kworker.0 00000100:00100000:1.0:1789760741.300414:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760741.300435:0:6707:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 196 00000100:00100000:2.0:1789760741.300447:0:6707:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4074a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3310:x1876699540223616:12345-0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1789760741.300454:0:3310:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1789760741.300462:0:3310:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8feac437fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3310:1876699540224000:0@lo:400:kworker.0 00002000:00000024:2.0:1789760741.300472:0:6707:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 528023552 pending 1703936 free 3833593856 avail 3811573760 00000100:00100000:1.0:1789760741.300492:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760741.300531:0:6707:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4074a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3310:x1876699540223616:12345-0@lo:13:osp-pre-0-0.0 Request processed in 85us (222us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760741.300539:0:6707:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 196 00000100:00100000:1.0:1789760741.300546:0:3310:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8febfbddc000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3310:1876699540223616:0@lo:13:osp-pre-0-0.0 00000100:00080000:2.0:1789760741.300569:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1789760741.300579:0:3311:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8feac437e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:6df98d54-eb52-4c5c-8fc9-ee9d74f90f98:3311:1876699540223744:0@lo:400:kworker.0 00000100:00100000:1.0:1789760741.300615:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760741.300649:0:3311:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8feac437d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3311:1876699540224256:0@lo:400:kworker.0 00000100:00100000:1.0:1789760741.300672:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760741.300728:0:3311:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [3->2] 00000100:00100000:2.0:1789760741.300731:0:5842:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540223744 00000100:00100000:1.0:1789760741.300736:0:3311:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8feac437df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3311:1876699540223872:0@lo:400:kworker.0 00000100:00100000:2.0:1789760741.300738:0:5842:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540223744 00000100:00100000:2.0:1789760741.300773:0:5842:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 397 00000100:00100000:2.0:1789760741.300782:0:5842:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febe0500450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:6df98d54-eb52-4c5c-8fc9-ee9d74f90f98+19:3311:x1876699540223744:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760741.300795:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760741.300828:0:5842:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febe0500450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:6df98d54-eb52-4c5c-8fc9-ee9d74f90f98+19:3311:x1876699540223744:12345-0@lo:400:kworker.0 Request processed in 47us (212us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760741.300835:0:5842:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 397 00000100:00100000:1.0:1789760741.300857:0:3311:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8feac4074700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3311:1876699540224384:0@lo:400:kworker.0 00000100:00100000:1.0:1789760741.300884:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760741.300909:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540224000 00000100:00100000:0.0:1789760741.300916:0:5847:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540224000 00000100:00100000:1.0:1789760741.300917:0:3311:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8feac437e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:6df98d54-eb52-4c5c-8fc9-ee9d74f90f98:3311:1876699540223744:0@lo:400:kworker.0 00000100:00100000:1.0:1789760741.300953:0:6704:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540224128 00000100:00100000:0.0:1789760741.300958:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540224256 00000100:00100000:1.0:1789760741.300962:0:6704:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540224128 00000100:00100000:0.0:1789760741.300964:0:5847:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540224256 00000100:00100000:0.0:1789760741.300988:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540223872 00000100:00100000:0.0:1789760741.300991:0:5847:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540223872 00000100:00100000:1.0:1789760741.300999:0:6704:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540224384 00000100:00100000:1.0:1789760741.301004:0:6704:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540224384 00000100:00100000:0.0:1789760741.301011:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 471 00000100:00100000:0.0:1789760741.301022:0:5847:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2fbb800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3310:x1876699540224000:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760741.301044:0:6704:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 181 00000100:00100000:1.0:1789760741.301054:0:6704:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febfc975180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3310:x1876699540224128:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760741.301074:0:5847:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2fbb800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3310:x1876699540224000:12345-0@lo:400:kworker.0 Request processed in 54us (581us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760741.301081:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 471 00000100:00100000:1.0:1789760741.301100:0:6704:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febfc975180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3310:x1876699540224128:12345-0@lo:400:kworker.0 Request processed in 46us (683us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760741.301101:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 472 00000100:00100000:1.0:1789760741.301110:0:6704:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 181 00000100:00100000:0.0:1789760741.301127:0:5847:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2fb8e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3311:x1876699540224256:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760741.301128:0:6704:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 182 00000100:00100000:1.0:1789760741.301140:0:6704:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2fbaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3311:x1876699540224384:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760741.301174:0:5847:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2fb8e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3311:x1876699540224256:12345-0@lo:400:kworker.0 Request processed in 50us (499us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760741.301184:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 472 00000100:00100000:1.0:1789760741.301185:0:6704:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2fbaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3311:x1876699540224384:12345-0@lo:400:kworker.0 Request processed in 45us (298us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760741.301194:0:6704:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 182 00000100:00100000:0.0:1789760741.301201:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 473 00000100:00100000:0.0:1789760741.301209:0:5847:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2fb9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3311:x1876699540223872:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760741.301247:0:5847:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2fb9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3311:x1876699540223872:12345-0@lo:400:kworker.0 Request processed in 39us (460us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760741.301251:0:3311:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8feac437df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3311:1876699540223872:0@lo:400:kworker.0 00000100:00100000:0.0:1789760741.301258:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 473 00000100:00100000:1.0:1789760741.301270:0:3311:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8feac4074700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3311:1876699540224384:0@lo:400:kworker.0 00000100:00080000:0.0:1789760741.301281:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1789760741.301283:0:3311:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8feac437d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3311:1876699540224256:0@lo:400:kworker.0 00000100:00080000:0.0:1789760741.301289:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1789760741.301318:0:3310:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8feac437fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3310:1876699540224000:0@lo:400:kworker.0 00000100:00100000:1.0:1789760741.301329:0:3310:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8feac437f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3310:1876699540224128:0@lo:400:kworker.0 00000001:00080000:2.0:1789760741.305994:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973332 is committed 00002000:00010000:0.0:1789760741.306229:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24629248 -> 24633344 00010000:00010000:0.0:1789760741.306245:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2e595 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24629248->24633343] (req 24629248->24633343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.306257:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2e595 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24629248->24633343] (req 24629248->24633343) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.306266:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2e595 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24629248->24633343] (req 24629248->24633343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.306273:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760741.306278:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2e595 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24629248->24633343] (req 24629248->24633343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.306287:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2e595 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24629248->24633343] (req 24629248->24633343) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.306306:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2e595 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24629248->24633343] (req 24629248->24633343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.306315:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2e595 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24629248->24633343] (req 24629248->24633343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760741.306414:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac38b1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512630272:12345-192.168.203.56@tcp:4:dd.0 Request processed in 67373us (68843us total) trans 4294973332 rc 0/0 00000100:00100000:0.0:1789760741.306425:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6043 00000100:00100000:0.0:1789760741.306434:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 720750103940 time 720803324796 state 0x2 ref 0 00000100:00100000:3.0:1789760741.313692:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760741.313822:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512630400 00000100:00100000:0.0:1789760741.313833:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512630400 00000100:00100000:0.0:1789760741.313887:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 720750103940 time 720803324796 state 0x0 ref 1 00000100:00100000:0.0:1789760741.313904:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 720803585212 time 720803324796 state 0x1 ref 1 00000100:00100000:0.0:1789760741.313940:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 720803585212 time 720878239872 state 0x0 ref 1 00000100:00100000:0.0:1789760741.313953:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5307100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512630400:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760741.314002:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24633344, mtime: 0x6aad94e4, atime: 0x0, ctime: 0x6aad94e4, blocks: 0xc007 00010000:00010000:0.0:1789760741.314013:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2e59c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.314029:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=720s v=(0 0) 00010000:00010000:0.0:1789760741.314034:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760741.314040:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2e59c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24633344->24637439] (req 24633344->24637439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.314060:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2e59c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24633344->24637439] (req 24633344->24637439) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760741.391567:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973333 is committed 00002000:00010000:0.0:1789760741.391793:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24633344 -> 24637440 00010000:00010000:0.0:1789760741.391808:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2e59c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24633344->24637439] (req 24633344->24637439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.391821:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2e59c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24633344->24637439] (req 24633344->24637439) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.391829:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2e59c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24633344->24637439] (req 24633344->24637439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.391836:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760741.391841:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2e59c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24633344->24637439] (req 24633344->24637439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.391849:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2e59c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24633344->24637439] (req 24633344->24637439) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.391869:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2e59c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24633344->24637439] (req 24633344->24637439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.391877:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2e59c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24633344->24637439] (req 24633344->24637439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760741.392039:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5307100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512630400:12345-192.168.203.56@tcp:4:dd.0 Request processed in 78083us (78351us total) trans 4294973333 rc 0/0 00000100:00100000:0.0:1789760741.392058:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6044 00000100:00100000:0.0:1789760741.392068:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 720803585212 time 720878239872 state 0x2 ref 0 00000100:00100000:2.0:1789760741.400422:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760741.400508:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512630528 00000100:00100000:0.0:1789760741.400518:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512630528 00000100:00100000:0.0:1789760741.400571:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 720803585212 time 720878239872 state 0x0 ref 1 00000100:00100000:0.0:1789760741.400586:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 720878500288 time 720878239872 state 0x1 ref 1 00000100:00100000:0.0:1789760741.400619:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 720878500288 time 720964918254 state 0x0 ref 1 00000100:00100000:0.0:1789760741.400631:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac531f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512630528:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760741.400681:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24637440, mtime: 0x6aad94e4, atime: 0x0, ctime: 0x6aad94e4, blocks: 0xc007 00010000:00010000:0.0:1789760741.400691:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2e5a3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.400753:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=720s v=(0 0) 00010000:00010000:0.0:1789760741.400757:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760741.400763:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2e5a3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24637440->24641535] (req 24637440->24641535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.400778:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2e5a3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24637440->24641535] (req 24637440->24641535) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760741.447658:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973334 is committed 00002000:00010000:2.0:1789760741.447848:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24637440 -> 24641536 00010000:00010000:2.0:1789760741.447864:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2e5a3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24637440->24641535] (req 24637440->24641535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.447877:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2e5a3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24637440->24641535] (req 24637440->24641535) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.447886:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2e5a3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24637440->24641535] (req 24637440->24641535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.447893:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760741.447898:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2e5a3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24637440->24641535] (req 24637440->24641535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.447907:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2e5a3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24637440->24641535] (req 24637440->24641535) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.447927:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2e5a3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24637440->24641535] (req 24637440->24641535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.447940:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2e5a3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24637440->24641535] (req 24637440->24641535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760741.448040:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac531f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512630528:12345-192.168.203.56@tcp:4:dd.0 Request processed in 47405us (47620us total) trans 4294973334 rc 0/0 00000100:00100000:2.0:1789760741.448256:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6045 00000100:00100000:2.0:1789760741.448269:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 720878500288 time 720964918254 state 0x2 ref 0 00000100:00100000:1.0:1789760741.451901:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760741.456261:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512630656 00000100:00100000:0.0:1789760741.456275:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512630656 00000100:00100000:1.0:1789760741.460236:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 720878500288 time 720964918254 state 0x0 ref 1 00000100:00100000:1.0:1789760741.460258:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 720965178670 time 720964918254 state 0x1 ref 1 00000100:00100000:1.0:1789760741.460301:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 720965178670 time 721024600286 state 0x0 ref 1 00000100:00100000:1.0:1789760741.460317:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2fb8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512630656:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760741.460378:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24641536, mtime: 0x6aad94e4, atime: 0x0, ctime: 0x6aad94e4, blocks: 0xc007 00010000:00010000:1.0:1789760741.460387:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2e5aa lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760741.460402:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=721s v=(0 0) 00010000:00010000:1.0:1789760741.460407:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760741.460413:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2e5aa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24641536->24645631] (req 24641536->24645631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760741.460426:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2e5aa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24641536->24645631] (req 24641536->24645631) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760741.500410:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973335 is committed 00002000:00010000:2.0:1789760741.500484:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24641536 -> 24645632 00010000:00010000:2.0:1789760741.500501:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2e5aa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24641536->24645631] (req 24641536->24645631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.500514:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2e5aa lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24641536->24645631] (req 24641536->24645631) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.500523:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2e5aa lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24641536->24645631] (req 24641536->24645631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.500530:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760741.500535:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2e5aa lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24641536->24645631] (req 24641536->24645631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.500544:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2e5aa lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24641536->24645631] (req 24641536->24645631) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.500569:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2e5aa lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24641536->24645631] (req 24641536->24645631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.500577:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2e5aa lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24641536->24645631] (req 24641536->24645631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760741.500732:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2fb8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512630656:12345-192.168.203.56@tcp:4:dd.0 Request processed in 40414us (48834us total) trans 4294973335 rc 0/0 00000100:00100000:2.0:1789760741.500743:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6046 00000100:00100000:2.0:1789760741.500753:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 720965178670 time 721024600286 state 0x2 ref 0 00000100:00100000:0.0:1789760741.503999:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760741.504153:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512630784 00000100:00100000:3.0:1789760741.504163:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512630784 00000100:00100000:3.0:1789760741.504213:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 720965178670 time 721024600286 state 0x0 ref 1 00000100:00100000:3.0:1789760741.504228:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 721024860702 time 721024600286 state 0x1 ref 1 00000100:00100000:3.0:1789760741.504258:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 721024860702 time 721068558152 state 0x0 ref 1 00000100:00100000:3.0:1789760741.504272:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac38b0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512630784:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760741.504320:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24645632, mtime: 0x6aad94e4, atime: 0x0, ctime: 0x6aad94e4, blocks: 0xc007 00010000:00010000:3.0:1789760741.504328:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2e5b1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.504352:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=721s v=(0 0) 00010000:00010000:3.0:1789760741.504357:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760741.504363:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2e5b1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24645632->24649727] (req 24645632->24649727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.504377:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2e5b1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24645632->24649727] (req 24645632->24649727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760741.531811:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973336 is committed 00002000:00010000:3.0:1789760741.531917:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24645632 -> 24649728 00002000:00010000:3.0:1789760741.531922:0:6708:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760740 -> 1789760741 00002000:00010000:3.0:1789760741.531925:0:6708:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760740 -> 1789760741 00010000:00010000:3.0:1789760741.531939:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2e5b1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24645632->24649727] (req 24645632->24649727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.531952:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2e5b1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24645632->24649727] (req 24645632->24649727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.531961:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2e5b1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24645632->24649727] (req 24645632->24649727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.531968:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760741.531974:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2e5b1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24645632->24649727] (req 24645632->24649727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.531983:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2e5b1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24645632->24649727] (req 24645632->24649727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.532002:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2e5b1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24645632->24649727] (req 24645632->24649727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.532009:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2e5b1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24645632->24649727] (req 24645632->24649727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760741.532182:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac38b0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512630784:12345-192.168.203.56@tcp:4:dd.0 Request processed in 27907us (28186us total) trans 4294973336 rc 0/0 00000100:00100000:3.0:1789760741.532193:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6047 00000100:00100000:3.0:1789760741.532202:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 721024860702 time 721068558152 state 0x2 ref 0 00000100:00100000:0.0:1789760741.536141:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760741.536254:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512630912 00000100:00100000:2.0:1789760741.536264:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512630912 00000100:00100000:2.0:1789760741.536316:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 721024860702 time 721068558152 state 0x0 ref 1 00000100:00100000:2.0:1789760741.536331:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 721068818568 time 721068558152 state 0x1 ref 1 00000100:00100000:2.0:1789760741.536406:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 721068818568 time 721100705478 state 0x0 ref 1 00000100:00100000:2.0:1789760741.536420:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febfb374380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512630912:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760741.536471:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24649728, mtime: 0x6aad94e5, atime: 0x0, ctime: 0x6aad94e5, blocks: 0xc007 00010000:00010000:2.0:1789760741.536480:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2e5b8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.536502:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=721s v=(0 0) 00010000:00010000:2.0:1789760741.536506:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760741.536512:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2e5b8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24649728->24653823] (req 24649728->24653823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.536524:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2e5b8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24649728->24653823] (req 24649728->24653823) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760741.564583:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973337 is committed 00002000:00010000:2.0:1789760741.564720:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24649728 -> 24653824 00010000:00010000:2.0:1789760741.564747:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2e5b8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24649728->24653823] (req 24649728->24653823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.564761:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2e5b8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24649728->24653823] (req 24649728->24653823) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.564888:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2e5b8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24649728->24653823] (req 24649728->24653823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.564897:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760741.564902:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2e5b8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24649728->24653823] (req 24649728->24653823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.564912:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2e5b8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24649728->24653823] (req 24649728->24653823) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.564933:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2e5b8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24649728->24653823] (req 24649728->24653823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.564942:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2e5b8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24649728->24653823] (req 24649728->24653823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760741.565042:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febfb374380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512630912:12345-192.168.203.56@tcp:4:dd.0 Request processed in 28619us (28908us total) trans 4294973337 rc 0/0 00000100:00100000:2.0:1789760741.565125:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6048 00000100:00100000:2.0:1789760741.565136:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 721068818568 time 721100705478 state 0x2 ref 0 00000100:00100000:0.0:1789760741.569171:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760741.569252:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512631040 00000100:00100000:2.0:1789760741.569262:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512631040 00000100:00100000:2.0:1789760741.569329:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 721068818568 time 721100705478 state 0x0 ref 1 00000100:00100000:2.0:1789760741.569356:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 721100965894 time 721100705478 state 0x1 ref 1 00000100:00100000:2.0:1789760741.569388:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 721100965894 time 721133687354 state 0x0 ref 1 00000100:00100000:2.0:1789760741.569402:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2747800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512631040:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760741.569456:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24653824, mtime: 0x6aad94e5, atime: 0x0, ctime: 0x6aad94e5, blocks: 0xc007 00010000:00010000:2.0:1789760741.569465:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2e5bf lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.569480:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=721s v=(0 0) 00010000:00010000:2.0:1789760741.569484:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760741.569490:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2e5bf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24653824->24657919] (req 24653824->24657919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.569510:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2e5bf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24653824->24657919] (req 24653824->24657919) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760741.606647:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973338 is committed 00002000:00010000:2.0:1789760741.606899:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24653824 -> 24657920 00010000:00010000:2.0:1789760741.606916:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2e5bf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24653824->24657919] (req 24653824->24657919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.606928:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2e5bf lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24653824->24657919] (req 24653824->24657919) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.606938:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2e5bf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24653824->24657919] (req 24653824->24657919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.606945:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760741.606950:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2e5bf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24653824->24657919] (req 24653824->24657919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.606958:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2e5bf lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24653824->24657919] (req 24653824->24657919) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.606978:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2e5bf lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24653824->24657919] (req 24653824->24657919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.606986:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2e5bf lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24653824->24657919] (req 24653824->24657919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760741.607162:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2747800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512631040:12345-192.168.203.56@tcp:4:dd.0 Request processed in 37758us (37995us total) trans 4294973338 rc 0/0 00000100:00100000:2.0:1789760741.607172:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6049 00000100:00100000:2.0:1789760741.607182:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 721100965894 time 721133687354 state 0x2 ref 0 00000100:00100000:0.0:1789760741.609865:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760741.610074:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512631168 00000100:00100000:3.0:1789760741.610082:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512631168 00000100:00100000:3.0:1789760741.610138:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 721100965894 time 721133687354 state 0x0 ref 1 00000100:00100000:3.0:1789760741.610154:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 721133947770 time 721133687354 state 0x1 ref 1 00000100:00100000:3.0:1789760741.610191:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 721133947770 time 721174491174 state 0x0 ref 1 00000100:00100000:3.0:1789760741.610204:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2745500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512631168:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760741.610251:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24657920, mtime: 0x6aad94e5, atime: 0x0, ctime: 0x6aad94e5, blocks: 0xc007 00010000:00010000:3.0:1789760741.610261:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2e5c6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.610276:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=721s v=(0 0) 00010000:00010000:3.0:1789760741.610280:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760741.610286:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2e5c6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24657920->24662015] (req 24657920->24662015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.610298:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2e5c6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24657920->24662015] (req 24657920->24662015) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760741.646631:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973339 is committed 00002000:00010000:3.0:1789760741.646752:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24657920 -> 24662016 00010000:00010000:3.0:1789760741.646768:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2e5c6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24657920->24662015] (req 24657920->24662015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.646781:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2e5c6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24657920->24662015] (req 24657920->24662015) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.646790:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2e5c6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24657920->24662015] (req 24657920->24662015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.646797:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760741.646801:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2e5c6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24657920->24662015] (req 24657920->24662015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.646811:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2e5c6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24657920->24662015] (req 24657920->24662015) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.646835:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2e5c6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24657920->24662015] (req 24657920->24662015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.646842:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2e5c6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24657920->24662015] (req 24657920->24662015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760741.646942:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2745500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512631168:12345-192.168.203.56@tcp:4:dd.0 Request processed in 36736us (37082us total) trans 4294973339 rc 0/0 00000100:00100000:3.0:1789760741.646951:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6050 00000100:00100000:3.0:1789760741.646960:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 721133947770 time 721174491174 state 0x2 ref 0 00000100:00100000:3.0:1789760741.650164:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760741.650259:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512631296 00000100:00100000:2.0:1789760741.650269:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512631296 00000100:00100000:2.0:1789760741.650331:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 721133947770 time 721174491174 state 0x0 ref 1 00000100:00100000:2.0:1789760741.650358:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 721174751590 time 721174491174 state 0x1 ref 1 00000100:00100000:2.0:1789760741.650395:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 721174751590 time 721214694167 state 0x0 ref 1 00000100:00100000:2.0:1789760741.650408:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5305f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512631296:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760741.650459:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24662016, mtime: 0x6aad94e5, atime: 0x0, ctime: 0x6aad94e5, blocks: 0xc007 00010000:00010000:2.0:1789760741.650468:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2e5cd lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.650486:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=721s v=(0 0) 00010000:00010000:2.0:1789760741.650491:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760741.650497:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2e5cd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24662016->24666111] (req 24662016->24666111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.650509:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2e5cd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24662016->24666111] (req 24662016->24666111) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760741.682603:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973340 is committed 00002000:00010000:2.0:1789760741.682744:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24662016 -> 24666112 00010000:00010000:2.0:1789760741.682760:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2e5cd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24662016->24666111] (req 24662016->24666111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.682772:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2e5cd lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24662016->24666111] (req 24662016->24666111) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.682780:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2e5cd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24662016->24666111] (req 24662016->24666111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.682787:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760741.682792:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2e5cd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24662016->24666111] (req 24662016->24666111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.682801:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2e5cd lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24662016->24666111] (req 24662016->24666111) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.682820:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2e5cd lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24662016->24666111] (req 24662016->24666111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.682827:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2e5cd lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24662016->24666111] (req 24662016->24666111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760741.683123:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5305f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512631296:12345-192.168.203.56@tcp:4:dd.0 Request processed in 32712us (32966us total) trans 4294973340 rc 0/0 00000100:00100000:2.0:1789760741.683139:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6051 00000100:00100000:2.0:1789760741.683148:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 721174751590 time 721214694167 state 0x2 ref 0 00000100:00100000:1.0:1789760741.687081:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760741.687479:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512631424 00000100:00100000:2.0:1789760741.687488:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512631424 00000100:00100000:2.0:1789760741.687541:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 721174751590 time 721214694167 state 0x0 ref 1 00000100:00100000:2.0:1789760741.687556:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 721214954583 time 721214694167 state 0x1 ref 1 00000100:00100000:2.0:1789760741.687619:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 721214954583 time 721251919204 state 0x0 ref 1 00000100:00100000:2.0:1789760741.687631:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2fba680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512631424:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760741.687679:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24666112, mtime: 0x6aad94e5, atime: 0x0, ctime: 0x6aad94e5, blocks: 0xc007 00010000:00010000:2.0:1789760741.687688:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2e5d4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.687757:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=721s v=(0 0) 00010000:00010000:2.0:1789760741.687762:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760741.687769:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2e5d4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24666112->24670207] (req 24666112->24670207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.687782:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2e5d4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24666112->24670207] (req 24666112->24670207) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760741.726356:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24666112 -> 24670208 00010000:00010000:2.0:1789760741.726376:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2e5d4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24666112->24670207] (req 24666112->24670207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.726389:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2e5d4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24666112->24670207] (req 24666112->24670207) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.726398:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2e5d4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24666112->24670207] (req 24666112->24670207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.726406:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760741.726411:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2e5d4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24666112->24670207] (req 24666112->24670207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.726419:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2e5d4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24666112->24670207] (req 24666112->24670207) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.726439:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2e5d4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24666112->24670207] (req 24666112->24670207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.726447:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2e5d4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24666112->24670207] (req 24666112->24670207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760741.726545:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2fba680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2361:x1876699512631424:12345-192.168.203.56@tcp:4:dd.0 Request processed in 38910us (39467us total) trans 4294973341 rc 0/0 00000100:00100000:2.0:1789760741.726554:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6052 00000100:00100000:2.0:1789760741.726564:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 721214954583 time 721251919204 state 0x2 ref 0 00000100:00100000:2.0:1789760741.729081:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760741.729187:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512631552 00000100:00100000:3.0:1789760741.729196:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512631552 00000100:00100000:3.0:1789760741.729246:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 721214954583 time 721251919204 state 0x0 ref 1 00000100:00100000:3.0:1789760741.729259:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 721252179620 time 721251919204 state 0x1 ref 1 00000100:00100000:3.0:1789760741.729279:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 721252179620 time 721293579831 state 0x0 ref 1 00000100:00100000:3.0:1789760741.729291:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac531c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2360:x1876699512631552:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760741.729346:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24670208, mtime: 0x6aad94e5, atime: 0x0, ctime: 0x6aad94e5, blocks: 0xc007 00010000:00010000:3.0:1789760741.729354:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2e5db lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.729374:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=721s v=(0 0) 00010000:00010000:3.0:1789760741.729378:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760741.729385:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2e5db lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24670208->24674303] (req 24670208->24674303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.729397:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2e5db lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24670208->24674303] (req 24670208->24674303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760741.729910:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973341 is committed 00010000:00010000:2.0:1789760741.751298:0:5836:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:0.0:1789760741.760670:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24670208 -> 24674304 00010000:00010000:0.0:1789760741.760689:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2e5db lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24670208->24674303] (req 24670208->24674303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.760721:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2e5db lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24670208->24674303] (req 24670208->24674303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760741.760827:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973342 is committed 00010000:00010000:0.0:1789760741.760879:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2e5db lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24670208->24674303] (req 24670208->24674303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.760887:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760741.760892:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2e5db lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24670208->24674303] (req 24670208->24674303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.760903:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2e5db lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24670208->24674303] (req 24670208->24674303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.760925:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2e5db lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24670208->24674303] (req 24670208->24674303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.760933:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2e5db lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24670208->24674303] (req 24670208->24674303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760741.761138:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac531c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512631552:12345-192.168.203.56@tcp:4:dd.0 Request processed in 31842us (32058us total) trans 4294973342 rc 0/0 00000100:00100000:0.0:1789760741.761149:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6053 00000100:00100000:0.0:1789760741.761159:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 721252179620 time 721293579831 state 0x2 ref 0 00000100:00100000:1.0:1789760741.764514:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760741.764582:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512631680 00000100:00100000:2.0:1789760741.764591:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512631680 00000100:00100000:2.0:1789760741.764642:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 721252179620 time 721293579831 state 0x0 ref 1 00000100:00100000:2.0:1789760741.764658:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 721293840247 time 721293579831 state 0x1 ref 1 00000100:00100000:2.0:1789760741.764692:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 721293840247 time 721328991520 state 0x0 ref 1 00000100:00100000:2.0:1789760741.764736:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2fb8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512631680:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760741.764788:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24674304, mtime: 0x6aad94e5, atime: 0x0, ctime: 0x6aad94e5, blocks: 0xc007 00010000:00010000:2.0:1789760741.764798:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2e5e2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.764814:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=721s v=(0 0) 00010000:00010000:2.0:1789760741.764819:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760741.764825:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2e5e2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24674304->24678399] (req 24674304->24678399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.764839:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2e5e2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24674304->24678399] (req 24674304->24678399) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760741.791565:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973343 is committed 00002000:00010000:2.0:1789760741.791641:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24674304 -> 24678400 00010000:00010000:2.0:1789760741.791656:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2e5e2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24674304->24678399] (req 24674304->24678399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.791665:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2e5e2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24674304->24678399] (req 24674304->24678399) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.791672:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2e5e2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24674304->24678399] (req 24674304->24678399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.791676:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760741.791679:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2e5e2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24674304->24678399] (req 24674304->24678399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.791685:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2e5e2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24674304->24678399] (req 24674304->24678399) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.791714:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2e5e2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24674304->24678399] (req 24674304->24678399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.791720:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2e5e2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24674304->24678399] (req 24674304->24678399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760741.791857:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2fb8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512631680:12345-192.168.203.56@tcp:4:dd.0 Request processed in 27124us (27347us total) trans 4294973343 rc 0/0 00000100:00100000:2.0:1789760741.791867:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6054 00000100:00100000:2.0:1789760741.791981:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 721293840247 time 721328991520 state 0x2 ref 0 00000100:00100000:0.0:1789760741.793894:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760741.794138:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512631808 00000100:00100000:2.0:1789760741.794147:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512631808 00000100:00100000:2.0:1789760741.794195:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 721293840247 time 721328991520 state 0x0 ref 1 00000100:00100000:2.0:1789760741.794210:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 721329251936 time 721328991520 state 0x1 ref 1 00000100:00100000:2.0:1789760741.794236:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 721329251936 time 721358535478 state 0x0 ref 1 00000100:00100000:2.0:1789760741.794248:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2746a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512631808:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760741.794289:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24678400, mtime: 0x6aad94e5, atime: 0x0, ctime: 0x6aad94e5, blocks: 0xc007 00010000:00010000:2.0:1789760741.794297:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2e5e9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.794312:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=721s v=(0 0) 00010000:00010000:2.0:1789760741.794322:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760741.794328:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2e5e9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24678400->24682495] (req 24678400->24682495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.794351:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2e5e9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24678400->24682495] (req 24678400->24682495) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760741.812938:0:3309:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8feac531ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3309:1876699540224512:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1789760741.813200:0:3309:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760741.813280:0:6707:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540224512 00000100:00100000:1.0:1789760741.813287:0:6707:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540224512 00000100:00100000:1.0:1789760741.813328:0:6707:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 197 00000100:00100000:1.0:1789760741.813348:0:6707:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4074700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3309:x1876699540224512:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1789760741.813383:0:6707:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3859808256 avail 3857711104 00000100:00100000:1.0:1789760741.813426:0:6707:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4074700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3309:x1876699540224512:12345-0@lo:13:osp-pre-1-0.0 Request processed in 80us (226us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760741.813435:0:6707:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 197 00000100:00100000:1.0:1789760741.813479:0:3309:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8feac531ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3309:1876699540224512:0@lo:13:osp-pre-1-0.0 00002000:00010000:3.0:1789760741.823948:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24678400 -> 24682496 00010000:00010000:3.0:1789760741.823968:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2e5e9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24678400->24682495] (req 24678400->24682495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.823981:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2e5e9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24678400->24682495] (req 24678400->24682495) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.823990:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2e5e9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24678400->24682495] (req 24678400->24682495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.823998:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760741.824003:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2e5e9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24678400->24682495] (req 24678400->24682495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.824021:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2e5e9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24678400->24682495] (req 24678400->24682495) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.824040:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2e5e9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24678400->24682495] (req 24678400->24682495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.824048:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2e5e9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24678400->24682495] (req 24678400->24682495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760741.824270:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2746a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2361:x1876699512631808:12345-192.168.203.56@tcp:4:dd.0 Request processed in 30018us (30378us total) trans 4294973344 rc 0/0 00000100:00100000:3.0:1789760741.824280:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6055 00000100:00100000:3.0:1789760741.824289:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 721329251936 time 721358535478 state 0x2 ref 0 00000100:00100000:3.0:1789760741.827305:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760741.827546:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512631936 00000100:00100000:2.0:1789760741.827556:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512631936 00000100:00100000:2.0:1789760741.827610:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 721329251936 time 721358535478 state 0x0 ref 1 00000100:00100000:2.0:1789760741.827625:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 721358795894 time 721358535478 state 0x1 ref 1 00000100:00100000:2.0:1789760741.827664:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 721358795894 time 721391963614 state 0x0 ref 1 00000100:00100000:2.0:1789760741.827676:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5307480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2360:x1876699512631936:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760741.827762:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24682496, mtime: 0x6aad94e5, atime: 0x0, ctime: 0x6aad94e5, blocks: 0xc007 00010000:00010000:2.0:1789760741.827771:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2e5f0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.827786:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=721s v=(0 0) 00010000:00010000:2.0:1789760741.827791:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760741.827797:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2e5f0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24682496->24686591] (req 24682496->24686591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.827811:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2e5f0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24682496->24686591] (req 24682496->24686591) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760741.828991:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973344 is committed 00000001:00080000:1.0:1789760741.863619:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973345 is committed 00002000:00010000:3.0:1789760741.863769:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24682496 -> 24686592 00010000:00010000:3.0:1789760741.863786:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2e5f0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24682496->24686591] (req 24682496->24686591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.863798:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2e5f0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24682496->24686591] (req 24682496->24686591) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.863807:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2e5f0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24682496->24686591] (req 24682496->24686591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.863816:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760741.863822:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2e5f0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24682496->24686591] (req 24682496->24686591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.863831:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2e5f0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24682496->24686591] (req 24682496->24686591) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.863854:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2e5f0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24682496->24686591] (req 24682496->24686591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.863862:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2e5f0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24682496->24686591] (req 24682496->24686591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760741.863943:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5307480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512631936:12345-192.168.203.56@tcp:4:dd.0 Request processed in 36265us (36680us total) trans 4294973345 rc 0/0 00000100:00100000:3.0:1789760741.863953:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6056 00000100:00100000:3.0:1789760741.863963:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 721358795894 time 721391963614 state 0x2 ref 0 00000100:00100000:2.0:1789760741.867014:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760741.867179:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512632064 00000100:00100000:0.0:1789760741.867189:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512632064 00000100:00100000:0.0:1789760741.867245:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 721358795894 time 721391963614 state 0x0 ref 1 00000100:00100000:0.0:1789760741.867260:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 721392224030 time 721391963614 state 0x1 ref 1 00000100:00100000:0.0:1789760741.867324:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 721392224030 time 721431622890 state 0x0 ref 1 00000100:00100000:0.0:1789760741.867351:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac531d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512632064:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760741.867406:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24686592, mtime: 0x6aad94e5, atime: 0x0, ctime: 0x6aad94e5, blocks: 0xc007 00010000:00010000:0.0:1789760741.867416:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2e5f7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.867431:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=721s v=(0 0) 00010000:00010000:0.0:1789760741.867436:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760741.867443:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2e5f7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24686592->24690687] (req 24686592->24690687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.867457:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2e5f7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24686592->24690687] (req 24686592->24690687) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760741.913684:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973346 is committed 00002000:00010000:3.0:1789760741.913837:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24686592 -> 24690688 00010000:00010000:3.0:1789760741.913853:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2e5f7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24686592->24690687] (req 24686592->24690687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.913868:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2e5f7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24686592->24690687] (req 24686592->24690687) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.913878:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2e5f7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24686592->24690687] (req 24686592->24690687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.913886:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760741.913891:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2e5f7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24686592->24690687] (req 24686592->24690687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.913907:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2e5f7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24686592->24690687] (req 24686592->24690687) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.913929:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2e5f7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24686592->24690687] (req 24686592->24690687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.913938:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2e5f7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24686592->24690687] (req 24686592->24690687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760741.914038:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac531d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512632064:12345-192.168.203.56@tcp:4:dd.0 Request processed in 46698us (47028us total) trans 4294973346 rc 0/0 00000100:00100000:3.0:1789760741.914049:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6057 00000100:00100000:3.0:1789760741.914059:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 721392224030 time 721431622890 state 0x2 ref 0 00000100:00100000:0.0:1789760741.919490:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760741.919761:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512632192 00000100:00100000:3.0:1789760741.919771:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512632192 00000100:00100000:3.0:1789760741.919832:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 721392224030 time 721431622890 state 0x0 ref 1 00000100:00100000:3.0:1789760741.919847:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 721431883306 time 721431622890 state 0x1 ref 1 00000100:00100000:3.0:1789760741.919881:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 721431883306 time 721484180612 state 0x0 ref 1 00000100:00100000:3.0:1789760741.919894:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2745880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512632192:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760741.919940:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24690688, mtime: 0x6aad94e5, atime: 0x0, ctime: 0x6aad94e5, blocks: 0xc007 00010000:00010000:3.0:1789760741.919950:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2e5fe lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.919967:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=721s v=(0 0) 00010000:00010000:3.0:1789760741.919972:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760741.919984:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2e5fe lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24690688->24694783] (req 24690688->24694783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.920001:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2e5fe lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24690688->24694783] (req 24690688->24694783) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760741.974131:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973347 is committed 00002000:00010000:3.0:1789760741.974214:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24690688 -> 24694784 00010000:00010000:3.0:1789760741.974229:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2e5fe lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24690688->24694783] (req 24690688->24694783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.974242:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2e5fe lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24690688->24694783] (req 24690688->24694783) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.974251:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2e5fe lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24690688->24694783] (req 24690688->24694783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.974259:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760741.974264:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2e5fe lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24690688->24694783] (req 24690688->24694783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.974273:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2e5fe lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24690688->24694783] (req 24690688->24694783) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.974292:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2e5fe lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24690688->24694783] (req 24690688->24694783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.974301:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2e5fe lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24690688->24694783] (req 24690688->24694783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760741.974439:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2745880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512632192:12345-192.168.203.56@tcp:4:dd.0 Request processed in 54541us (54952us total) trans 4294973347 rc 0/0 00000100:00100000:3.0:1789760741.974450:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6058 00000100:00100000:3.0:1789760741.974459:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 721431883306 time 721484180612 state 0x2 ref 0 00000100:00100000:2.0:1789760741.977180:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760741.977251:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512632320 00000100:00100000:3.0:1789760741.977261:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512632320 00000100:00100000:3.0:1789760741.977322:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 721431883306 time 721484180612 state 0x0 ref 1 00000100:00100000:3.0:1789760741.977346:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 721484441028 time 721484180612 state 0x1 ref 1 00000100:00100000:3.0:1789760741.977422:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 721484441028 time 721541721499 state 0x0 ref 1 00000100:00100000:3.0:1789760741.977435:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac531e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512632320:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760741.977485:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24694784, mtime: 0x6aad94e5, atime: 0x0, ctime: 0x6aad94e5, blocks: 0xc007 00010000:00010000:3.0:1789760741.977494:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2e605 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.977509:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=721s v=(0 0) 00010000:00010000:3.0:1789760741.977514:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760741.977520:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2e605 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24694784->24698879] (req 24694784->24698879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.977538:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2e605 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24694784->24698879] (req 24694784->24698879) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760742.008998:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973348 is committed 00002000:00010000:3.0:1789760742.009095:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24694784 -> 24698880 00010000:00010000:3.0:1789760742.009114:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2e605 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24694784->24698879] (req 24694784->24698879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.009257:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2e605 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24694784->24698879] (req 24694784->24698879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.009268:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2e605 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24694784->24698879] (req 24694784->24698879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.009276:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760742.009281:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2e605 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24694784->24698879] (req 24694784->24698879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.009290:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2e605 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24694784->24698879] (req 24694784->24698879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.009312:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2e605 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24694784->24698879] (req 24694784->24698879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.009319:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2e605 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24694784->24698879] (req 24694784->24698879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760742.009947:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac531e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512632320:12345-192.168.203.56@tcp:4:dd.0 Request processed in 32508us (32769us total) trans 4294973348 rc 0/0 00000100:00100000:3.0:1789760742.009960:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6059 00000100:00100000:3.0:1789760742.009969:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 721484441028 time 721541721499 state 0x2 ref 0 00000100:00100000:2.0:1789760742.013148:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760742.013501:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512632448 00000100:00100000:3.0:1789760742.013511:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512632448 00000100:00100000:3.0:1789760742.013566:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 721484441028 time 721541721499 state 0x0 ref 1 00000100:00100000:3.0:1789760742.013581:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 721541981915 time 721541721499 state 0x1 ref 1 00000100:00100000:3.0:1789760742.013611:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 721541981915 time 721577911214 state 0x0 ref 1 00000100:00100000:3.0:1789760742.013626:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac531ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512632448:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760742.013872:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24698880, mtime: 0x6aad94e5, atime: 0x0, ctime: 0x6aad94e5, blocks: 0xc007 00010000:00010000:3.0:1789760742.013882:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2e60c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.013898:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=721s v=(0 0) 00010000:00010000:3.0:1789760742.013903:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760742.013909:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2e60c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24698880->24702975] (req 24698880->24702975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.013923:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2e60c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24698880->24702975] (req 24698880->24702975) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760742.059249:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24698880 -> 24702976 00000001:00080000:2.0:1789760742.059257:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973349 is committed 00010000:00010000:0.0:1789760742.059270:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2e60c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24698880->24702975] (req 24698880->24702975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.059283:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2e60c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24698880->24702975] (req 24698880->24702975) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.059360:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2e60c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24698880->24702975] (req 24698880->24702975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.059369:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760742.059374:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2e60c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24698880->24702975] (req 24698880->24702975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.059384:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2e60c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24698880->24702975] (req 24698880->24702975) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.059408:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2e60c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24698880->24702975] (req 24698880->24702975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.059416:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2e60c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24698880->24702975] (req 24698880->24702975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760742.059567:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac531ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512632448:12345-192.168.203.56@tcp:4:dd.0 Request processed in 45939us (46423us total) trans 4294973349 rc 0/0 00000100:00100000:0.0:1789760742.059577:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6060 00000100:00100000:0.0:1789760742.059587:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 721541981915 time 721577911214 state 0x2 ref 0 00000100:00100000:0.0:1789760742.062439:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760742.062517:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512632576 00000100:00100000:3.0:1789760742.062527:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512632576 00000100:00100000:3.0:1789760742.062586:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 721541981915 time 721577911214 state 0x0 ref 1 00000100:00100000:3.0:1789760742.062679:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 721578171630 time 721577911214 state 0x1 ref 1 00000100:00100000:3.0:1789760742.062728:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 721578171630 time 721627026807 state 0x0 ref 1 00000100:00100000:3.0:1789760742.062742:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2746300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512632576:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760742.063015:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24702976, mtime: 0x6aad94e5, atime: 0x0, ctime: 0x6aad94e5, blocks: 0xc007 00010000:00010000:3.0:1789760742.063024:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2e613 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.063040:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=721s v=(0 0) 00010000:00010000:3.0:1789760742.063044:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760742.063050:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2e613 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24702976->24707071] (req 24702976->24707071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.063064:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2e613 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24702976->24707071] (req 24702976->24707071) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760742.102290:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973350 is committed 00002000:00010000:2.0:1789760742.102641:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24702976 -> 24707072 00010000:00010000:2.0:1789760742.102658:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2e613 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24702976->24707071] (req 24702976->24707071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.102671:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2e613 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24702976->24707071] (req 24702976->24707071) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.102681:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2e613 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24702976->24707071] (req 24702976->24707071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.102688:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760742.102693:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2e613 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24702976->24707071] (req 24702976->24707071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.102722:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2e613 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24702976->24707071] (req 24702976->24707071) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.102741:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2e613 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24702976->24707071] (req 24702976->24707071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.103527:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2e613 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24702976->24707071] (req 24702976->24707071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760742.104967:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2746300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512632576:12345-192.168.203.56@tcp:4:dd.0 Request processed in 42220us (42529us total) trans 4294973350 rc 0/0 00000100:00100000:2.0:1789760742.104980:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6061 00000100:00100000:2.0:1789760742.104990:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 721578171630 time 721627026807 state 0x2 ref 0 00000100:00100000:2.0:1789760742.111739:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760742.112039:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512632704 00000100:00100000:3.0:1789760742.112048:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512632704 00000100:00100000:3.0:1789760742.112105:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 721578171630 time 721627026807 state 0x0 ref 1 00000100:00100000:3.0:1789760742.112123:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 721627287223 time 721627026807 state 0x1 ref 1 00000100:00100000:3.0:1789760742.112154:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 721627287223 time 721676453900 state 0x0 ref 1 00000100:00100000:3.0:1789760742.112168:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febea651500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512632704:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760742.112216:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24707072, mtime: 0x6aad94e5, atime: 0x0, ctime: 0x6aad94e5, blocks: 0xc007 00010000:00010000:3.0:1789760742.112224:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2e61a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.112240:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=721s v=(0 0) 00010000:00010000:3.0:1789760742.112245:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760742.112251:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2e61a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24707072->24711167] (req 24707072->24711167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.112265:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2e61a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24707072->24711167] (req 24707072->24711167) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760742.144878:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24707072 -> 24711168 00010000:00010000:3.0:1789760742.144963:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2e61a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24707072->24711167] (req 24707072->24711167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.144977:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2e61a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24707072->24711167] (req 24707072->24711167) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.144988:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2e61a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24707072->24711167] (req 24707072->24711167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.145055:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760742.145061:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2e61a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24707072->24711167] (req 24707072->24711167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.145072:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2e61a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24707072->24711167] (req 24707072->24711167) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.145092:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2e61a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24707072->24711167] (req 24707072->24711167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.145101:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2e61a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24707072->24711167] (req 24707072->24711167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760742.145266:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febea651500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2361:x1876699512632704:12345-192.168.203.56@tcp:4:dd.0 Request processed in 33095us (33530us total) trans 4294973351 rc 0/0 00000100:00100000:3.0:1789760742.145278:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6062 00000100:00100000:3.0:1789760742.145288:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 721627287223 time 721676453900 state 0x2 ref 0 00000001:00080000:3.0:1789760742.145341:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973351 is committed 00000100:00100000:2.0:1789760742.150680:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760742.150855:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512632832 00000100:00100000:0.0:1789760742.150863:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512632832 00000100:00100000:1.0:1789760742.150968:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 721627287223 time 721676453900 state 0x0 ref 1 00000100:00100000:1.0:1789760742.150984:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 721676714316 time 721676453900 state 0x1 ref 1 00000100:00100000:1.0:1789760742.151078:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 721676714316 time 721715378048 state 0x0 ref 1 00000100:00100000:1.0:1789760742.151088:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febea650380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512632832:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760742.151128:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24711168, mtime: 0x6aad94e5, atime: 0x0, ctime: 0x6aad94e5, blocks: 0xc007 00010000:00010000:1.0:1789760742.151135:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2e621 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.151146:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=721s v=(0 0) 00010000:00010000:1.0:1789760742.151149:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760742.151154:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2e621 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24711168->24715263] (req 24711168->24715263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.151169:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2e621 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24711168->24715263] (req 24711168->24715263) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760742.175466:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24711168 -> 24715264 00010000:00010000:2.0:1789760742.175488:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2e621 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24711168->24715263] (req 24711168->24715263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.175501:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2e621 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24711168->24715263] (req 24711168->24715263) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.175511:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2e621 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24711168->24715263] (req 24711168->24715263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.175524:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760742.175530:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2e621 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24711168->24715263] (req 24711168->24715263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.175539:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2e621 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24711168->24715263] (req 24711168->24715263) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.175559:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2e621 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24711168->24715263] (req 24711168->24715263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.175567:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2e621 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24711168->24715263] (req 24711168->24715263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760742.175729:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973352 is committed 00000100:00100000:2.0:1789760742.175737:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febea650380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2360:x1876699512632832:12345-192.168.203.56@tcp:4:dd.0 Request processed in 24601us (25018us total) trans 4294973352 rc 0/0 00000100:00100000:2.0:1789760742.175758:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6063 00000100:00100000:2.0:1789760742.175768:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 721676714316 time 721715378048 state 0x2 ref 0 00000100:00100000:1.0:1789760742.182910:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760742.183949:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512632960 00000100:00100000:2.0:1789760742.183962:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512632960 00000100:00100000:2.0:1789760742.184015:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 721676714316 time 721715378048 state 0x0 ref 1 00000100:00100000:2.0:1789760742.184030:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 721715638464 time 721715378048 state 0x1 ref 1 00000100:00100000:2.0:1789760742.184051:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 721715638464 time 721748351508 state 0x0 ref 1 00000100:00100000:2.0:1789760742.184149:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac437f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512632960:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760742.184193:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24715264, mtime: 0x6aad94e5, atime: 0x0, ctime: 0x6aad94e5, blocks: 0xc007 00010000:00010000:2.0:1789760742.184202:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2e628 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.184217:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=721s v=(0 0) 00010000:00010000:2.0:1789760742.184221:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760742.184227:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2e628 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24715264->24719359] (req 24715264->24719359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.184241:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2e628 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24715264->24719359] (req 24715264->24719359) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760742.217777:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973353 is committed 00002000:00010000:2.0:1789760742.217875:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24715264 -> 24719360 00010000:00010000:2.0:1789760742.217890:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2e628 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24715264->24719359] (req 24715264->24719359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.217902:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2e628 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24715264->24719359] (req 24715264->24719359) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.217910:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2e628 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24715264->24719359] (req 24715264->24719359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.217917:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760742.217923:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2e628 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24715264->24719359] (req 24715264->24719359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.217932:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2e628 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24715264->24719359] (req 24715264->24719359) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.217951:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2e628 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24715264->24719359] (req 24715264->24719359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.217959:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2e628 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24715264->24719359] (req 24715264->24719359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760742.218156:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac437f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512632960:12345-192.168.203.56@tcp:4:dd.0 Request processed in 34006us (35250us total) trans 4294973353 rc 0/0 00000100:00100000:2.0:1789760742.218166:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6064 00000100:00100000:2.0:1789760742.218177:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 721715638464 time 721748351508 state 0x2 ref 0 00000100:00100000:0.0:1789760742.222385:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760742.222503:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512633088 00000100:00100000:3.0:1789760742.222512:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512633088 00000100:00100000:3.0:1789760742.222559:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 721715638464 time 721748351508 state 0x0 ref 1 00000100:00100000:3.0:1789760742.222570:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 721748611924 time 721748351508 state 0x1 ref 1 00000100:00100000:3.0:1789760742.222594:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 721748611924 time 721786894656 state 0x0 ref 1 00000100:00100000:3.0:1789760742.222607:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2745180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512633088:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760742.222650:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24719360, mtime: 0x6aad94e5, atime: 0x0, ctime: 0x6aad94e5, blocks: 0xc007 00010000:00010000:3.0:1789760742.222659:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2e62f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.222673:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=721s v=(0 0) 00010000:00010000:3.0:1789760742.222677:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760742.222681:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2e62f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24719360->24723455] (req 24719360->24723455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.222692:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2e62f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24719360->24723455] (req 24719360->24723455) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760742.257667:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973354 is committed 00002000:00010000:1.0:1789760742.257845:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24719360 -> 24723456 00010000:00010000:1.0:1789760742.257862:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2e62f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24719360->24723455] (req 24719360->24723455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.257876:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2e62f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24719360->24723455] (req 24719360->24723455) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.257885:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2e62f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24719360->24723455] (req 24719360->24723455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.257892:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760742.257897:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2e62f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24719360->24723455] (req 24719360->24723455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.257906:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2e62f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24719360->24723455] (req 24719360->24723455) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.257926:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2e62f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24719360->24723455] (req 24719360->24723455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.257933:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2e62f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24719360->24723455] (req 24719360->24723455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760742.258163:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2745180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512633088:12345-192.168.203.56@tcp:4:dd.0 Request processed in 35553us (35784us total) trans 4294973354 rc 0/0 00000100:00100000:1.0:1789760742.258174:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6065 00000100:00100000:1.0:1789760742.258184:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 721748611924 time 721786894656 state 0x2 ref 0 00000100:00100000:0.0:1789760742.260631:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760742.262163:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512633216 00000100:00100000:2.0:1789760742.262176:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512633216 00000100:00100000:3.0:1789760742.262311:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 721748611924 time 721786894656 state 0x0 ref 1 00000100:00100000:3.0:1789760742.262395:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 721787155072 time 721786894656 state 0x1 ref 1 00000100:00100000:3.0:1789760742.262426:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 721787155072 time 721826725635 state 0x0 ref 1 00000100:00100000:3.0:1789760742.262440:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2744a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512633216:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760742.262492:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24723456, mtime: 0x6aad94e5, atime: 0x0, ctime: 0x6aad94e5, blocks: 0xc007 00010000:00010000:3.0:1789760742.262562:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2e636 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.262578:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=721s v=(0 0) 00010000:00010000:3.0:1789760742.262582:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760742.262587:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2e636 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24723456->24727551] (req 24723456->24727551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.262600:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2e636 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24723456->24727551] (req 24723456->24727551) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760742.291382:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973355 is committed 00002000:00010000:1.0:1789760742.291512:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24723456 -> 24727552 00010000:00010000:1.0:1789760742.291528:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2e636 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24723456->24727551] (req 24723456->24727551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.291541:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2e636 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24723456->24727551] (req 24723456->24727551) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.291550:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2e636 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24723456->24727551] (req 24723456->24727551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.291558:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760742.291563:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2e636 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24723456->24727551] (req 24723456->24727551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.291572:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2e636 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24723456->24727551] (req 24723456->24727551) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.291590:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2e636 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24723456->24727551] (req 24723456->24727551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.291603:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2e636 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24723456->24727551] (req 24723456->24727551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760742.291754:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2744a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512633216:12345-192.168.203.56@tcp:4:dd.0 Request processed in 29312us (31126us total) trans 4294973355 rc 0/0 00000100:00100000:1.0:1789760742.291763:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6066 00000100:00100000:1.0:1789760742.291773:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 721787155072 time 721826725635 state 0x2 ref 0 00000100:00100000:0.0:1789760742.295305:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760742.295425:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512633344 00000100:00100000:2.0:1789760742.295435:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512633344 00000100:00100000:2.0:1789760742.295492:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 721787155072 time 721826725635 state 0x0 ref 1 00000100:00100000:2.0:1789760742.295507:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 721826986051 time 721826725635 state 0x1 ref 1 00000100:00100000:2.0:1789760742.295536:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 721826986051 time 721859836461 state 0x0 ref 1 00000100:00100000:2.0:1789760742.295549:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2746680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512633344:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760742.295599:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24727552, mtime: 0x6aad94e5, atime: 0x0, ctime: 0x6aad94e5, blocks: 0xc007 00010000:00010000:2.0:1789760742.295609:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2e63d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.295624:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=721s v=(0 0) 00010000:00010000:2.0:1789760742.295629:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760742.295635:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2e63d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24727552->24731647] (req 24727552->24731647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.295648:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2e63d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24727552->24731647] (req 24727552->24731647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760742.353083:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973356 is committed 00002000:00010000:1.0:1789760742.353133:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24727552 -> 24731648 00010000:00010000:1.0:1789760742.353149:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2e63d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24727552->24731647] (req 24727552->24731647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.353162:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2e63d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24727552->24731647] (req 24727552->24731647) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.353171:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2e63d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24727552->24731647] (req 24727552->24731647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.353178:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760742.353183:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2e63d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24727552->24731647] (req 24727552->24731647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.353191:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2e63d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24727552->24731647] (req 24727552->24731647) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.353215:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2e63d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24727552->24731647] (req 24727552->24731647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.353224:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2e63d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24727552->24731647] (req 24727552->24731647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760742.353321:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2746680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512633344:12345-192.168.203.56@tcp:4:dd.0 Request processed in 57762us (58013us total) trans 4294973356 rc 0/0 00000100:00100000:1.0:1789760742.353339:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6067 00000100:00100000:1.0:1789760742.353348:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 721826986051 time 721859836461 state 0x2 ref 0 00000100:00100000:0.0:1789760742.358183:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760742.358291:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512633472 00000100:00100000:1.0:1789760742.358348:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512633472 00000100:00100000:1.0:1789760742.358406:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 721826986051 time 721859836461 state 0x0 ref 1 00000100:00100000:1.0:1789760742.358421:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 721860096877 time 721859836461 state 0x1 ref 1 00000100:00100000:1.0:1789760742.359138:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 721860096877 time 721923435002 state 0x0 ref 1 00000100:00100000:1.0:1789760742.359155:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febea4b4850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512633472:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760742.359215:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24731648, mtime: 0x6aad94e5, atime: 0x0, ctime: 0x6aad94e5, blocks: 0xc007 00010000:00010000:1.0:1789760742.359224:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2e644 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.359239:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=721s v=(0 0) 00010000:00010000:1.0:1789760742.359244:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760742.359251:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2e644 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24731648->24735743] (req 24731648->24735743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.359265:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2e644 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24731648->24735743] (req 24731648->24735743) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760742.389790:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24731648 -> 24735744 00010000:00010000:1.0:1789760742.389807:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2e644 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24731648->24735743] (req 24731648->24735743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.389816:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2e644 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24731648->24735743] (req 24731648->24735743) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.389822:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2e644 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24731648->24735743] (req 24731648->24735743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.389827:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760742.389831:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2e644 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24731648->24735743] (req 24731648->24735743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.389851:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2e644 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24731648->24735743] (req 24731648->24735743) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.389872:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2e644 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24731648->24735743] (req 24731648->24735743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.389880:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2e644 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24731648->24735743] (req 24731648->24735743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760742.389909:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973357 is committed 00000100:00100000:1.0:1789760742.390003:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febea4b4850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512633472:12345-192.168.203.56@tcp:4:dd.0 Request processed in 30846us (31823us total) trans 4294973357 rc 0/0 00000100:00100000:1.0:1789760742.390013:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6068 00000100:00100000:1.0:1789760742.390023:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 721860096877 time 721923435002 state 0x2 ref 0 00000100:00100000:0.0:1789760742.393834:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760742.394807:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512633600 00000100:00100000:1.0:1789760742.394820:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512633600 00000100:00100000:1.0:1789760742.394884:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 721860096877 time 721923435002 state 0x0 ref 1 00000100:00100000:1.0:1789760742.394900:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 721923695418 time 721923435002 state 0x1 ref 1 00000100:00100000:1.0:1789760742.394915:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 721923695418 time 721959216886 state 0x0 ref 1 00000100:00100000:1.0:1789760742.394925:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2747b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512633600:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760742.394963:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24735744, mtime: 0x6aad94e5, atime: 0x0, ctime: 0x6aad94e5, blocks: 0xc007 00010000:00010000:1.0:1789760742.394970:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2e64b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.394983:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=721s v=(0 0) 00010000:00010000:1.0:1789760742.394986:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760742.394990:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2e64b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24735744->24739839] (req 24735744->24739839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.395001:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2e64b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24735744->24739839] (req 24735744->24739839) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760742.435548:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24735744 -> 24739840 00010000:00010000:1.0:1789760742.435576:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2e64b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24735744->24739839] (req 24735744->24739839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.435590:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2e64b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24735744->24739839] (req 24735744->24739839) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.435600:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2e64b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24735744->24739839] (req 24735744->24739839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.435608:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760742.435613:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2e64b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24735744->24739839] (req 24735744->24739839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.435623:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2e64b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24735744->24739839] (req 24735744->24739839) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.435645:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2e64b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24735744->24739839] (req 24735744->24739839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.435652:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2e64b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24735744->24739839] (req 24735744->24739839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760742.435797:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2747b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2360:x1876699512633600:12345-192.168.203.56@tcp:4:dd.0 Request processed in 40868us (41968us total) trans 4294973358 rc 0/0 00000100:00100000:1.0:1789760742.435811:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6069 00000100:00100000:1.0:1789760742.435821:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 721923695418 time 721959216886 state 0x2 ref 0 00000001:00080000:3.0:1789760742.436733:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973358 is committed 00000100:00100000:0.0:1789760742.440545:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760742.440626:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512633728 00000100:00100000:1.0:1789760742.440636:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512633728 00000100:00100000:1.0:1789760742.440694:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 721923695418 time 721959216886 state 0x0 ref 1 00000100:00100000:1.0:1789760742.440758:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 721959477302 time 721959216886 state 0x1 ref 1 00000100:00100000:1.0:1789760742.440796:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 721959477302 time 722005095070 state 0x0 ref 1 00000100:00100000:1.0:1789760742.440808:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2747800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512633728:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760742.440859:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24739840, mtime: 0x6aad94e5, atime: 0x0, ctime: 0x6aad94e5, blocks: 0xc007 00010000:00010000:1.0:1789760742.440869:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2e652 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.440886:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=722s v=(0 0) 00010000:00010000:1.0:1789760742.440889:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760742.440895:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2e652 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24739840->24743935] (req 24739840->24743935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.440912:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2e652 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24739840->24743935] (req 24739840->24743935) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760742.478678:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24739840 -> 24743936 00010000:00010000:3.0:1789760742.478720:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2e652 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24739840->24743935] (req 24739840->24743935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.478734:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2e652 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24739840->24743935] (req 24739840->24743935) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.478743:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2e652 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24739840->24743935] (req 24739840->24743935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.478751:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760742.478755:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2e652 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24739840->24743935] (req 24739840->24743935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.478764:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2e652 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24739840->24743935] (req 24739840->24743935) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760742.478776:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973359 is committed 00010000:00010000:3.0:1789760742.478788:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2e652 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24739840->24743935] (req 24739840->24743935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.478795:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2e652 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24739840->24743935] (req 24739840->24743935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760742.478889:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2747800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512633728:12345-192.168.203.56@tcp:4:dd.0 Request processed in 38078us (38349us total) trans 4294973359 rc 0/0 00000100:00100000:3.0:1789760742.478898:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6070 00000100:00100000:3.0:1789760742.478909:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 721959477302 time 722005095070 state 0x2 ref 0 00000100:00100000:1.0:1789760742.481788:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760742.481828:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512633856 00000100:00100000:3.0:1789760742.481838:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512633856 00000100:00100000:3.0:1789760742.481893:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 721959477302 time 722005095070 state 0x0 ref 1 00000100:00100000:3.0:1789760742.481908:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 722005355486 time 722005095070 state 0x1 ref 1 00000100:00100000:3.0:1789760742.481929:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 722005355486 time 722046228586 state 0x0 ref 1 00000100:00100000:3.0:1789760742.481941:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac437f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512633856:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760742.482000:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24743936, mtime: 0x6aad94e5, atime: 0x0, ctime: 0x6aad94e5, blocks: 0xc007 00010000:00010000:3.0:1789760742.482010:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2e659 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.482024:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=722s v=(0 0) 00010000:00010000:3.0:1789760742.482028:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760742.482034:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2e659 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24743936->24748031] (req 24743936->24748031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.482056:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2e659 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24743936->24748031] (req 24743936->24748031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760742.541253:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973360 is committed 00002000:00010000:2.0:1789760742.541890:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24743936 -> 24748032 00010000:00010000:2.0:1789760742.541906:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2e659 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24743936->24748031] (req 24743936->24748031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.541920:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2e659 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24743936->24748031] (req 24743936->24748031) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.541928:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2e659 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24743936->24748031] (req 24743936->24748031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.541936:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760742.541941:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2e659 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24743936->24748031] (req 24743936->24748031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.541950:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2e659 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24743936->24748031] (req 24743936->24748031) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.541972:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2e659 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24743936->24748031] (req 24743936->24748031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.541980:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2e659 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24743936->24748031] (req 24743936->24748031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760742.542091:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac437f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512633856:12345-192.168.203.56@tcp:4:dd.0 Request processed in 60146us (60303us total) trans 4294973360 rc 0/0 00000100:00100000:2.0:1789760742.542101:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6071 00000100:00100000:2.0:1789760742.542121:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 722005355486 time 722046228586 state 0x2 ref 0 00000100:00100000:0.0:1789760742.544632:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760742.544816:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512633984 00000100:00100000:1.0:1789760742.544826:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512633984 00000100:00100000:1.0:1789760742.544876:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 722005355486 time 722046228586 state 0x0 ref 1 00000100:00100000:1.0:1789760742.544891:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 722046489002 time 722046228586 state 0x1 ref 1 00000100:00100000:1.0:1789760742.544920:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 722046489002 time 722109219991 state 0x0 ref 1 00000100:00100000:1.0:1789760742.544932:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2745500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512633984:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760742.544979:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24748032, mtime: 0x6aad94e5, atime: 0x0, ctime: 0x6aad94e5, blocks: 0xc007 00010000:00010000:1.0:1789760742.544987:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2e660 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.545012:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=722s v=(0 0) 00010000:00010000:1.0:1789760742.545017:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760742.545024:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2e660 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24748032->24752127] (req 24748032->24752127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.545185:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2e660 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24748032->24752127] (req 24748032->24752127) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760742.579301:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973361 is committed 00002000:00010000:1.0:1789760742.579528:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24748032 -> 24752128 00002000:00010000:1.0:1789760742.579533:0:13062:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760741 -> 1789760742 00002000:00010000:1.0:1789760742.579536:0:13062:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760741 -> 1789760742 00010000:00010000:1.0:1789760742.579549:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2e660 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24748032->24752127] (req 24748032->24752127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.579561:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2e660 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24748032->24752127] (req 24748032->24752127) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.579571:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2e660 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24748032->24752127] (req 24748032->24752127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.579578:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760742.579583:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2e660 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24748032->24752127] (req 24748032->24752127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.579593:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2e660 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24748032->24752127] (req 24748032->24752127) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.579613:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2e660 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24748032->24752127] (req 24748032->24752127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.579621:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2e660 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24748032->24752127] (req 24748032->24752127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760742.579863:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2745500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512633984:12345-192.168.203.56@tcp:4:dd.0 Request processed in 34926us (35233us total) trans 4294973361 rc 0/0 00000100:00100000:1.0:1789760742.579874:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6072 00000100:00100000:1.0:1789760742.579884:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 722046489002 time 722109219991 state 0x2 ref 0 00000100:00100000:0.0:1789760742.584062:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760742.584175:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512634112 00000100:00100000:3.0:1789760742.584188:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512634112 00000100:00100000:3.0:1789760742.584244:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 722046489002 time 722109219991 state 0x0 ref 1 00000100:00100000:3.0:1789760742.584258:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 722109480407 time 722109219991 state 0x1 ref 1 00000100:00100000:3.0:1789760742.584287:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 722109480407 time 722148587285 state 0x0 ref 1 00000100:00100000:3.0:1789760742.584300:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2746a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512634112:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760742.584358:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24752128, mtime: 0x6aad94e6, atime: 0x0, ctime: 0x6aad94e6, blocks: 0xc007 00010000:00010000:3.0:1789760742.584366:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2e667 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.584382:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=722s v=(0 0) 00010000:00010000:3.0:1789760742.584386:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760742.584393:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2e667 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24752128->24756223] (req 24752128->24756223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.584407:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2e667 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24752128->24756223] (req 24752128->24756223) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760742.626033:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24752128 -> 24756224 00000001:00080000:3.0:1789760742.626033:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973362 is committed 00010000:00010000:2.0:1789760742.626049:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2e667 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24752128->24756223] (req 24752128->24756223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.626063:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2e667 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24752128->24756223] (req 24752128->24756223) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.626172:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2e667 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24752128->24756223] (req 24752128->24756223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.626179:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760742.626184:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2e667 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24752128->24756223] (req 24752128->24756223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.626198:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2e667 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24752128->24756223] (req 24752128->24756223) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.626216:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2e667 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24752128->24756223] (req 24752128->24756223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.626223:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2e667 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24752128->24756223] (req 24752128->24756223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760742.626350:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2746a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512634112:12345-192.168.203.56@tcp:4:dd.0 Request processed in 42046us (42293us total) trans 4294973362 rc 0/0 00000100:00100000:2.0:1789760742.626428:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6073 00000100:00100000:2.0:1789760742.626440:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 722109480407 time 722148587285 state 0x2 ref 0 00000100:00100000:0.0:1789760742.634505:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760742.634775:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512634240 00000100:00100000:2.0:1789760742.634786:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512634240 00000100:00100000:2.0:1789760742.634839:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 722109480407 time 722148587285 state 0x0 ref 1 00000100:00100000:2.0:1789760742.634855:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 722148847701 time 722148587285 state 0x1 ref 1 00000100:00100000:2.0:1789760742.634888:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 722148847701 time 722199187383 state 0x0 ref 1 00000100:00100000:2.0:1789760742.634901:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2745880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512634240:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760742.634950:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24756224, mtime: 0x6aad94e6, atime: 0x0, ctime: 0x6aad94e6, blocks: 0xc007 00010000:00010000:2.0:1789760742.634959:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2e66e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.634975:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=722s v=(0 0) 00010000:00010000:2.0:1789760742.634979:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760742.634986:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2e66e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24756224->24760319] (req 24756224->24760319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.635000:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2e66e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24756224->24760319] (req 24756224->24760319) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760742.672337:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973363 is committed 00002000:00010000:0.0:1789760742.675018:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24756224 -> 24760320 00010000:00010000:0.0:1789760742.675057:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2e66e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24756224->24760319] (req 24756224->24760319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.675071:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2e66e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24756224->24760319] (req 24756224->24760319) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.675080:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2e66e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24756224->24760319] (req 24756224->24760319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.675087:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760742.675093:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2e66e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24756224->24760319] (req 24756224->24760319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.675102:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2e66e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24756224->24760319] (req 24756224->24760319) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.675122:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2e66e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24756224->24760319] (req 24756224->24760319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.675130:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2e66e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24756224->24760319] (req 24756224->24760319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760742.675236:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2745880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512634240:12345-192.168.203.56@tcp:4:dd.0 Request processed in 40332us (40735us total) trans 4294973363 rc 0/0 00000100:00100000:0.0:1789760742.675250:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6074 00000100:00100000:0.0:1789760742.675260:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 722148847701 time 722199187383 state 0x2 ref 0 00000100:00100000:0.0:1789760742.680165:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760742.680231:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512634368 00000100:00100000:1.0:1789760742.680242:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512634368 00000100:00100000:1.0:1789760742.680301:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 722148847701 time 722199187383 state 0x0 ref 1 00000100:00100000:1.0:1789760742.680317:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 722199447799 time 722199187383 state 0x1 ref 1 00000100:00100000:1.0:1789760742.680356:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 722199447799 time 722244655765 state 0x0 ref 1 00000100:00100000:1.0:1789760742.680369:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2746300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512634368:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760742.680416:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24760320, mtime: 0x6aad94e6, atime: 0x0, ctime: 0x6aad94e6, blocks: 0xc007 00010000:00010000:1.0:1789760742.680426:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2e675 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.680441:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=722s v=(0 0) 00010000:00010000:1.0:1789760742.680446:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760742.680453:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2e675 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24760320->24764415] (req 24760320->24764415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.680469:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2e675 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24760320->24764415] (req 24760320->24764415) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760742.707842:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973364 is committed 00002000:00010000:0.0:1789760742.708064:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24760320 -> 24764416 00010000:00010000:0.0:1789760742.708080:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2e675 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24760320->24764415] (req 24760320->24764415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.708093:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2e675 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24760320->24764415] (req 24760320->24764415) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.708102:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2e675 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24760320->24764415] (req 24760320->24764415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.708110:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760742.708115:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2e675 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24760320->24764415] (req 24760320->24764415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.708124:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2e675 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24760320->24764415] (req 24760320->24764415) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.708142:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2e675 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24760320->24764415] (req 24760320->24764415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.708150:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2e675 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24760320->24764415] (req 24760320->24764415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760742.708244:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2746300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512634368:12345-192.168.203.56@tcp:4:dd.0 Request processed in 27871us (28082us total) trans 4294973364 rc 0/0 00000100:00100000:0.0:1789760742.708254:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6075 00000100:00100000:0.0:1789760742.708264:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 722199447799 time 722244655765 state 0x2 ref 0 00000100:00100000:2.0:1789760742.711827:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760742.711969:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512634496 00000100:00100000:1.0:1789760742.711978:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512634496 00000100:00100000:1.0:1789760742.712025:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 722199447799 time 722244655765 state 0x0 ref 1 00000100:00100000:1.0:1789760742.712038:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 722244916181 time 722244655765 state 0x1 ref 1 00000100:00100000:1.0:1789760742.712061:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 722244916181 time 722276360844 state 0x0 ref 1 00000100:00100000:1.0:1789760742.712073:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febea651500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512634496:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760742.712123:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24764416, mtime: 0x6aad94e6, atime: 0x0, ctime: 0x6aad94e6, blocks: 0xc007 00010000:00010000:1.0:1789760742.712133:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2e67c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.712148:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=722s v=(0 0) 00010000:00010000:1.0:1789760742.712153:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760742.712158:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2e67c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24764416->24768511] (req 24764416->24768511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.712171:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2e67c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24764416->24768511] (req 24764416->24768511) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760742.752974:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973365 is committed 00002000:00010000:2.0:1789760742.753142:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24764416 -> 24768512 00010000:00010000:2.0:1789760742.753159:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2e67c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24764416->24768511] (req 24764416->24768511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.753173:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2e67c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24764416->24768511] (req 24764416->24768511) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.753183:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2e67c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24764416->24768511] (req 24764416->24768511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.753190:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760742.753196:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2e67c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24764416->24768511] (req 24764416->24768511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.753206:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2e67c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24764416->24768511] (req 24764416->24768511) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.753231:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2e67c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24764416->24768511] (req 24764416->24768511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.753239:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2e67c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24764416->24768511] (req 24764416->24768511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760742.753342:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febea651500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512634496:12345-192.168.203.56@tcp:4:dd.0 Request processed in 41266us (41519us total) trans 4294973365 rc 0/0 00000100:00100000:2.0:1789760742.753353:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6076 00000100:00100000:2.0:1789760742.753363:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 722244916181 time 722276360844 state 0x2 ref 0 00000100:00100000:1.0:1789760742.756445:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760742.756522:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512634624 00000100:00100000:0.0:1789760742.756531:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512634624 00000100:00100000:0.0:1789760742.756583:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 722244916181 time 722276360844 state 0x0 ref 1 00000100:00100000:0.0:1789760742.756596:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 722276621260 time 722276360844 state 0x1 ref 1 00000100:00100000:0.0:1789760742.756624:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 722276621260 time 722320924221 state 0x0 ref 1 00000100:00100000:0.0:1789760742.756637:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac437fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512634624:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760742.756686:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24768512, mtime: 0x6aad94e6, atime: 0x0, ctime: 0x6aad94e6, blocks: 0xc007 00010000:00010000:0.0:1789760742.756693:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2e683 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.756736:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=722s v=(0 0) 00010000:00010000:0.0:1789760742.756741:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760742.756747:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2e683 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24768512->24772607] (req 24768512->24772607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.756761:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2e683 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24768512->24772607] (req 24768512->24772607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.772798:0:5835:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:1.0:1789760742.793690:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973366 is committed 00002000:00010000:3.0:1789760742.794097:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24768512 -> 24772608 00010000:00010000:3.0:1789760742.794113:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2e683 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24768512->24772607] (req 24768512->24772607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.794126:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2e683 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24768512->24772607] (req 24768512->24772607) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.794135:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2e683 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24768512->24772607] (req 24768512->24772607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.794142:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760742.794147:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2e683 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24768512->24772607] (req 24768512->24772607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.794155:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2e683 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24768512->24772607] (req 24768512->24772607) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.794172:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2e683 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24768512->24772607] (req 24768512->24772607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.794179:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2e683 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24768512->24772607] (req 24768512->24772607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760742.794285:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac437fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512634624:12345-192.168.203.56@tcp:4:dd.0 Request processed in 37644us (37843us total) trans 4294973366 rc 0/0 00000100:00100000:3.0:1789760742.794296:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6077 00000100:00100000:3.0:1789760742.794306:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 722276621260 time 722320924221 state 0x2 ref 0 00000100:00100000:0.0:1789760742.799055:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760742.799283:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512634752 00000100:00100000:1.0:1789760742.799293:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512634752 00000100:00100000:1.0:1789760742.799359:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 722276621260 time 722320924221 state 0x0 ref 1 00000100:00100000:1.0:1789760742.799373:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 722321184637 time 722320924221 state 0x1 ref 1 00000100:00100000:0.0:1789760742.799600:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 722321184637 time 722363896818 state 0x0 ref 1 00000100:00100000:0.0:1789760742.799616:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2745180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512634752:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760742.799671:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24772608, mtime: 0x6aad94e6, atime: 0x0, ctime: 0x6aad94e6, blocks: 0xc007 00010000:00010000:0.0:1789760742.799680:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2e68a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.799712:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=722s v=(0 0) 00010000:00010000:0.0:1789760742.799718:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760742.799733:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2e68a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24772608->24776703] (req 24772608->24776703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.799748:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2e68a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24772608->24776703] (req 24772608->24776703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760742.833322:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973367 is committed 00002000:00010000:0.0:1789760742.833452:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24772608 -> 24776704 00010000:00010000:0.0:1789760742.833467:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2e68a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24772608->24776703] (req 24772608->24776703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.833480:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2e68a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24772608->24776703] (req 24772608->24776703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.833490:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2e68a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24772608->24776703] (req 24772608->24776703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.833497:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760742.833503:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2e68a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24772608->24776703] (req 24772608->24776703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.833513:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2e68a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24772608->24776703] (req 24772608->24776703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.833532:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2e68a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24772608->24776703] (req 24772608->24776703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.833540:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2e68a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24772608->24776703] (req 24772608->24776703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760742.833630:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2745180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512634752:12345-192.168.203.56@tcp:4:dd.0 Request processed in 34014us (34590us total) trans 4294973367 rc 0/0 00000100:00100000:0.0:1789760742.833641:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6078 00000100:00100000:0.0:1789760742.833652:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 722321184637 time 722363896818 state 0x2 ref 0 00000100:00100000:0.0:1789760742.836161:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760742.836500:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512634880 00000100:00100000:2.0:1789760742.836510:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512634880 00000100:00100000:2.0:1789760742.836627:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 722321184637 time 722363896818 state 0x0 ref 1 00000100:00100000:2.0:1789760742.836644:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 722364157234 time 722363896818 state 0x1 ref 1 00000100:00100000:2.0:1789760742.836669:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 722364157234 time 722400969067 state 0x0 ref 1 00000100:00100000:2.0:1789760742.836683:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2744a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512634880:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760742.836912:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24776704, mtime: 0x6aad94e6, atime: 0x0, ctime: 0x6aad94e6, blocks: 0xc007 00010000:00010000:2.0:1789760742.836923:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2e691 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.836939:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=722s v=(0 0) 00010000:00010000:2.0:1789760742.836945:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760742.836951:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2e691 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24776704->24780799] (req 24776704->24780799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.836965:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2e691 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24776704->24780799] (req 24776704->24780799) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760742.884987:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973368 is committed 00002000:00010000:2.0:1789760742.885176:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24776704 -> 24780800 00010000:00010000:2.0:1789760742.885193:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2e691 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24776704->24780799] (req 24776704->24780799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.885207:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2e691 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24776704->24780799] (req 24776704->24780799) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.885217:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2e691 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24776704->24780799] (req 24776704->24780799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.885223:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760742.885228:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2e691 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24776704->24780799] (req 24776704->24780799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.885237:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2e691 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24776704->24780799] (req 24776704->24780799) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.885253:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2e691 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24776704->24780799] (req 24776704->24780799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.885259:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2e691 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24776704->24780799] (req 24776704->24780799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760742.885400:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2744a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512634880:12345-192.168.203.56@tcp:4:dd.0 Request processed in 48714us (49239us total) trans 4294973368 rc 0/0 00000100:00100000:2.0:1789760742.885411:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6079 00000100:00100000:2.0:1789760742.885430:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 722364157234 time 722400969067 state 0x2 ref 0 00000100:00100000:1.0:1789760742.889499:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760742.889658:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512635008 00000100:00100000:3.0:1789760742.889667:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512635008 00000100:00100000:3.0:1789760742.889900:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 722364157234 time 722400969067 state 0x0 ref 1 00000100:00100000:3.0:1789760742.889914:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 722401229483 time 722400969067 state 0x1 ref 1 00000100:00100000:0.0:1789760742.889956:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 722401229483 time 722454253242 state 0x0 ref 1 00000100:00100000:0.0:1789760742.889971:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac437d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512635008:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760742.890017:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24780800, mtime: 0x6aad94e6, atime: 0x0, ctime: 0x6aad94e6, blocks: 0xc007 00010000:00010000:0.0:1789760742.890025:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2e698 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.890041:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=722s v=(0 0) 00010000:00010000:0.0:1789760742.890046:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760742.890053:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2e698 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24780800->24784895] (req 24780800->24784895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.890066:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2e698 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24780800->24784895] (req 24780800->24784895) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760742.926371:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973369 is committed 00002000:00010000:1.0:1789760742.926584:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24780800 -> 24784896 00010000:00010000:1.0:1789760742.926600:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2e698 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24780800->24784895] (req 24780800->24784895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.926613:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2e698 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24780800->24784895] (req 24780800->24784895) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.926622:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2e698 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24780800->24784895] (req 24780800->24784895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.926629:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760742.926645:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2e698 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24780800->24784895] (req 24780800->24784895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.926654:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2e698 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24780800->24784895] (req 24780800->24784895) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.926672:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2e698 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24780800->24784895] (req 24780800->24784895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.926680:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2e698 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24780800->24784895] (req 24780800->24784895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760742.926917:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac437d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512635008:12345-192.168.203.56@tcp:4:dd.0 Request processed in 36944us (37421us total) trans 4294973369 rc 0/0 00000100:00100000:1.0:1789760742.926927:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6080 00000100:00100000:1.0:1789760742.926938:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 722401229483 time 722454253242 state 0x2 ref 0 00000100:00100000:0.0:1789760742.933159:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760742.933233:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512635136 00000100:00100000:2.0:1789760742.933243:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512635136 00000100:00100000:2.0:1789760742.933292:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 722401229483 time 722454253242 state 0x0 ref 1 00000100:00100000:2.0:1789760742.933304:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 722454513658 time 722454253242 state 0x1 ref 1 00000100:00100000:3.0:1789760742.933362:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 722454513658 time 722497659380 state 0x0 ref 1 00000100:00100000:3.0:1789760742.933380:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2746680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512635136:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760742.933434:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24784896, mtime: 0x6aad94e6, atime: 0x0, ctime: 0x6aad94e6, blocks: 0xc007 00010000:00010000:3.0:1789760742.933443:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2e69f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.933458:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=722s v=(0 0) 00010000:00010000:3.0:1789760742.933463:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760742.933469:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2e69f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24784896->24788991] (req 24784896->24788991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.933486:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2e69f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24784896->24788991] (req 24784896->24788991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760742.960169:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24784896 -> 24788992 00010000:00010000:0.0:1789760742.960189:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2e69f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24784896->24788991] (req 24784896->24788991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.960203:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2e69f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24784896->24788991] (req 24784896->24788991) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.960212:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2e69f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24784896->24788991] (req 24784896->24788991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.960219:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760742.960225:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2e69f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24784896->24788991] (req 24784896->24788991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.960234:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2e69f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24784896->24788991] (req 24784896->24788991) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.960258:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2e69f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24784896->24788991] (req 24784896->24788991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.960266:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2e69f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24784896->24788991] (req 24784896->24788991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760742.960346:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973370 is committed 00000100:00100000:0.0:1789760742.960396:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2746680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512635136:12345-192.168.203.56@tcp:4:dd.0 Request processed in 26993us (27225us total) trans 4294973370 rc 0/0 00000100:00100000:0.0:1789760742.960407:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6081 00000100:00100000:0.0:1789760742.960416:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 722454513658 time 722497659380 state 0x2 ref 0 00000100:00100000:0.0:1789760742.963974:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760742.964226:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512635264 00000100:00100000:2.0:1789760742.964236:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512635264 00000100:00100000:2.0:1789760742.964288:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 722454513658 time 722497659380 state 0x0 ref 1 00000100:00100000:2.0:1789760742.964303:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 722497919796 time 722497659380 state 0x1 ref 1 00000100:00100000:2.0:1789760742.964380:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 722497919796 time 722528679255 state 0x0 ref 1 00000100:00100000:2.0:1789760742.964393:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febfc0a3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512635264:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760742.964643:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24788992, mtime: 0x6aad94e6, atime: 0x0, ctime: 0x6aad94e6, blocks: 0xc007 00010000:00010000:2.0:1789760742.964652:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2e6a6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.964668:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=722s v=(0 0) 00010000:00010000:2.0:1789760742.964672:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760742.964679:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2e6a6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24788992->24793087] (req 24788992->24793087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.964692:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2e6a6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24788992->24793087] (req 24788992->24793087) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760742.991486:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973371 is committed 00002000:00010000:2.0:1789760742.991518:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24788992 -> 24793088 00010000:00010000:2.0:1789760742.991533:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2e6a6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24788992->24793087] (req 24788992->24793087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.991547:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2e6a6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24788992->24793087] (req 24788992->24793087) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.991557:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2e6a6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24788992->24793087] (req 24788992->24793087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.991565:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760742.991569:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2e6a6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24788992->24793087] (req 24788992->24793087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.991578:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2e6a6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24788992->24793087] (req 24788992->24793087) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.991596:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2e6a6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24788992->24793087] (req 24788992->24793087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.991603:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2e6a6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24788992->24793087] (req 24788992->24793087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760742.991689:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febfc0a3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512635264:12345-192.168.203.56@tcp:4:dd.0 Request processed in 27292us (27718us total) trans 4294973371 rc 0/0 00000100:00100000:2.0:1789760742.991769:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6082 00000100:00100000:2.0:1789760742.991789:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 722497919796 time 722528679255 state 0x2 ref 0 00000100:00100000:2.0:1789760742.994912:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760742.994989:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512635392 00000100:00100000:1.0:1789760742.995047:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512635392 00000100:00100000:1.0:1789760742.995104:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 722497919796 time 722528679255 state 0x0 ref 1 00000100:00100000:1.0:1789760742.995122:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 722528939671 time 722528679255 state 0x1 ref 1 00000100:00100000:1.0:1789760742.995152:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 722528939671 time 722559452007 state 0x0 ref 1 00000100:00100000:1.0:1789760742.995164:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febea650380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512635392:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760742.995209:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24793088, mtime: 0x6aad94e6, atime: 0x0, ctime: 0x6aad94e6, blocks: 0xc007 00010000:00010000:1.0:1789760742.995218:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2e6ad lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.995234:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=722s v=(0 0) 00010000:00010000:1.0:1789760742.995238:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760742.995245:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2e6ad lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24793088->24797183] (req 24793088->24797183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.995258:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2e6ad lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24793088->24797183] (req 24793088->24797183) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760743.018425:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973372 is committed 00002000:00010000:2.0:1789760743.018534:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24793088 -> 24797184 00010000:00010000:2.0:1789760743.018619:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2e6ad lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24793088->24797183] (req 24793088->24797183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.018633:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2e6ad lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24793088->24797183] (req 24793088->24797183) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.018643:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2e6ad lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24793088->24797183] (req 24793088->24797183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.018650:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760743.018655:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2e6ad lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24793088->24797183] (req 24793088->24797183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.018664:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2e6ad lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24793088->24797183] (req 24793088->24797183) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.018685:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2e6ad lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24793088->24797183] (req 24793088->24797183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.018728:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2e6ad lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24793088->24797183] (req 24793088->24797183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760743.018815:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febea650380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512635392:12345-192.168.203.56@tcp:4:dd.0 Request processed in 23648us (23909us total) trans 4294973372 rc 0/0 00000100:00100000:2.0:1789760743.018830:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6083 00000100:00100000:2.0:1789760743.018841:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 722528939671 time 722559452007 state 0x2 ref 0 00000100:00100000:0.0:1789760743.022026:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760743.022176:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512635520 00000100:00100000:3.0:1789760743.022184:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512635520 00000100:00080000:0.0:1789760743.022230:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1789760743.022248:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 722528939671 time 722559452007 state 0x0 ref 1 00000100:00100000:3.0:1789760743.022263:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 722559712423 time 722559452007 state 0x1 ref 1 00000100:00100000:3.0:1789760743.022285:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 722559712423 time 722586585470 state 0x0 ref 1 00000100:00100000:3.0:1789760743.022309:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febfc0a3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512635520:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760743.022363:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24797184, mtime: 0x6aad94e6, atime: 0x0, ctime: 0x6aad94e6, blocks: 0xc007 00010000:00010000:3.0:1789760743.022372:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2e6b4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.022389:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=722s v=(0 0) 00010000:00010000:3.0:1789760743.022394:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760743.022401:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2e6b4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24797184->24801279] (req 24797184->24801279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.022420:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2e6b4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24797184->24801279] (req 24797184->24801279) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760743.060076:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24797184 -> 24801280 00010000:00010000:0.0:1789760743.060097:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2e6b4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24797184->24801279] (req 24797184->24801279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.060111:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2e6b4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24797184->24801279] (req 24797184->24801279) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.060120:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2e6b4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24797184->24801279] (req 24797184->24801279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.060133:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760743.060138:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2e6b4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24797184->24801279] (req 24797184->24801279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.060147:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2e6b4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24797184->24801279] (req 24797184->24801279) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.060165:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2e6b4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24797184->24801279] (req 24797184->24801279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.060173:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2e6b4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24797184->24801279] (req 24797184->24801279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760743.060260:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febfc0a3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2358:x1876699512635520:12345-192.168.203.56@tcp:4:dd.0 Request processed in 37961us (38238us total) trans 4294973373 rc 0/0 00000100:00100000:0.0:1789760743.060270:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6084 00000100:00100000:0.0:1789760743.060279:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 722559712423 time 722586585470 state 0x2 ref 0 00000001:00080000:2.0:1789760743.060604:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973373 is committed 00000100:00100000:0.0:1789760743.063671:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760743.063848:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512635648 00000100:00100000:2.0:1789760743.063858:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512635648 00000100:00100000:3.0:1789760743.063999:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 722559712423 time 722586585470 state 0x0 ref 1 00000100:00100000:3.0:1789760743.064014:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 722586845886 time 722586585470 state 0x1 ref 1 00000100:00100000:3.0:1789760743.064152:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 722586845886 time 722628450766 state 0x0 ref 1 00000100:00100000:3.0:1789760743.064167:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac287b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512635648:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760743.064212:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24801280, mtime: 0x6aad94e6, atime: 0x0, ctime: 0x6aad94e6, blocks: 0xc007 00010000:00010000:3.0:1789760743.064221:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2e6bb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.064237:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=722s v=(0 0) 00010000:00010000:3.0:1789760743.064242:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760743.064248:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2e6bb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24801280->24805375] (req 24801280->24805375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.064260:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2e6bb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24801280->24805375] (req 24801280->24805375) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760743.094976:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24801280 -> 24805376 00010000:00010000:2.0:1789760743.095115:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2e6bb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24801280->24805375] (req 24801280->24805375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.095129:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2e6bb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24801280->24805375] (req 24801280->24805375) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.095139:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2e6bb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24801280->24805375] (req 24801280->24805375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.095147:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760743.095153:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2e6bb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24801280->24805375] (req 24801280->24805375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.095163:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2e6bb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24801280->24805375] (req 24801280->24805375) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.095182:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2e6bb lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24801280->24805375] (req 24801280->24805375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.095190:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2e6bb lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24801280->24805375] (req 24801280->24805375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760743.095315:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac287b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2358:x1876699512635648:12345-192.168.203.56@tcp:4:dd.0 Request processed in 31146us (31647us total) trans 4294973374 rc 0/0 00000100:00100000:2.0:1789760743.095334:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6085 00000100:00100000:2.0:1789760743.095344:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 722586845886 time 722628450766 state 0x2 ref 0 00000001:00080000:2.0:1789760743.095380:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973374 is committed 00000100:00100000:2.0:1789760743.101897:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760743.101999:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512635776 00000100:00100000:1.0:1789760743.102009:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512635776 00000100:00100000:1.0:1789760743.102066:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 722586845886 time 722628450766 state 0x0 ref 1 00000100:00100000:1.0:1789760743.102081:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 722628711182 time 722628450766 state 0x1 ref 1 00000100:00100000:1.0:1789760743.102116:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 722628711182 time 722666415544 state 0x0 ref 1 00000100:00100000:1.0:1789760743.102127:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac28a2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512635776:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760743.102180:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24805376, mtime: 0x6aad94e6, atime: 0x0, ctime: 0x6aad94e6, blocks: 0xc007 00010000:00010000:1.0:1789760743.102190:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2e6c2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760743.102208:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=722s v=(0 0) 00010000:00010000:1.0:1789760743.102213:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760743.102220:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2e6c2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24805376->24809471] (req 24805376->24809471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760743.102235:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2e6c2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24805376->24809471] (req 24805376->24809471) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760743.124042:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24805376 -> 24809472 00010000:00010000:1.0:1789760743.124062:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2e6c2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24805376->24809471] (req 24805376->24809471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760743.124075:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2e6c2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24805376->24809471] (req 24805376->24809471) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760743.124085:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2e6c2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24805376->24809471] (req 24805376->24809471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760743.124092:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760743.124097:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2e6c2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24805376->24809471] (req 24805376->24809471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760743.124106:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2e6c2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24805376->24809471] (req 24805376->24809471) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760743.124126:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2e6c2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24805376->24809471] (req 24805376->24809471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760743.124133:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2e6c2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24805376->24809471] (req 24805376->24809471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760743.124258:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac28a2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2360:x1876699512635776:12345-192.168.203.56@tcp:4:dd.0 Request processed in 22127us (22365us total) trans 4294973375 rc 0/0 00000100:00100000:1.0:1789760743.124269:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6086 00000100:00100000:1.0:1789760743.124278:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 722628711182 time 722666415544 state 0x2 ref 0 00000001:00080000:2.0:1789760743.124425:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973375 is committed 00000100:00100000:0.0:1789760743.131140:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760743.131264:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512635904 00000100:00100000:3.0:1789760743.131275:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512635904 00000100:00100000:3.0:1789760743.131341:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 722628711182 time 722666415544 state 0x0 ref 1 00000100:00100000:3.0:1789760743.131355:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 722666675960 time 722666415544 state 0x1 ref 1 00000100:00100000:3.0:1789760743.131381:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 722666675960 time 722695681140 state 0x0 ref 1 00000100:00100000:3.0:1789760743.131393:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2878000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512635904:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760743.131440:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24809472, mtime: 0x6aad94e6, atime: 0x0, ctime: 0x6aad94e6, blocks: 0xc007 00010000:00010000:3.0:1789760743.131449:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2e6c9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.131465:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=722s v=(0 0) 00010000:00010000:3.0:1789760743.131469:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760743.131476:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2e6c9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24809472->24813567] (req 24809472->24813567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.131494:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2e6c9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24809472->24813567] (req 24809472->24813567) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760743.172006:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973376 is committed 00002000:00010000:0.0:1789760743.172110:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24809472 -> 24813568 00010000:00010000:0.0:1789760743.172127:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2e6c9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24809472->24813567] (req 24809472->24813567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.172139:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2e6c9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24809472->24813567] (req 24809472->24813567) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.172150:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2e6c9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24809472->24813567] (req 24809472->24813567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.172158:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760743.172163:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2e6c9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24809472->24813567] (req 24809472->24813567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.172173:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2e6c9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24809472->24813567] (req 24809472->24813567) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.172191:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2e6c9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24809472->24813567] (req 24809472->24813567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.172208:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2e6c9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24809472->24813567] (req 24809472->24813567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760743.172297:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2878000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512635904:12345-192.168.203.56@tcp:4:dd.0 Request processed in 40901us (41162us total) trans 4294973376 rc 0/0 00000100:00100000:0.0:1789760743.172307:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6087 00000100:00100000:0.0:1789760743.172317:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 722666675960 time 722695681140 state 0x2 ref 0 00000100:00100000:1.0:1789760743.175272:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760743.178444:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512636032 00000100:00100000:2.0:1789760743.178458:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512636032 00000100:00100000:2.0:1789760743.178520:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 722666675960 time 722695681140 state 0x0 ref 1 00000100:00100000:2.0:1789760743.178537:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 722695941556 time 722695681140 state 0x1 ref 1 00000100:00100000:2.0:1789760743.178608:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 722695941556 time 722742907547 state 0x0 ref 1 00000100:00100000:2.0:1789760743.178623:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac437df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512636032:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760743.178668:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24813568, mtime: 0x6aad94e6, atime: 0x0, ctime: 0x6aad94e6, blocks: 0xc007 00010000:00010000:2.0:1789760743.178678:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2e6d0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.178693:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=722s v=(0 0) 00010000:00010000:2.0:1789760743.178713:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760743.178720:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2e6d0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24813568->24817663] (req 24813568->24817663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.178735:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2e6d0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24813568->24817663] (req 24813568->24817663) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760743.207752:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973377 is committed 00002000:00010000:3.0:1789760743.207962:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24813568 -> 24817664 00010000:00010000:3.0:1789760743.207980:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2e6d0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24813568->24817663] (req 24813568->24817663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.208061:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2e6d0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24813568->24817663] (req 24813568->24817663) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.208073:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2e6d0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24813568->24817663] (req 24813568->24817663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.208081:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760743.208086:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2e6d0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24813568->24817663] (req 24813568->24817663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.208101:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2e6d0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24813568->24817663] (req 24813568->24817663) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.208120:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2e6d0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24813568->24817663] (req 24813568->24817663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.208128:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2e6d0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24813568->24817663] (req 24813568->24817663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760743.208338:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac437df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512636032:12345-192.168.203.56@tcp:4:dd.0 Request processed in 29714us (33138us total) trans 4294973377 rc 0/0 00000100:00100000:3.0:1789760743.208348:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6088 00000100:00100000:3.0:1789760743.208359:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 722695941556 time 722742907547 state 0x2 ref 0 00000100:00100000:2.0:1789760743.212175:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760743.214273:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512636160 00000100:00100000:1.0:1789760743.214289:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512636160 00000100:00100000:1.0:1789760743.214392:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 722695941556 time 722742907547 state 0x0 ref 1 00000100:00100000:1.0:1789760743.214409:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 722743167963 time 722742907547 state 0x1 ref 1 00000100:00100000:1.0:1789760743.214449:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 722743167963 time 722778748422 state 0x0 ref 1 00000100:00100000:1.0:1789760743.214465:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac28a1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512636160:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760743.214514:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24817664, mtime: 0x6aad94e6, atime: 0x0, ctime: 0x6aad94e6, blocks: 0xc007 00010000:00010000:1.0:1789760743.214524:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2e6d7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760743.214545:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=722s v=(0 0) 00010000:00010000:1.0:1789760743.214549:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760743.214556:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2e6d7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24817664->24821759] (req 24817664->24821759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760743.214583:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2e6d7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24817664->24821759] (req 24817664->24821759) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760743.248977:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973378 is committed 00002000:00010000:3.0:1789760743.249103:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24817664 -> 24821760 00010000:00010000:3.0:1789760743.249118:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2e6d7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24817664->24821759] (req 24817664->24821759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.249131:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2e6d7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24817664->24821759] (req 24817664->24821759) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.249140:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2e6d7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24817664->24821759] (req 24817664->24821759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.249147:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760743.249153:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2e6d7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24817664->24821759] (req 24817664->24821759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.249161:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2e6d7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24817664->24821759] (req 24817664->24821759) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.249182:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2e6d7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24817664->24821759] (req 24817664->24821759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.249190:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2e6d7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24817664->24821759] (req 24817664->24821759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760743.249284:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac28a1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512636160:12345-192.168.203.56@tcp:4:dd.0 Request processed in 34819us (37115us total) trans 4294973378 rc 0/0 00000100:00100000:3.0:1789760743.249295:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6089 00000100:00100000:3.0:1789760743.249306:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 722743167963 time 722778748422 state 0x2 ref 0 00000100:00100000:0.0:1789760743.257967:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760743.258196:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512636288 00000100:00100000:3.0:1789760743.258206:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512636288 00000100:00100000:3.0:1789760743.258262:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 722743167963 time 722778748422 state 0x0 ref 1 00000100:00100000:3.0:1789760743.258277:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 722779008838 time 722778748422 state 0x1 ref 1 00000100:00100000:3.0:1789760743.258307:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 722779008838 time 722822606502 state 0x0 ref 1 00000100:00100000:3.0:1789760743.258319:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac287b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512636288:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760743.258376:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24821760, mtime: 0x6aad94e6, atime: 0x0, ctime: 0x6aad94e6, blocks: 0xc007 00010000:00010000:3.0:1789760743.258385:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2e6de lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.258402:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=722s v=(0 0) 00010000:00010000:3.0:1789760743.258406:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760743.258413:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2e6de lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24821760->24825855] (req 24821760->24825855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.258426:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2e6de lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24821760->24825855] (req 24821760->24825855) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760743.290249:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973379 is committed 00002000:00010000:0.0:1789760743.290357:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24821760 -> 24825856 00010000:00010000:0.0:1789760743.290372:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2e6de lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24821760->24825855] (req 24821760->24825855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.290388:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2e6de lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24821760->24825855] (req 24821760->24825855) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.290396:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2e6de lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24821760->24825855] (req 24821760->24825855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.290402:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760743.290406:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2e6de lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24821760->24825855] (req 24821760->24825855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.290413:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2e6de lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24821760->24825855] (req 24821760->24825855) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.290432:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2e6de lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24821760->24825855] (req 24821760->24825855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.290439:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2e6de lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24821760->24825855] (req 24821760->24825855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760743.290537:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac287b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512636288:12345-192.168.203.56@tcp:4:dd.0 Request processed in 32215us (32574us total) trans 4294973379 rc 0/0 00000100:00100000:0.0:1789760743.290547:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6090 00000100:00100000:0.0:1789760743.290557:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 722779008838 time 722822606502 state 0x2 ref 0 00000100:00100000:1.0:1789760743.295058:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760743.295334:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512636416 00000100:00100000:3.0:1789760743.295344:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512636416 00000100:00100000:3.0:1789760743.295401:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 722779008838 time 722822606502 state 0x0 ref 1 00000100:00100000:3.0:1789760743.295417:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 722822866918 time 722822606502 state 0x1 ref 1 00000100:00100000:3.0:1789760743.295478:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 722822866918 time 722859776501 state 0x0 ref 1 00000100:00100000:3.0:1789760743.295493:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac437e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512636416:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760743.295545:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24825856, mtime: 0x6aad94e6, atime: 0x0, ctime: 0x6aad94e6, blocks: 0xc007 00010000:00010000:3.0:1789760743.295554:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2e6e5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.295573:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=722s v=(0 0) 00010000:00010000:3.0:1789760743.295578:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760743.295584:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2e6e5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24825856->24829951] (req 24825856->24829951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.295600:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2e6e5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24825856->24829951] (req 24825856->24829951) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760743.332388:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973380 is committed 00002000:00010000:3.0:1789760743.332535:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24825856 -> 24829952 00010000:00010000:3.0:1789760743.332552:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2e6e5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24825856->24829951] (req 24825856->24829951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.332565:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2e6e5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24825856->24829951] (req 24825856->24829951) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.332574:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2e6e5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24825856->24829951] (req 24825856->24829951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.332582:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760743.332588:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2e6e5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24825856->24829951] (req 24825856->24829951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.332597:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2e6e5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24825856->24829951] (req 24825856->24829951) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.332617:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2e6e5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24825856->24829951] (req 24825856->24829951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.332624:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2e6e5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24825856->24829951] (req 24825856->24829951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760743.332867:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac437e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512636416:12345-192.168.203.56@tcp:4:dd.0 Request processed in 37372us (37870us total) trans 4294973380 rc 0/0 00000100:00100000:3.0:1789760743.332878:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6091 00000100:00100000:3.0:1789760743.332887:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 722822866918 time 722859776501 state 0x2 ref 0 00000100:00100000:2.0:1789760743.341896:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760743.341984:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512636544 00000100:00100000:3.0:1789760743.342001:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512636544 00000100:00100000:3.0:1789760743.342065:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 722822866918 time 722859776501 state 0x0 ref 1 00000100:00100000:3.0:1789760743.342080:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 722860036917 time 722859776501 state 0x1 ref 1 00000100:00100000:3.0:1789760743.342115:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 722860036917 time 722906414230 state 0x0 ref 1 00000100:00100000:3.0:1789760743.342128:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac28a3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512636544:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760743.342183:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24829952, mtime: 0x6aad94e6, atime: 0x0, ctime: 0x6aad94e6, blocks: 0xc007 00010000:00010000:3.0:1789760743.342193:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2e6ec lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.342210:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=722s v=(0 0) 00010000:00010000:3.0:1789760743.342215:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760743.342222:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2e6ec lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24829952->24834047] (req 24829952->24834047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.342236:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2e6ec lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24829952->24834047] (req 24829952->24834047) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760743.383407:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973381 is committed 00002000:00010000:3.0:1789760743.383539:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24829952 -> 24834048 00010000:00010000:3.0:1789760743.383554:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2e6ec lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24829952->24834047] (req 24829952->24834047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.383579:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2e6ec lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24829952->24834047] (req 24829952->24834047) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.383589:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2e6ec lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24829952->24834047] (req 24829952->24834047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.383596:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760743.383602:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2e6ec lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24829952->24834047] (req 24829952->24834047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.383611:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2e6ec lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24829952->24834047] (req 24829952->24834047) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.383630:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2e6ec lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24829952->24834047] (req 24829952->24834047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.383638:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2e6ec lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24829952->24834047] (req 24829952->24834047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760743.383761:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac28a3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512636544:12345-192.168.203.56@tcp:4:dd.0 Request processed in 41630us (41870us total) trans 4294973381 rc 0/0 00000100:00100000:3.0:1789760743.383776:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6092 00000100:00100000:3.0:1789760743.383785:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 722860036917 time 722906414230 state 0x2 ref 0 00000100:00100000:2.0:1789760743.386972:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760743.387085:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512636672 00000100:00100000:0.0:1789760743.387095:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512636672 00000100:00100000:0.0:1789760743.387153:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 722860036917 time 722906414230 state 0x0 ref 1 00000100:00100000:0.0:1789760743.387169:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 722906674646 time 722906414230 state 0x1 ref 1 00000100:00100000:0.0:1789760743.387318:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 722906674646 time 722951616594 state 0x0 ref 1 00000100:00100000:0.0:1789760743.387351:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febc7b33480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512636672:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760743.387404:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24834048, mtime: 0x6aad94e6, atime: 0x0, ctime: 0x6aad94e6, blocks: 0xc007 00010000:00010000:0.0:1789760743.387413:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2e6f3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.387429:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=722s v=(0 0) 00010000:00010000:0.0:1789760743.387433:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760743.387440:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2e6f3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24834048->24838143] (req 24834048->24838143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.387453:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2e6f3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24834048->24838143] (req 24834048->24838143) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760743.414979:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973382 is committed 00002000:00010000:3.0:1789760743.415276:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24834048 -> 24838144 00010000:00010000:3.0:1789760743.415292:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2e6f3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24834048->24838143] (req 24834048->24838143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.415304:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2e6f3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24834048->24838143] (req 24834048->24838143) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.415312:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2e6f3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24834048->24838143] (req 24834048->24838143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.415328:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760743.415334:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2e6f3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24834048->24838143] (req 24834048->24838143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.415342:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2e6f3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24834048->24838143] (req 24834048->24838143) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.415360:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2e6f3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24834048->24838143] (req 24834048->24838143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.415366:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2e6f3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24834048->24838143] (req 24834048->24838143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760743.415496:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febc7b33480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512636672:12345-192.168.203.56@tcp:4:dd.0 Request processed in 28144us (28528us total) trans 4294973382 rc 0/0 00000100:00100000:3.0:1789760743.415505:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6093 00000100:00100000:3.0:1789760743.415514:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 722906674646 time 722951616594 state 0x2 ref 0 00000100:00100000:3.0:1789760743.419507:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760743.421289:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512636800 00000100:00100000:0.0:1789760743.421300:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512636800 00000100:00100000:1.0:1789760743.421450:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 722906674646 time 722951616594 state 0x0 ref 1 00000100:00100000:1.0:1789760743.421465:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 722951877010 time 722951616594 state 0x1 ref 1 00000100:00100000:1.0:1789760743.421532:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 722951877010 time 722985830661 state 0x0 ref 1 00000100:00100000:1.0:1789760743.421545:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5304700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512636800:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760743.421598:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24838144, mtime: 0x6aad94e6, atime: 0x0, ctime: 0x6aad94e6, blocks: 0xc007 00010000:00010000:1.0:1789760743.421608:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2e6fa lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760743.421629:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=722s v=(0 0) 00010000:00010000:1.0:1789760743.421633:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760743.421640:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2e6fa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24838144->24842239] (req 24838144->24842239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760743.421652:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2e6fa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24838144->24842239] (req 24838144->24842239) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760743.457205:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973383 is committed 00002000:00010000:3.0:1789760743.457391:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24838144 -> 24842240 00010000:00010000:3.0:1789760743.457406:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2e6fa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24838144->24842239] (req 24838144->24842239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.457418:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2e6fa lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24838144->24842239] (req 24838144->24842239) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.457427:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2e6fa lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24838144->24842239] (req 24838144->24842239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.457433:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760743.457438:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2e6fa lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24838144->24842239] (req 24838144->24842239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.457447:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2e6fa lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24838144->24842239] (req 24838144->24842239) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.457467:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2e6fa lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24838144->24842239] (req 24838144->24842239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.457479:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2e6fa lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24838144->24842239] (req 24838144->24842239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760743.457570:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5304700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512636800:12345-192.168.203.56@tcp:4:dd.0 Request processed in 36023us (38067us total) trans 4294973383 rc 0/0 00000100:00100000:3.0:1789760743.457580:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6094 00000100:00100000:3.0:1789760743.457590:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 722951877010 time 722985830661 state 0x2 ref 0 00000100:00100000:2.0:1789760743.462653:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760743.462926:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512636928 00000100:00100000:3.0:1789760743.462936:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512636928 00000100:00100000:2.0:1789760743.463123:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 722951877010 time 722985830661 state 0x0 ref 1 00000100:00100000:2.0:1789760743.463138:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 722986091077 time 722985830661 state 0x1 ref 1 00000100:00100000:2.0:1789760743.463172:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 722986091077 time 723027471315 state 0x0 ref 1 00000100:00100000:2.0:1789760743.463185:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febc7b32680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512636928:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760743.463238:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24842240, mtime: 0x6aad94e6, atime: 0x0, ctime: 0x6aad94e6, blocks: 0xc007 00010000:00010000:2.0:1789760743.463518:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2e701 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.463536:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=723s v=(0 0) 00010000:00010000:2.0:1789760743.463541:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760743.463548:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2e701 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24842240->24846335] (req 24842240->24846335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.463563:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2e701 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24842240->24846335] (req 24842240->24846335) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760743.494176:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24842240 -> 24846336 00010000:00010000:3.0:1789760743.494195:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2e701 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24842240->24846335] (req 24842240->24846335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.494206:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2e701 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24842240->24846335] (req 24842240->24846335) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.494215:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2e701 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24842240->24846335] (req 24842240->24846335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.494222:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760743.494227:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2e701 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24842240->24846335] (req 24842240->24846335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.494236:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2e701 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24842240->24846335] (req 24842240->24846335) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.494257:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2e701 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24842240->24846335] (req 24842240->24846335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.494265:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2e701 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24842240->24846335] (req 24842240->24846335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760743.494379:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febc7b32680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2361:x1876699512636928:12345-192.168.203.56@tcp:4:dd.0 Request processed in 31188us (31727us total) trans 4294973384 rc 0/0 00000100:00100000:3.0:1789760743.494390:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6095 00000100:00100000:3.0:1789760743.494399:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 722986091077 time 723027471315 state 0x2 ref 0 00000001:00080000:1.0:1789760743.494945:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973384 is committed 00000100:00100000:2.0:1789760743.498196:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760743.498279:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512637056 00000100:00100000:3.0:1789760743.498288:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512637056 00000100:00100000:3.0:1789760743.498499:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 722986091077 time 723027471315 state 0x0 ref 1 00000100:00100000:3.0:1789760743.498514:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 723027731731 time 723027471315 state 0x1 ref 1 00000100:00100000:3.0:1789760743.498548:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 723027731731 time 723062847472 state 0x0 ref 1 00000100:00100000:3.0:1789760743.498561:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febea899180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512637056:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760743.498607:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24846336, mtime: 0x6aad94e6, atime: 0x0, ctime: 0x6aad94e6, blocks: 0xc007 00010000:00010000:3.0:1789760743.498616:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2e708 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.498632:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=723s v=(0 0) 00010000:00010000:3.0:1789760743.498638:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760743.498644:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2e708 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24846336->24850431] (req 24846336->24850431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.498657:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2e708 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24846336->24850431] (req 24846336->24850431) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760743.535629:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973385 is committed 00002000:00010000:2.0:1789760743.535856:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24846336 -> 24850432 00002000:00010000:2.0:1789760743.535861:0:6709:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760742 -> 1789760743 00002000:00010000:2.0:1789760743.535864:0:6709:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760742 -> 1789760743 00010000:00010000:2.0:1789760743.535877:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2e708 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24846336->24850431] (req 24846336->24850431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.535891:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2e708 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24846336->24850431] (req 24846336->24850431) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.535900:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2e708 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24846336->24850431] (req 24846336->24850431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.535907:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760743.535913:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2e708 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24846336->24850431] (req 24846336->24850431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.535922:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2e708 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24846336->24850431] (req 24846336->24850431) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.535946:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2e708 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24846336->24850431] (req 24846336->24850431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.535954:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2e708 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24846336->24850431] (req 24846336->24850431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760743.536109:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febea899180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512637056:12345-192.168.203.56@tcp:4:dd.0 Request processed in 37544us (37917us total) trans 4294973385 rc 0/0 00000100:00100000:2.0:1789760743.536121:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6096 00000100:00100000:2.0:1789760743.536133:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 723027731731 time 723062847472 state 0x2 ref 0 00000100:00100000:2.0:1789760743.539943:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760743.540012:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512637184 00000100:00100000:3.0:1789760743.540022:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512637184 00000100:00100000:3.0:1789760743.540096:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 723027731731 time 723062847472 state 0x0 ref 1 00000100:00100000:3.0:1789760743.540111:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 723063107888 time 723062847472 state 0x1 ref 1 00000100:00100000:3.0:1789760743.540133:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 723063107888 time 723104432999 state 0x0 ref 1 00000100:00100000:3.0:1789760743.540146:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febea89b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512637184:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760743.540197:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24850432, mtime: 0x6aad94e7, atime: 0x0, ctime: 0x6aad94e7, blocks: 0xc007 00010000:00010000:3.0:1789760743.540205:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2e70f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.540221:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=723s v=(0 0) 00010000:00010000:3.0:1789760743.540226:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760743.540232:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2e70f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24850432->24854527] (req 24850432->24854527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.540244:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2e70f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24850432->24854527] (req 24850432->24854527) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760743.568047:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24850432 -> 24854528 00010000:00010000:1.0:1789760743.568067:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2e70f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24850432->24854527] (req 24850432->24854527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760743.568082:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2e70f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24850432->24854527] (req 24850432->24854527) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760743.568091:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2e70f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24850432->24854527] (req 24850432->24854527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760743.568098:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760743.568103:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2e70f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24850432->24854527] (req 24850432->24854527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760743.568112:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2e70f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24850432->24854527] (req 24850432->24854527) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760743.568136:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2e70f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24850432->24854527] (req 24850432->24854527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760743.568144:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2e70f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24850432->24854527] (req 24850432->24854527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760743.568240:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febea89b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2358:x1876699512637184:12345-192.168.203.56@tcp:4:dd.0 Request processed in 28091us (28300us total) trans 4294973386 rc 0/0 00000100:00100000:1.0:1789760743.568250:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6097 00000100:00100000:1.0:1789760743.568259:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 723063107888 time 723104432999 state 0x2 ref 0 00000001:00080000:2.0:1789760743.569101:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973386 is committed 00000100:00100000:3.0:1789760743.572409:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760743.572553:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512637312 00000100:00100000:2.0:1789760743.572562:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512637312 00000100:00100000:2.0:1789760743.572607:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 723063107888 time 723104432999 state 0x0 ref 1 00000100:00100000:2.0:1789760743.572621:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 723104693415 time 723104432999 state 0x1 ref 1 00000100:00100000:2.0:1789760743.572649:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 723104693415 time 723136949413 state 0x0 ref 1 00000100:00100000:2.0:1789760743.572660:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5306a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512637312:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760743.572740:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24854528, mtime: 0x6aad94e7, atime: 0x0, ctime: 0x6aad94e7, blocks: 0xc007 00010000:00010000:2.0:1789760743.572749:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2e716 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.572770:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=723s v=(0 0) 00010000:00010000:2.0:1789760743.572774:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760743.572780:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2e716 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24854528->24858623] (req 24854528->24858623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.572794:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2e716 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24854528->24858623] (req 24854528->24858623) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760743.599090:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973387 is committed 00002000:00010000:3.0:1789760743.599106:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24854528 -> 24858624 00010000:00010000:3.0:1789760743.599121:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2e716 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24854528->24858623] (req 24854528->24858623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.599426:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2e716 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24854528->24858623] (req 24854528->24858623) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.599438:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2e716 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24854528->24858623] (req 24854528->24858623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.599446:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760743.599450:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2e716 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24854528->24858623] (req 24854528->24858623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.599460:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2e716 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24854528->24858623] (req 24854528->24858623) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.599480:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2e716 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24854528->24858623] (req 24854528->24858623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.599488:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2e716 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24854528->24858623] (req 24854528->24858623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760743.599581:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5306a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512637312:12345-192.168.203.56@tcp:4:dd.0 Request processed in 26916us (27174us total) trans 4294973387 rc 0/0 00000100:00100000:3.0:1789760743.599592:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6098 00000100:00100000:3.0:1789760743.599601:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 723104693415 time 723136949413 state 0x2 ref 0 00000100:00100000:3.0:1789760743.604138:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760743.604394:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512637440 00000100:00100000:2.0:1789760743.604404:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512637440 00000100:00100000:2.0:1789760743.604456:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 723104693415 time 723136949413 state 0x0 ref 1 00000100:00100000:2.0:1789760743.604471:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 723137209829 time 723136949413 state 0x1 ref 1 00000100:00100000:2.0:1789760743.604503:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 723137209829 time 723168803140 state 0x0 ref 1 00000100:00100000:2.0:1789760743.604515:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5305880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512637440:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760743.604563:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24858624, mtime: 0x6aad94e7, atime: 0x0, ctime: 0x6aad94e7, blocks: 0xc007 00010000:00010000:2.0:1789760743.604571:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2e71d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.604586:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=723s v=(0 0) 00010000:00010000:2.0:1789760743.604591:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760743.604597:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2e71d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24858624->24862719] (req 24858624->24862719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.604617:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2e71d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24858624->24862719] (req 24858624->24862719) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760743.634275:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973388 is committed 00002000:00010000:3.0:1789760743.634511:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24858624 -> 24862720 00010000:00010000:3.0:1789760743.634527:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2e71d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24858624->24862719] (req 24858624->24862719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.634539:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2e71d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24858624->24862719] (req 24858624->24862719) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.634548:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2e71d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24858624->24862719] (req 24858624->24862719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.634555:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760743.634560:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2e71d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24858624->24862719] (req 24858624->24862719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.634568:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2e71d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24858624->24862719] (req 24858624->24862719) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.634587:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2e71d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24858624->24862719] (req 24858624->24862719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.634594:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2e71d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24858624->24862719] (req 24858624->24862719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760743.634743:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5305880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512637440:12345-192.168.203.56@tcp:4:dd.0 Request processed in 30180us (30565us total) trans 4294973388 rc 0/0 00000100:00100000:3.0:1789760743.634753:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6099 00000100:00100000:3.0:1789760743.634762:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 723137209829 time 723168803140 state 0x2 ref 0 00000100:00100000:2.0:1789760743.639257:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760743.642251:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512637568 00000100:00100000:1.0:1789760743.642266:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512637568 00000100:00100000:1.0:1789760743.642345:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 723137209829 time 723168803140 state 0x0 ref 1 00000100:00100000:1.0:1789760743.642360:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 723169063556 time 723168803140 state 0x1 ref 1 00000100:00100000:1.0:1789760743.642392:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 723169063556 time 723206692055 state 0x0 ref 1 00000100:00100000:1.0:1789760743.642404:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febea89b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512637568:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760743.642463:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24862720, mtime: 0x6aad94e7, atime: 0x0, ctime: 0x6aad94e7, blocks: 0xc007 00010000:00010000:1.0:1789760743.642471:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2e724 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760743.642490:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=723s v=(0 0) 00010000:00010000:1.0:1789760743.642494:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760743.642500:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2e724 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24862720->24866815] (req 24862720->24866815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760743.642514:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2e724 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24862720->24866815] (req 24862720->24866815) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760743.675434:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24862720 -> 24866816 00010000:00010000:3.0:1789760743.675452:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2e724 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24862720->24866815] (req 24862720->24866815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760743.675456:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973389 is committed 00010000:00010000:3.0:1789760743.675469:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2e724 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24862720->24866815] (req 24862720->24866815) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.675479:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2e724 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24862720->24866815] (req 24862720->24866815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.675486:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760743.675491:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2e724 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24862720->24866815] (req 24862720->24866815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.675501:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2e724 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24862720->24866815] (req 24862720->24866815) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.675521:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2e724 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24862720->24866815] (req 24862720->24866815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.675533:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2e724 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24862720->24866815] (req 24862720->24866815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760743.675635:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febea89b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512637568:12345-192.168.203.56@tcp:4:dd.0 Request processed in 33228us (36381us total) trans 4294973389 rc 0/0 00000100:00100000:3.0:1789760743.675645:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6100 00000100:00100000:3.0:1789760743.675655:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 723169063556 time 723206692055 state 0x2 ref 0 00000100:00100000:0.0:1789760743.684921:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760743.685122:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512637696 00000100:00100000:3.0:1789760743.685132:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512637696 00000100:00100000:3.0:1789760743.685186:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 723169063556 time 723206692055 state 0x0 ref 1 00000100:00100000:3.0:1789760743.685200:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 723206952471 time 723206692055 state 0x1 ref 1 00000100:00100000:3.0:1789760743.685275:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 723206952471 time 723249572767 state 0x0 ref 1 00000100:00100000:3.0:1789760743.685289:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2878700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512637696:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760743.685350:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24866816, mtime: 0x6aad94e7, atime: 0x0, ctime: 0x6aad94e7, blocks: 0xc007 00010000:00010000:3.0:1789760743.685359:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2e72b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.685375:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=723s v=(0 0) 00010000:00010000:3.0:1789760743.685380:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760743.685386:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2e72b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24866816->24870911] (req 24866816->24870911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.685405:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2e72b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24866816->24870911] (req 24866816->24870911) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760743.717666:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973390 is committed 00002000:00010000:3.0:1789760743.717784:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24866816 -> 24870912 00010000:00010000:3.0:1789760743.717798:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2e72b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24866816->24870911] (req 24866816->24870911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.717811:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2e72b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24866816->24870911] (req 24866816->24870911) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.717820:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2e72b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24866816->24870911] (req 24866816->24870911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.717828:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760743.717832:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2e72b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24866816->24870911] (req 24866816->24870911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.717842:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2e72b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24866816->24870911] (req 24866816->24870911) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.717860:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2e72b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24866816->24870911] (req 24866816->24870911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.717868:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2e72b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24866816->24870911] (req 24866816->24870911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760743.717999:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2878700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512637696:12345-192.168.203.56@tcp:4:dd.0 Request processed in 32702us (33076us total) trans 4294973390 rc 0/0 00000100:00100000:3.0:1789760743.718009:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6101 00000100:00100000:3.0:1789760743.718017:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 723206952471 time 723249572767 state 0x2 ref 0 00000100:00100000:2.0:1789760743.722904:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760743.726535:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512637824 00000100:00100000:3.0:1789760743.726550:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512637824 00000100:00100000:3.0:1789760743.726613:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 723206952471 time 723249572767 state 0x0 ref 1 00000100:00100000:3.0:1789760743.726628:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 723249833183 time 723249572767 state 0x1 ref 1 00000100:00100000:3.0:1789760743.726651:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 723249833183 time 723290951245 state 0x0 ref 1 00000100:00100000:3.0:1789760743.726664:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febea89aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512637824:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760743.730669:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24870912, mtime: 0x6aad94e7, atime: 0x0, ctime: 0x6aad94e7, blocks: 0xc007 00010000:00010000:2.0:1789760743.730684:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2e732 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.730734:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=723s v=(0 0) 00010000:00010000:2.0:1789760743.730739:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760743.730746:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2e732 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24870912->24875007] (req 24870912->24875007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.730762:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2e732 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24870912->24875007] (req 24870912->24875007) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760743.773356:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973391 is committed 00002000:00010000:2.0:1789760743.773459:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24870912 -> 24875008 00010000:00010000:2.0:1789760743.773475:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2e732 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24870912->24875007] (req 24870912->24875007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.773488:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2e732 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24870912->24875007] (req 24870912->24875007) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.773503:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2e732 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24870912->24875007] (req 24870912->24875007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.773511:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760743.773516:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2e732 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24870912->24875007] (req 24870912->24875007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.773525:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2e732 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24870912->24875007] (req 24870912->24875007) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.773543:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2e732 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24870912->24875007] (req 24870912->24875007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.773551:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2e732 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24870912->24875007] (req 24870912->24875007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760743.773641:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febea89aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512637824:12345-192.168.203.56@tcp:4:dd.0 Request processed in 46974us (50741us total) trans 4294973391 rc 0/0 00000100:00100000:2.0:1789760743.773651:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6102 00000100:00100000:2.0:1789760743.773660:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 723249833183 time 723290951245 state 0x2 ref 0 00000100:00100000:2.0:1789760743.778210:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760743.778289:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512637952 00000100:00100000:3.0:1789760743.778300:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512637952 00000100:00100000:3.0:1789760743.778366:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 723249833183 time 723290951245 state 0x0 ref 1 00000100:00100000:3.0:1789760743.778381:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 723291211661 time 723290951245 state 0x1 ref 1 00000100:00100000:3.0:1789760743.778413:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 723291211661 time 723342712562 state 0x0 ref 1 00000100:00100000:3.0:1789760743.778426:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febffb9f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512637952:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760743.778474:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24875008, mtime: 0x6aad94e7, atime: 0x0, ctime: 0x6aad94e7, blocks: 0xc007 00010000:00010000:3.0:1789760743.778484:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2e739 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.778499:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=723s v=(0 0) 00010000:00010000:3.0:1789760743.778504:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760743.778510:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2e739 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24875008->24879103] (req 24875008->24879103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.778524:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2e739 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24875008->24879103] (req 24875008->24879103) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.797376:0:7647:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:0.0:1789760743.809056:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973392 is committed 00002000:00010000:3.0:1789760743.809391:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24875008 -> 24879104 00010000:00010000:3.0:1789760743.809407:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2e739 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24875008->24879103] (req 24875008->24879103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.809420:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2e739 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24875008->24879103] (req 24875008->24879103) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.809429:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2e739 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24875008->24879103] (req 24875008->24879103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.809438:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760743.809443:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2e739 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24875008->24879103] (req 24875008->24879103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.809453:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2e739 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24875008->24879103] (req 24875008->24879103) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.809477:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2e739 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24875008->24879103] (req 24875008->24879103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.809484:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2e739 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24875008->24879103] (req 24875008->24879103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760743.809577:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febffb9f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512637952:12345-192.168.203.56@tcp:4:dd.0 Request processed in 31148us (31373us total) trans 4294973392 rc 0/0 00000100:00100000:3.0:1789760743.809587:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6103 00000100:00100000:3.0:1789760743.809597:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 723291211661 time 723342712562 state 0x2 ref 0 00000100:00100000:0.0:1789760743.815829:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760743.815957:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512638080 00000100:00100000:3.0:1789760743.815967:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512638080 00000100:00100000:3.0:1789760743.816028:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 723291211661 time 723342712562 state 0x0 ref 1 00000100:00100000:3.0:1789760743.816044:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 723342972978 time 723342712562 state 0x1 ref 1 00000100:00100000:3.0:1789760743.816071:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 723342972978 time 723380370685 state 0x0 ref 1 00000100:00100000:3.0:1789760743.816083:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac287aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512638080:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760743.816127:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24879104, mtime: 0x6aad94e7, atime: 0x0, ctime: 0x6aad94e7, blocks: 0xc007 00010000:00010000:3.0:1789760743.816136:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2e740 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.816151:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=723s v=(0 0) 00010000:00010000:3.0:1789760743.816156:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760743.816162:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2e740 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24879104->24883199] (req 24879104->24883199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.816176:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2e740 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24879104->24883199] (req 24879104->24883199) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760743.850303:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973393 is committed 00002000:00010000:2.0:1789760743.850395:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24879104 -> 24883200 00010000:00010000:2.0:1789760743.850412:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2e740 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24879104->24883199] (req 24879104->24883199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.850426:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2e740 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24879104->24883199] (req 24879104->24883199) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.850435:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2e740 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24879104->24883199] (req 24879104->24883199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.850443:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760743.850448:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2e740 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24879104->24883199] (req 24879104->24883199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.850456:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2e740 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24879104->24883199] (req 24879104->24883199) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.850475:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2e740 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24879104->24883199] (req 24879104->24883199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.850482:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2e740 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24879104->24883199] (req 24879104->24883199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760743.850614:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac287aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512638080:12345-192.168.203.56@tcp:4:dd.0 Request processed in 34528us (34789us total) trans 4294973393 rc 0/0 00000100:00100000:2.0:1789760743.850624:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6104 00000100:00100000:2.0:1789760743.850633:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 723342972978 time 723380370685 state 0x2 ref 0 00000100:00100000:0.0:1789760743.854473:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760743.857892:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512638208 00000100:00100000:2.0:1789760743.857909:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512638208 00000100:00100000:0.0:1789760743.858106:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 723342972978 time 723380370685 state 0x0 ref 1 00000100:00100000:0.0:1789760743.858124:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 723380631101 time 723380370685 state 0x1 ref 1 00000100:00100000:0.0:1789760743.858186:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 723380631101 time 723422485809 state 0x0 ref 1 00000100:00100000:0.0:1789760743.858204:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2879880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512638208:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760743.858251:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24883200, mtime: 0x6aad94e7, atime: 0x0, ctime: 0x6aad94e7, blocks: 0xc007 00010000:00010000:0.0:1789760743.858260:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2e747 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.858276:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=723s v=(0 0) 00010000:00010000:0.0:1789760743.858281:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760743.858298:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2e747 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24883200->24887295] (req 24883200->24887295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.858312:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2e747 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24883200->24887295] (req 24883200->24887295) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760743.888661:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973394 is committed 00002000:00010000:2.0:1789760743.888724:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24883200 -> 24887296 00010000:00010000:2.0:1789760743.888743:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2e747 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24883200->24887295] (req 24883200->24887295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.888757:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2e747 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24883200->24887295] (req 24883200->24887295) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.888993:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2e747 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24883200->24887295] (req 24883200->24887295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.889001:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760743.889006:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2e747 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24883200->24887295] (req 24883200->24887295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.889015:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2e747 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24883200->24887295] (req 24883200->24887295) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.889036:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2e747 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24883200->24887295] (req 24883200->24887295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.889043:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2e747 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24883200->24887295] (req 24883200->24887295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760743.889130:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2879880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512638208:12345-192.168.203.56@tcp:4:dd.0 Request processed in 30929us (34662us total) trans 4294973394 rc 0/0 00000100:00100000:2.0:1789760743.889140:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6105 00000100:00100000:2.0:1789760743.889149:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 723380631101 time 723422485809 state 0x2 ref 0 00000100:00100000:0.0:1789760743.891915:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760743.892000:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512638336 00000100:00100000:3.0:1789760743.892010:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512638336 00000100:00100000:3.0:1789760743.892068:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 723380631101 time 723422485809 state 0x0 ref 1 00000100:00100000:3.0:1789760743.892084:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 723422746225 time 723422485809 state 0x1 ref 1 00000100:00100000:2.0:1789760743.892234:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 723422746225 time 723456530459 state 0x0 ref 1 00000100:00100000:2.0:1789760743.892251:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2879180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512638336:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760743.892302:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24887296, mtime: 0x6aad94e7, atime: 0x0, ctime: 0x6aad94e7, blocks: 0xc007 00010000:00010000:2.0:1789760743.892311:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2e74e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.892411:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=723s v=(0 0) 00010000:00010000:2.0:1789760743.892415:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760743.892421:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2e74e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24887296->24891391] (req 24887296->24891391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.892435:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2e74e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24887296->24891391] (req 24887296->24891391) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760743.919841:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973395 is committed 00002000:00010000:0.0:1789760743.920182:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24887296 -> 24891392 00010000:00010000:0.0:1789760743.920197:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2e74e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24887296->24891391] (req 24887296->24891391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.920210:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2e74e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24887296->24891391] (req 24887296->24891391) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.920219:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2e74e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24887296->24891391] (req 24887296->24891391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.920226:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760743.920231:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2e74e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24887296->24891391] (req 24887296->24891391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.920240:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2e74e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24887296->24891391] (req 24887296->24891391) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.920257:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2e74e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24887296->24891391] (req 24887296->24891391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.920264:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2e74e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24887296->24891391] (req 24887296->24891391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760743.920363:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2879180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512638336:12345-192.168.203.56@tcp:4:dd.0 Request processed in 28111us (28453us total) trans 4294973395 rc 0/0 00000100:00100000:0.0:1789760743.920373:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6106 00000100:00100000:0.0:1789760743.920383:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 723422746225 time 723456530459 state 0x2 ref 0 00000100:00100000:0.0:1789760743.926271:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760743.926389:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512638464 00000100:00100000:2.0:1789760743.926399:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512638464 00000100:00100000:2.0:1789760743.926453:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 723422746225 time 723456530459 state 0x0 ref 1 00000100:00100000:2.0:1789760743.926468:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 723456790875 time 723456530459 state 0x1 ref 1 00000100:00100000:2.0:1789760743.926537:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 723456790875 time 723490836459 state 0x0 ref 1 00000100:00100000:2.0:1789760743.926550:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2879500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512638464:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760743.926605:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24891392, mtime: 0x6aad94e7, atime: 0x0, ctime: 0x6aad94e7, blocks: 0xc007 00010000:00010000:2.0:1789760743.926614:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2e755 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.926631:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=723s v=(0 0) 00010000:00010000:2.0:1789760743.926636:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760743.926643:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2e755 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24891392->24895487] (req 24891392->24895487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.926657:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2e755 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24891392->24895487] (req 24891392->24895487) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760743.955622:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24891392 -> 24895488 00000001:00080000:0.0:1789760743.955623:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973396 is committed 00010000:00010000:3.0:1789760743.955640:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2e755 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24891392->24895487] (req 24891392->24895487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.955653:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2e755 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24891392->24895487] (req 24891392->24895487) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.955663:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2e755 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24891392->24895487] (req 24891392->24895487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.955670:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760743.955675:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2e755 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24891392->24895487] (req 24891392->24895487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.955690:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2e755 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24891392->24895487] (req 24891392->24895487) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.955738:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2e755 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24891392->24895487] (req 24891392->24895487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.955746:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2e755 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24891392->24895487] (req 24891392->24895487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760743.955926:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2879500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512638464:12345-192.168.203.56@tcp:4:dd.0 Request processed in 29373us (29657us total) trans 4294973396 rc 0/0 00000100:00100000:3.0:1789760743.955936:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6107 00000100:00100000:3.0:1789760743.955946:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 723456790875 time 723490836459 state 0x2 ref 0 00000100:00100000:3.0:1789760743.959206:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760743.959483:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512638592 00000100:00100000:2.0:1789760743.959494:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512638592 00000100:00100000:2.0:1789760743.959549:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 723456790875 time 723490836459 state 0x0 ref 1 00000100:00100000:2.0:1789760743.959564:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 723491096875 time 723490836459 state 0x1 ref 1 00000100:00100000:2.0:1789760743.959588:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 723491096875 time 723523888510 state 0x0 ref 1 00000100:00100000:2.0:1789760743.959599:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5304a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512638592:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760743.959647:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24895488, mtime: 0x6aad94e7, atime: 0x0, ctime: 0x6aad94e7, blocks: 0xc007 00010000:00010000:2.0:1789760743.959657:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2e75c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.959674:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=723s v=(0 0) 00010000:00010000:2.0:1789760743.959678:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760743.959684:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2e75c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24895488->24899583] (req 24895488->24899583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.959731:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2e75c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24895488->24899583] (req 24895488->24899583) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760743.990430:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973397 is committed 00002000:00010000:0.0:1789760743.990608:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24895488 -> 24899584 00010000:00010000:0.0:1789760743.990623:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2e75c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24895488->24899583] (req 24895488->24899583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.990636:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2e75c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24895488->24899583] (req 24895488->24899583) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.990647:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2e75c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24895488->24899583] (req 24895488->24899583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.990655:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760743.990661:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2e75c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24895488->24899583] (req 24895488->24899583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.990670:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2e75c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24895488->24899583] (req 24895488->24899583) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.990689:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2e75c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24895488->24899583] (req 24895488->24899583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.990728:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2e75c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24895488->24899583] (req 24895488->24899583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760743.990822:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5304a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512638592:12345-192.168.203.56@tcp:4:dd.0 Request processed in 31219us (31658us total) trans 4294973397 rc 0/0 00000100:00100000:0.0:1789760743.990832:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6108 00000100:00100000:0.0:1789760743.990842:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 723491096875 time 723523888510 state 0x2 ref 0 00000100:00100000:3.0:1789760743.993928:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760743.994049:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512638720 00000100:00100000:0.0:1789760743.994059:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512638720 00000100:00100000:0.0:1789760743.994176:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 723491096875 time 723523888510 state 0x0 ref 1 00000100:00100000:0.0:1789760743.994193:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 723524148926 time 723523888510 state 0x1 ref 1 00000100:00100000:0.0:1789760743.994223:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 723524148926 time 723558522324 state 0x0 ref 1 00000100:00100000:0.0:1789760743.994235:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5306d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512638720:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760743.994281:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24899584, mtime: 0x6aad94e7, atime: 0x0, ctime: 0x6aad94e7, blocks: 0xc007 00010000:00010000:0.0:1789760743.994291:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2e763 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.994449:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=723s v=(0 0) 00010000:00010000:0.0:1789760743.994455:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760743.994461:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2e763 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24899584->24903679] (req 24899584->24903679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.994476:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2e763 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24899584->24903679] (req 24899584->24903679) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760744.044095:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973398 is committed 00002000:00010000:0.0:1789760744.045143:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24899584 -> 24903680 00010000:00010000:0.0:1789760744.045159:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2e763 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24899584->24903679] (req 24899584->24903679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.045171:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2e763 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24899584->24903679] (req 24899584->24903679) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.045181:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2e763 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24899584->24903679] (req 24899584->24903679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.045189:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760744.045193:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2e763 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24899584->24903679] (req 24899584->24903679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.045202:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2e763 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24899584->24903679] (req 24899584->24903679) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.045220:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2e763 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24899584->24903679] (req 24899584->24903679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.045229:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2e763 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24899584->24903679] (req 24899584->24903679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760744.045329:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5306d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512638720:12345-192.168.203.56@tcp:4:dd.0 Request processed in 51084us (51456us total) trans 4294973398 rc 0/0 00000100:00100000:0.0:1789760744.045339:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6109 00000100:00100000:0.0:1789760744.045349:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 723524148926 time 723558522324 state 0x2 ref 0 00000100:00100000:3.0:1789760744.049279:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760744.051914:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512638848 00000100:00100000:2.0:1789760744.051931:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512638848 00000100:00100000:0.0:1789760744.052047:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 723524148926 time 723558522324 state 0x0 ref 1 00000100:00100000:0.0:1789760744.052065:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 723558782740 time 723558522324 state 0x1 ref 1 00000100:00100000:0.0:1789760744.052154:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 723558782740 time 723616452610 state 0x0 ref 1 00000100:00100000:0.0:1789760744.052168:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5306300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512638848:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760744.052221:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24903680, mtime: 0x6aad94e7, atime: 0x0, ctime: 0x6aad94e7, blocks: 0xc007 00010000:00010000:0.0:1789760744.052231:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2e76a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.052248:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=723s v=(0 0) 00010000:00010000:0.0:1789760744.052253:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760744.052259:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2e76a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24903680->24907775] (req 24903680->24907775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.052273:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2e76a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24903680->24907775] (req 24903680->24907775) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760744.118305:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973399 is committed 00002000:00010000:0.0:1789760744.118418:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24903680 -> 24907776 00010000:00010000:0.0:1789760744.118434:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2e76a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24903680->24907775] (req 24903680->24907775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.118447:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2e76a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24903680->24907775] (req 24903680->24907775) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.118457:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2e76a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24903680->24907775] (req 24903680->24907775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.118465:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760744.118470:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2e76a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24903680->24907775] (req 24903680->24907775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.118478:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2e76a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24903680->24907775] (req 24903680->24907775) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.118496:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2e76a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24903680->24907775] (req 24903680->24907775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.118514:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2e76a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24903680->24907775] (req 24903680->24907775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760744.118606:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5306300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512638848:12345-192.168.203.56@tcp:4:dd.0 Request processed in 66437us (69333us total) trans 4294973399 rc 0/0 00000100:00100000:0.0:1789760744.118617:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6110 00000100:00100000:0.0:1789760744.118626:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 723558782740 time 723616452610 state 0x2 ref 0 00000100:00100000:3.0:1789760744.126917:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760744.127023:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512638976 00000100:00100000:0.0:1789760744.127033:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512638976 00000100:00100000:0.0:1789760744.127087:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 723558782740 time 723616452610 state 0x0 ref 1 00000100:00100000:0.0:1789760744.127102:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 723616713026 time 723616452610 state 0x1 ref 1 00000100:00100000:0.0:1789760744.127134:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 723616713026 time 723691433305 state 0x0 ref 1 00000100:00100000:0.0:1789760744.127146:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5307800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512638976:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760744.127196:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24907776, mtime: 0x6aad94e7, atime: 0x0, ctime: 0x6aad94e7, blocks: 0xc007 00010000:00010000:0.0:1789760744.127205:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2e771 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.127221:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=723s v=(0 0) 00010000:00010000:0.0:1789760744.127225:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760744.127232:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2e771 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24907776->24911871] (req 24907776->24911871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.127251:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2e771 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24907776->24911871] (req 24907776->24911871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760744.156350:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973400 is committed 00002000:00010000:0.0:1789760744.156430:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24907776 -> 24911872 00010000:00010000:0.0:1789760744.156445:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2e771 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24907776->24911871] (req 24907776->24911871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.156458:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2e771 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24907776->24911871] (req 24907776->24911871) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.156467:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2e771 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24907776->24911871] (req 24907776->24911871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.156474:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760744.156479:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2e771 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24907776->24911871] (req 24907776->24911871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.156488:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2e771 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24907776->24911871] (req 24907776->24911871) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.156574:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2e771 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24907776->24911871] (req 24907776->24911871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.156583:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2e771 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24907776->24911871] (req 24907776->24911871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760744.156731:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5307800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512638976:12345-192.168.203.56@tcp:4:dd.0 Request processed in 29554us (29789us total) trans 4294973400 rc 0/0 00000100:00100000:0.0:1789760744.156741:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6111 00000100:00100000:0.0:1789760744.156750:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 723616713026 time 723691433305 state 0x2 ref 0 00000100:00100000:3.0:1789760744.160091:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760744.160145:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512639104 00000100:00100000:0.0:1789760744.160155:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512639104 00000100:00100000:0.0:1789760744.160209:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 723616713026 time 723691433305 state 0x0 ref 1 00000100:00100000:0.0:1789760744.160224:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 723691693721 time 723691433305 state 0x1 ref 1 00000100:00100000:0.0:1789760744.160326:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 723691693721 time 723724615363 state 0x0 ref 1 00000100:00100000:0.0:1789760744.160341:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5304000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512639104:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760744.160392:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24911872, mtime: 0x6aad94e7, atime: 0x0, ctime: 0x6aad94e7, blocks: 0xc007 00010000:00010000:0.0:1789760744.160401:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2e778 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.160416:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=723s v=(0 0) 00010000:00010000:0.0:1789760744.160421:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760744.160524:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2e778 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24911872->24915967] (req 24911872->24915967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.160542:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2e778 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24911872->24915967] (req 24911872->24915967) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760744.200432:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24911872 -> 24915968 00010000:00010000:0.0:1789760744.200452:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2e778 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24911872->24915967] (req 24911872->24915967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.200465:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2e778 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24911872->24915967] (req 24911872->24915967) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.200474:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2e778 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24911872->24915967] (req 24911872->24915967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.200481:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760744.200486:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2e778 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24911872->24915967] (req 24911872->24915967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.200495:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2e778 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24911872->24915967] (req 24911872->24915967) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.200514:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2e778 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24911872->24915967] (req 24911872->24915967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.200522:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2e778 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24911872->24915967] (req 24911872->24915967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760744.200612:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5304000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2358:x1876699512639104:12345-192.168.203.56@tcp:4:dd.0 Request processed in 40269us (40523us total) trans 4294973401 rc 0/0 00000100:00100000:0.0:1789760744.200625:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6112 00000100:00100000:0.0:1789760744.200635:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 723691693721 time 723724615363 state 0x2 ref 0 00000001:00080000:3.0:1789760744.200825:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973401 is committed 00000100:00100000:3.0:1789760744.205411:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760744.205508:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512639232 00000100:00100000:0.0:1789760744.205518:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512639232 00000100:00100000:0.0:1789760744.205637:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 723691693721 time 723724615363 state 0x0 ref 1 00000100:00100000:0.0:1789760744.205652:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 723724875779 time 723724615363 state 0x1 ref 1 00000100:00100000:0.0:1789760744.205683:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 723724875779 time 723769983045 state 0x0 ref 1 00000100:00100000:0.0:1789760744.205896:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5304380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512639232:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760744.205945:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24915968, mtime: 0x6aad94e7, atime: 0x0, ctime: 0x6aad94e7, blocks: 0xc007 00010000:00010000:0.0:1789760744.205955:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2e77f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.205970:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=723s v=(0 0) 00010000:00010000:0.0:1789760744.205974:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760744.205980:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2e77f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24915968->24920063] (req 24915968->24920063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.206054:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2e77f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24915968->24920063] (req 24915968->24920063) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760744.251010:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973402 is committed 00002000:00010000:0.0:1789760744.251634:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24915968 -> 24920064 00010000:00010000:0.0:1789760744.251650:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2e77f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24915968->24920063] (req 24915968->24920063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.251661:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2e77f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24915968->24920063] (req 24915968->24920063) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.251669:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2e77f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24915968->24920063] (req 24915968->24920063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.251674:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760744.251678:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2e77f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24915968->24920063] (req 24915968->24920063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.251686:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2e77f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24915968->24920063] (req 24915968->24920063) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.251718:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2e77f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24915968->24920063] (req 24915968->24920063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.251727:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2e77f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24915968->24920063] (req 24915968->24920063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760744.251823:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5304380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512639232:12345-192.168.203.56@tcp:4:dd.0 Request processed in 46124us (46415us total) trans 4294973402 rc 0/0 00000100:00100000:0.0:1789760744.251836:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6113 00000100:00100000:0.0:1789760744.251845:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 723724875779 time 723769983045 state 0x2 ref 0 00000100:00100000:1.0:1789760744.255503:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760744.255693:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512639360 00000100:00100000:0.0:1789760744.255732:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512639360 00000100:00100000:0.0:1789760744.255785:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 723724875779 time 723769983045 state 0x0 ref 1 00000100:00100000:0.0:1789760744.255799:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 723770243461 time 723769983045 state 0x1 ref 1 00000100:00100000:0.0:1789760744.255818:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 723770243461 time 723820118693 state 0x0 ref 1 00000100:00100000:0.0:1789760744.255829:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febfb89ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512639360:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760744.255876:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24920064, mtime: 0x6aad94e7, atime: 0x0, ctime: 0x6aad94e7, blocks: 0xc007 00010000:00010000:0.0:1789760744.255885:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2e786 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.255901:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=723s v=(0 0) 00010000:00010000:0.0:1789760744.255905:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760744.255911:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2e786 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24920064->24924159] (req 24920064->24924159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.255925:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2e786 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24920064->24924159] (req 24920064->24924159) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760744.298552:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973403 is committed 00002000:00010000:0.0:1789760744.298635:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24920064 -> 24924160 00010000:00010000:0.0:1789760744.298650:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2e786 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24920064->24924159] (req 24920064->24924159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.298663:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2e786 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24920064->24924159] (req 24920064->24924159) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.298678:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2e786 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24920064->24924159] (req 24920064->24924159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.298686:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760744.298692:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2e786 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24920064->24924159] (req 24920064->24924159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.298793:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2e786 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24920064->24924159] (req 24920064->24924159) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.298810:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2e786 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24920064->24924159] (req 24920064->24924159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.298818:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2e786 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24920064->24924159] (req 24920064->24924159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760744.299005:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febfb89ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512639360:12345-192.168.203.56@tcp:4:dd.0 Request processed in 43173us (43609us total) trans 4294973403 rc 0/0 00000100:00100000:0.0:1789760744.299015:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6114 00000100:00100000:0.0:1789760744.299025:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 723770243461 time 723820118693 state 0x2 ref 0 00000100:00100000:3.0:1789760744.302378:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760744.302490:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512639488 00000100:00100000:0.0:1789760744.302499:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512639488 00000100:00100000:0.0:1789760744.302543:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 723770243461 time 723820118693 state 0x0 ref 1 00000100:00100000:0.0:1789760744.302555:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 723820379109 time 723820118693 state 0x1 ref 1 00000100:00100000:0.0:1789760744.302579:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 723820379109 time 723866879554 state 0x0 ref 1 00000100:00100000:0.0:1789760744.302588:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5306680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512639488:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760744.302625:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24924160, mtime: 0x6aad94e7, atime: 0x0, ctime: 0x6aad94e7, blocks: 0xc007 00010000:00010000:0.0:1789760744.302632:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2e78d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.302644:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=723s v=(0 0) 00010000:00010000:0.0:1789760744.302647:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760744.302651:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2e78d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24924160->24928255] (req 24924160->24928255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.302661:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2e78d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24924160->24928255] (req 24924160->24928255) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760744.343711:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973404 is committed 00002000:00010000:0.0:1789760744.344974:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24924160 -> 24928256 00010000:00010000:0.0:1789760744.345119:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2e78d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24924160->24928255] (req 24924160->24928255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.345133:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2e78d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24924160->24928255] (req 24924160->24928255) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.345151:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2e78d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24924160->24928255] (req 24924160->24928255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.345159:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760744.345164:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2e78d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24924160->24928255] (req 24924160->24928255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.345172:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2e78d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24924160->24928255] (req 24924160->24928255) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.345191:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2e78d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24924160->24928255] (req 24924160->24928255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.345198:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2e78d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24924160->24928255] (req 24924160->24928255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760744.345295:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5306680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512639488:12345-192.168.203.56@tcp:4:dd.0 Request processed in 42702us (42925us total) trans 4294973404 rc 0/0 00000100:00100000:0.0:1789760744.345306:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6115 00000100:00100000:0.0:1789760744.345323:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 723820379109 time 723866879554 state 0x2 ref 0 00000100:00100000:2.0:1789760744.349224:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760744.349454:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512639616 00000100:00100000:0.0:1789760744.349465:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512639616 00000100:00100000:0.0:1789760744.349521:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 723820379109 time 723866879554 state 0x0 ref 1 00000100:00100000:0.0:1789760744.349536:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 723867139970 time 723866879554 state 0x1 ref 1 00000100:00100000:0.0:1789760744.349603:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 723867139970 time 723913901864 state 0x0 ref 1 00000100:00100000:0.0:1789760744.349615:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febffb9ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512639616:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760744.349668:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24928256, mtime: 0x6aad94e7, atime: 0x0, ctime: 0x6aad94e7, blocks: 0xc007 00010000:00010000:0.0:1789760744.349677:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2e794 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.349694:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=723s v=(0 0) 00010000:00010000:0.0:1789760744.349732:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760744.349739:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2e794 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24928256->24932351] (req 24928256->24932351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.349754:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2e794 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24928256->24932351] (req 24928256->24932351) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760744.383754:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24928256 -> 24932352 00010000:00010000:0.0:1789760744.383774:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2e794 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24928256->24932351] (req 24928256->24932351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.383788:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2e794 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24928256->24932351] (req 24928256->24932351) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.383798:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2e794 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24928256->24932351] (req 24928256->24932351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.383806:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760744.383812:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2e794 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24928256->24932351] (req 24928256->24932351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.383822:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2e794 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24928256->24932351] (req 24928256->24932351) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.383847:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2e794 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24928256->24932351] (req 24928256->24932351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.383855:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2e794 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24928256->24932351] (req 24928256->24932351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760744.383951:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febffb9ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2358:x1876699512639616:12345-192.168.203.56@tcp:4:dd.0 Request processed in 34333us (34732us total) trans 4294973405 rc 0/0 00000100:00100000:0.0:1789760744.383962:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6116 00000100:00100000:0.0:1789760744.383972:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 723867139970 time 723913901864 state 0x2 ref 0 00000001:00080000:1.0:1789760744.384395:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973405 is committed 00000100:00100000:1.0:1789760744.392001:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760744.392135:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512639744 00000100:00100000:0.0:1789760744.392145:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512639744 00000100:00100000:0.0:1789760744.392199:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 723867139970 time 723913901864 state 0x0 ref 1 00000100:00100000:0.0:1789760744.392215:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 723914162280 time 723913901864 state 0x1 ref 1 00000100:00100000:0.0:1789760744.392248:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 723914162280 time 723956546499 state 0x0 ref 1 00000100:00100000:0.0:1789760744.392259:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febfb89fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512639744:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760744.392305:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24932352, mtime: 0x6aad94e7, atime: 0x0, ctime: 0x6aad94e7, blocks: 0xc007 00010000:00010000:0.0:1789760744.392323:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2e79b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.392339:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=723s v=(0 0) 00010000:00010000:0.0:1789760744.392343:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760744.392349:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2e79b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24932352->24936447] (req 24932352->24936447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.392363:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2e79b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24932352->24936447] (req 24932352->24936447) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760744.436484:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973406 is committed 00002000:00010000:1.0:1789760744.436632:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24932352 -> 24936448 00010000:00010000:1.0:1789760744.436651:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2e79b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24932352->24936447] (req 24932352->24936447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.436665:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2e79b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24932352->24936447] (req 24932352->24936447) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.436675:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2e79b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24932352->24936447] (req 24932352->24936447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.436683:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760744.436688:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2e79b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24932352->24936447] (req 24932352->24936447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.436793:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2e79b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24932352->24936447] (req 24932352->24936447) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.436817:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2e79b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24932352->24936447] (req 24932352->24936447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.436826:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2e79b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24932352->24936447] (req 24932352->24936447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760744.437038:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febfb89fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512639744:12345-192.168.203.56@tcp:4:dd.0 Request processed in 44768us (45044us total) trans 4294973406 rc 0/0 00000100:00100000:1.0:1789760744.437049:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6117 00000100:00100000:1.0:1789760744.437060:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 723914162280 time 723956546499 state 0x2 ref 0 00000100:00100000:2.0:1789760744.441405:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760744.446159:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512639872 00000100:00100000:1.0:1789760744.446182:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512639872 00000100:00100000:1.0:1789760744.446267:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 723914162280 time 723956546499 state 0x0 ref 1 00000100:00100000:1.0:1789760744.446288:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 723956806915 time 723956546499 state 0x1 ref 1 00000100:00100000:1.0:1789760744.446337:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 723956806915 time 724010635548 state 0x0 ref 1 00000100:00100000:1.0:1789760744.446353:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febffb9c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512639872:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760744.446420:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24936448, mtime: 0x6aad94e7, atime: 0x0, ctime: 0x6aad94e7, blocks: 0xc007 00010000:00010000:1.0:1789760744.446431:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2e7a2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.446452:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=724s v=(0 0) 00010000:00010000:1.0:1789760744.446457:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760744.446466:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2e7a2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24936448->24940543] (req 24936448->24940543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.446483:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2e7a2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24936448->24940543] (req 24936448->24940543) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760744.489338:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973407 is committed 00002000:00010000:1.0:1789760744.489354:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24936448 -> 24940544 00010000:00010000:1.0:1789760744.489438:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2e7a2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24936448->24940543] (req 24936448->24940543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.489454:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2e7a2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24936448->24940543] (req 24936448->24940543) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.489463:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2e7a2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24936448->24940543] (req 24936448->24940543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.489470:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760744.489476:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2e7a2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24936448->24940543] (req 24936448->24940543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.489484:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2e7a2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24936448->24940543] (req 24936448->24940543) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.489507:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2e7a2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24936448->24940543] (req 24936448->24940543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.489516:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2e7a2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24936448->24940543] (req 24936448->24940543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760744.489959:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febffb9c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512639872:12345-192.168.203.56@tcp:4:dd.0 Request processed in 43344us (48298us total) trans 4294973407 rc 0/0 00000100:00100000:1.0:1789760744.489972:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6118 00000100:00100000:1.0:1789760744.489984:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 723956806915 time 724010635548 state 0x2 ref 0 00000100:00100000:2.0:1789760744.495017:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760744.495200:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512640000 00000100:00100000:0.0:1789760744.495214:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512640000 00000100:00100000:0.0:1789760744.495290:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 723956806915 time 724010635548 state 0x0 ref 1 00000100:00100000:0.0:1789760744.495309:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 724010895964 time 724010635548 state 0x1 ref 1 00000100:00100000:0.0:1789760744.495391:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 724010895964 time 724059689993 state 0x0 ref 1 00000100:00100000:0.0:1789760744.495407:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febffb9c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512640000:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760744.495473:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24940544, mtime: 0x6aad94e7, atime: 0x0, ctime: 0x6aad94e7, blocks: 0xc007 00010000:00010000:0.0:1789760744.495485:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2e7a9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.495505:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=724s v=(0 0) 00010000:00010000:0.0:1789760744.495510:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760744.495519:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2e7a9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24940544->24944639] (req 24940544->24944639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.495537:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2e7a9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24940544->24944639] (req 24940544->24944639) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760744.533783:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24940544 -> 24944640 00002000:00010000:1.0:1789760744.533795:0:13062:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760743 -> 1789760744 00002000:00010000:1.0:1789760744.533798:0:13062:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760743 -> 1789760744 00010000:00010000:1.0:1789760744.533815:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2e7a9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24940544->24944639] (req 24940544->24944639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.533841:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2e7a9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24940544->24944639] (req 24940544->24944639) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.533852:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2e7a9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24940544->24944639] (req 24940544->24944639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.533860:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760744.533865:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2e7a9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24940544->24944639] (req 24940544->24944639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.533875:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2e7a9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24940544->24944639] (req 24940544->24944639) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.533901:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2e7a9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24940544->24944639] (req 24940544->24944639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.533909:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2e7a9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24940544->24944639] (req 24940544->24944639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760744.534012:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febffb9c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2358:x1876699512640000:12345-192.168.203.56@tcp:4:dd.0 Request processed in 38602us (39004us total) trans 4294973408 rc 0/0 00000100:00100000:1.0:1789760744.534025:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6119 00000100:00100000:1.0:1789760744.534037:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 724010895964 time 724059689993 state 0x2 ref 0 00000001:00080000:3.0:1789760744.534215:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973408 is committed 00000100:00100000:3.0:1789760744.537977:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760744.538055:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512640128 00000100:00100000:1.0:1789760744.538071:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512640128 00000100:00100000:1.0:1789760744.538152:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 724010895964 time 724059689993 state 0x0 ref 1 00000100:00100000:1.0:1789760744.538170:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 724059950409 time 724059689993 state 0x1 ref 1 00000100:00100000:1.0:1789760744.538265:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 724059950409 time 724102564200 state 0x0 ref 1 00000100:00100000:1.0:1789760744.538282:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5307b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512640128:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760744.538372:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24944640, mtime: 0x6aad94e8, atime: 0x0, ctime: 0x6aad94e8, blocks: 0xc007 00010000:00010000:1.0:1789760744.538383:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2e7b0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.538402:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=724s v=(0 0) 00010000:00010000:1.0:1789760744.538409:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760744.538417:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2e7b0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24944640->24948735] (req 24944640->24948735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.538433:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2e7b0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24944640->24948735] (req 24944640->24948735) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760744.571873:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973409 is committed 00002000:00010000:1.0:1789760744.571906:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24944640 -> 24948736 00010000:00010000:1.0:1789760744.571927:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2e7b0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24944640->24948735] (req 24944640->24948735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.571940:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2e7b0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24944640->24948735] (req 24944640->24948735) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.571949:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2e7b0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24944640->24948735] (req 24944640->24948735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.571956:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760744.571960:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2e7b0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24944640->24948735] (req 24944640->24948735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.571969:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2e7b0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24944640->24948735] (req 24944640->24948735) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.571990:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2e7b0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24944640->24948735] (req 24944640->24948735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.571996:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2e7b0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24944640->24948735] (req 24944640->24948735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760744.572102:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5307b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512640128:12345-192.168.203.56@tcp:4:dd.0 Request processed in 33820us (34129us total) trans 4294973409 rc 0/0 00000100:00100000:1.0:1789760744.572116:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6120 00000100:00100000:1.0:1789760744.572126:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 724059950409 time 724102564200 state 0x2 ref 0 00000100:00100000:2.0:1789760744.580930:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760744.581509:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512640256 00000100:00100000:1.0:1789760744.581525:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512640256 00000100:00100000:1.0:1789760744.581586:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 724059950409 time 724102564200 state 0x0 ref 1 00000100:00100000:1.0:1789760744.581602:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 724102824616 time 724102564200 state 0x1 ref 1 00000100:00100000:1.0:1789760744.581633:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 724102824616 time 724145933991 state 0x0 ref 1 00000100:00100000:1.0:1789760744.581645:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febffb9f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512640256:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760744.581732:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24948736, mtime: 0x6aad94e8, atime: 0x0, ctime: 0x6aad94e8, blocks: 0xc007 00010000:00010000:1.0:1789760744.581741:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2e7b7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.581756:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=724s v=(0 0) 00010000:00010000:1.0:1789760744.581760:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760744.581766:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2e7b7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24948736->24952831] (req 24948736->24952831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.581779:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2e7b7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24948736->24952831] (req 24948736->24952831) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760744.626447:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973410 is committed 00002000:00010000:1.0:1789760744.626662:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24948736 -> 24952832 00010000:00010000:1.0:1789760744.626683:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2e7b7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24948736->24952831] (req 24948736->24952831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.626718:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2e7b7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24948736->24952831] (req 24948736->24952831) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.626729:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2e7b7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24948736->24952831] (req 24948736->24952831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.626737:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760744.626743:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2e7b7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24948736->24952831] (req 24948736->24952831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.626752:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2e7b7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24948736->24952831] (req 24948736->24952831) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.626778:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2e7b7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24948736->24952831] (req 24948736->24952831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.626786:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2e7b7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24948736->24952831] (req 24948736->24952831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760744.626911:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febffb9f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512640256:12345-192.168.203.56@tcp:4:dd.0 Request processed in 45259us (45988us total) trans 4294973410 rc 0/0 00000100:00100000:1.0:1789760744.626927:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6121 00000100:00100000:1.0:1789760744.626998:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 724102824616 time 724145933991 state 0x2 ref 0 00000100:00100000:2.0:1789760744.630550:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760744.630652:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512640384 00000100:00100000:1.0:1789760744.630674:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512640384 00000100:00100000:1.0:1789760744.631159:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 724102824616 time 724145933991 state 0x0 ref 1 00000100:00100000:1.0:1789760744.631179:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 724146194407 time 724145933991 state 0x1 ref 1 00000100:00100000:1.0:1789760744.631255:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 724146194407 time 724195553831 state 0x0 ref 1 00000100:00100000:1.0:1789760744.631270:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4132d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512640384:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760744.631347:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24952832, mtime: 0x6aad94e8, atime: 0x0, ctime: 0x6aad94e8, blocks: 0xc007 00010000:00010000:1.0:1789760744.631359:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2e7be lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.631469:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=724s v=(0 0) 00010000:00010000:1.0:1789760744.631475:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760744.631482:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2e7be lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24952832->24956927] (req 24952832->24956927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.631500:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2e7be lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24952832->24956927] (req 24952832->24956927) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760744.663373:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973411 is committed 00002000:00010000:2.0:1789760744.663611:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24952832 -> 24956928 00010000:00010000:2.0:1789760744.663633:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2e7be lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24952832->24956927] (req 24952832->24956927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.663649:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2e7be lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24952832->24956927] (req 24952832->24956927) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.663660:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2e7be lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24952832->24956927] (req 24952832->24956927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.663668:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760744.663674:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2e7be lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24952832->24956927] (req 24952832->24956927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.663689:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2e7be lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24952832->24956927] (req 24952832->24956927) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.663737:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2e7be lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24952832->24956927] (req 24952832->24956927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.663745:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2e7be lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24952832->24956927] (req 24952832->24956927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760744.664404:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4132d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512640384:12345-192.168.203.56@tcp:4:dd.0 Request processed in 33122us (33850us total) trans 4294973411 rc 0/0 00000100:00100000:2.0:1789760744.664417:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6122 00000100:00100000:2.0:1789760744.664429:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 724146194407 time 724195553831 state 0x2 ref 0 00000100:00100000:3.0:1789760744.670920:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760744.671113:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512640512 00000100:00100000:1.0:1789760744.671128:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512640512 00000100:00100000:1.0:1789760744.671202:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 724146194407 time 724195553831 state 0x0 ref 1 00000100:00100000:1.0:1789760744.671221:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 724195814247 time 724195553831 state 0x1 ref 1 00000100:00100000:1.0:1789760744.671258:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 724195814247 time 724235557576 state 0x0 ref 1 00000100:00100000:1.0:1789760744.671272:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5307100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512640512:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760744.671348:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24956928, mtime: 0x6aad94e8, atime: 0x0, ctime: 0x6aad94e8, blocks: 0xc007 00010000:00010000:1.0:1789760744.671360:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2e7c5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.671380:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=724s v=(0 0) 00010000:00010000:1.0:1789760744.671385:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760744.671393:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2e7c5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24956928->24961023] (req 24956928->24961023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.671408:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2e7c5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24956928->24961023] (req 24956928->24961023) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760744.705541:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24956928 -> 24961024 00010000:00010000:0.0:1789760744.705565:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2e7c5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24956928->24961023] (req 24956928->24961023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.705576:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2e7c5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24956928->24961023] (req 24956928->24961023) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.705583:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2e7c5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24956928->24961023] (req 24956928->24961023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.705588:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760744.705593:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2e7c5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24956928->24961023] (req 24956928->24961023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.705601:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2e7c5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24956928->24961023] (req 24956928->24961023) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.705621:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2e7c5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24956928->24961023] (req 24956928->24961023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.705625:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2e7c5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24956928->24961023] (req 24956928->24961023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760744.705931:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5307100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2360:x1876699512640512:12345-192.168.203.56@tcp:4:dd.0 Request processed in 34651us (35017us total) trans 4294973412 rc 0/0 00000100:00100000:0.0:1789760744.705947:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6123 00000100:00100000:0.0:1789760744.705959:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 724195814247 time 724235557576 state 0x2 ref 0 00000001:00080000:3.0:1789760744.706120:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973412 is committed 00000100:00100000:0.0:1789760744.708928:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760744.708986:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760744.709132:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760744.709198:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512640640 00000100:00100000:2.0:1789760744.709212:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512640640 00000100:00100000:2.0:1789760744.709279:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 398 00000100:00100000:2.0:1789760744.709291:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febe0503850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:e6cf7267-9b19-462a-a263-14115df15dab+13:2361:x1876699512640640:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:2.0:1789760744.709388:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febe0503850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:e6cf7267-9b19-462a-a263-14115df15dab+13:2361:x1876699512640640:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 99us (473us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760744.709396:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 398 00000100:00100000:2.0:1789760744.711361:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760744.711471:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512641024 00000100:00100000:3.0:1789760744.711482:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512641024 00000100:00100000:3.0:1789760744.711553:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 724195814247 time 724235557576 state 0x0 ref 1 00000100:00100000:3.0:1789760744.711571:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 724235817992 time 724235557576 state 0x1 ref 1 00000100:00100000:3.0:1789760744.711609:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 724235817992 time 724275908983 state 0x0 ref 1 00000100:00100000:3.0:1789760744.711623:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4132300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512641024:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760744.711685:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24961024, mtime: 0x6aad94e8, atime: 0x0, ctime: 0x6aad94e8, blocks: 0xc007 00010000:00010000:3.0:1789760744.711737:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2e7cc lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.711758:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=724s v=(0 0) 00010000:00010000:3.0:1789760744.711763:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760744.711771:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2e7cc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24961024->24965119] (req 24961024->24965119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.711789:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2e7cc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24961024->24965119] (req 24961024->24965119) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760744.715262:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512640896 00000100:00100000:1.0:1789760744.715284:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512640896 00000100:00080000:0.0:1789760744.715480:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1789760744.715530:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512640768 00000100:00100000:1.0:1789760744.715534:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512640768 00000100:00080000:0.0:1789760744.715588:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789760744.715630:0:5848:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 474 00000100:00100000:2.0:1789760744.715645:0:5848:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2878a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:d2c40150-e97b-40da-952b-cb9f18bf7e96+5:2361:x1876699512640896:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:2.0:1789760744.715952:0:5848:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2878a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:d2c40150-e97b-40da-952b-cb9f18bf7e96+5:2361:x1876699512640896:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 308us (6963us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760744.715962:0:5848:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 474 00000100:00100000:2.0:1789760744.715995:0:5848:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 475 00000100:00100000:2.0:1789760744.716004:0:5848:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac287ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+8:2361:x1876699512640768:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:2.0:1789760744.716034:0:5848:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac287ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+8:2361:x1876699512640768:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 32us (6902us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760744.716342:0:5848:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 475 00000001:00080000:3.0:1789760744.760530:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973413 is committed 00002000:00010000:2.0:1789760744.760785:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24961024 -> 24965120 00010000:00010000:2.0:1789760744.760806:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2e7cc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24961024->24965119] (req 24961024->24965119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.760822:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2e7cc lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24961024->24965119] (req 24961024->24965119) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.760832:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2e7cc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24961024->24965119] (req 24961024->24965119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.760840:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760744.760846:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2e7cc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24961024->24965119] (req 24961024->24965119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.760856:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2e7cc lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24961024->24965119] (req 24961024->24965119) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.760886:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2e7cc lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24961024->24965119] (req 24961024->24965119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.760894:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2e7cc lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24961024->24965119] (req 24961024->24965119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760744.761022:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4132300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512641024:12345-192.168.203.56@tcp:4:dd.0 Request processed in 49394us (49664us total) trans 4294973413 rc 0/0 00000100:00100000:2.0:1789760744.761250:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6124 00000100:00100000:2.0:1789760744.761266:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 724235817992 time 724275908983 state 0x2 ref 0 00000100:00100000:0.0:1789760744.765445:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760744.765623:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512641152 00000100:00100000:2.0:1789760744.765638:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512641152 00000100:00100000:2.0:1789760744.767234:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 724235817992 time 724275908983 state 0x0 ref 1 00000100:00100000:2.0:1789760744.767262:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 724276169399 time 724275908983 state 0x1 ref 1 00000100:00100000:2.0:1789760744.767332:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 724276169399 time 724331631881 state 0x0 ref 1 00000100:00100000:2.0:1789760744.767361:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febea4b4c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512641152:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760744.767438:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24965120, mtime: 0x6aad94e8, atime: 0x0, ctime: 0x6aad94e8, blocks: 0xc007 00010000:00010000:2.0:1789760744.767447:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2e7d3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.767468:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=724s v=(0 0) 00010000:00010000:2.0:1789760744.767474:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760744.767482:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2e7d3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24965120->24969215] (req 24965120->24969215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.767500:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2e7d3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24965120->24969215] (req 24965120->24969215) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760744.796360:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973414 is committed 00002000:00010000:2.0:1789760744.796577:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24965120 -> 24969216 00010000:00010000:2.0:1789760744.796597:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2e7d3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24965120->24969215] (req 24965120->24969215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.796611:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2e7d3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24965120->24969215] (req 24965120->24969215) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.796621:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2e7d3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24965120->24969215] (req 24965120->24969215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.796629:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760744.796635:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2e7d3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24965120->24969215] (req 24965120->24969215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.796645:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2e7d3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24965120->24969215] (req 24965120->24969215) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.796670:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2e7d3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24965120->24969215] (req 24965120->24969215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.796683:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2e7d3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24965120->24969215] (req 24965120->24969215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760744.796834:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febea4b4c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512641152:12345-192.168.203.56@tcp:4:dd.0 Request processed in 29481us (31395us total) trans 4294973414 rc 0/0 00000100:00100000:2.0:1789760744.796847:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6125 00000100:00100000:2.0:1789760744.796859:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 724276169399 time 724331631881 state 0x2 ref 0 00000100:00100000:3.0:1789760744.802750:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760744.802854:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512641280 00000100:00100000:0.0:1789760744.802871:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512641280 00000100:00100000:0.0:1789760744.802951:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 724276169399 time 724331631881 state 0x0 ref 1 00000100:00100000:0.0:1789760744.802971:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 724331892297 time 724331631881 state 0x1 ref 1 00000100:00100000:0.0:1789760744.803019:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 724331892297 time 724367317331 state 0x0 ref 1 00000100:00100000:0.0:1789760744.803037:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5305f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512641280:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760744.803251:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24969216, mtime: 0x6aad94e8, atime: 0x0, ctime: 0x6aad94e8, blocks: 0xc007 00010000:00010000:0.0:1789760744.803264:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2e7da lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.803287:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=724s v=(0 0) 00010000:00010000:0.0:1789760744.803293:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760744.803300:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2e7da lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24969216->24973311] (req 24969216->24973311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.803331:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2e7da lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24969216->24973311] (req 24969216->24973311) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.822436:0:5836:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:0.0:1789760744.836215:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973415 is committed 00002000:00010000:1.0:1789760744.836388:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24969216 -> 24973312 00010000:00010000:1.0:1789760744.836409:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2e7da lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24969216->24973311] (req 24969216->24973311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.836431:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2e7da lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24969216->24973311] (req 24969216->24973311) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.836441:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2e7da lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24969216->24973311] (req 24969216->24973311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.836450:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760744.836456:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2e7da lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24969216->24973311] (req 24969216->24973311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.836467:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2e7da lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24969216->24973311] (req 24969216->24973311) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.836491:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2e7da lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24969216->24973311] (req 24969216->24973311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.836499:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2e7da lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24969216->24973311] (req 24969216->24973311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760744.836619:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5305f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512641280:12345-192.168.203.56@tcp:4:dd.0 Request processed in 33580us (34033us total) trans 4294973415 rc 0/0 00000100:00100000:1.0:1789760744.836632:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6126 00000100:00100000:1.0:1789760744.836644:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 724331892297 time 724367317331 state 0x2 ref 0 00000100:00100000:0.0:1789760744.841902:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760744.842007:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512641408 00000100:00100000:1.0:1789760744.842021:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512641408 00000100:00100000:1.0:1789760744.842093:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 724331892297 time 724367317331 state 0x0 ref 1 00000100:00100000:1.0:1789760744.842112:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 724367577747 time 724367317331 state 0x1 ref 1 00000100:00100000:2.0:1789760744.842181:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 724367577747 time 724406476779 state 0x0 ref 1 00000100:00100000:2.0:1789760744.842208:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac287a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512641408:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760744.842273:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24973312, mtime: 0x6aad94e8, atime: 0x0, ctime: 0x6aad94e8, blocks: 0xc007 00010000:00010000:2.0:1789760744.842284:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2e7e1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.842304:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=724s v=(0 0) 00010000:00010000:2.0:1789760744.842321:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760744.842329:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2e7e1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24973312->24977407] (req 24973312->24977407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.842346:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2e7e1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24973312->24977407] (req 24973312->24977407) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760744.881586:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973416 is committed 00002000:00010000:3.0:1789760744.881899:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24973312 -> 24977408 00010000:00010000:3.0:1789760744.881919:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2e7e1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24973312->24977407] (req 24973312->24977407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.881936:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2e7e1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24973312->24977407] (req 24973312->24977407) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.881946:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2e7e1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24973312->24977407] (req 24973312->24977407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.881955:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760744.881962:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2e7e1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24973312->24977407] (req 24973312->24977407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.881973:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2e7e1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24973312->24977407] (req 24973312->24977407) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.881997:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2e7e1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24973312->24977407] (req 24973312->24977407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.882005:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2e7e1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24973312->24977407] (req 24973312->24977407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760744.882188:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac287a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512641408:12345-192.168.203.56@tcp:4:dd.0 Request processed in 39977us (40294us total) trans 4294973416 rc 0/0 00000100:00100000:3.0:1789760744.882201:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6127 00000100:00100000:3.0:1789760744.882215:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 724367577747 time 724406476779 state 0x2 ref 0 00000100:00100000:0.0:1789760744.885520:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760744.885584:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512641536 00000100:00100000:2.0:1789760744.885599:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512641536 00000100:00100000:2.0:1789760744.886137:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 724367577747 time 724406476779 state 0x0 ref 1 00000100:00100000:2.0:1789760744.886161:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 724406737195 time 724406476779 state 0x1 ref 1 00000100:00100000:2.0:1789760744.886197:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 724406737195 time 724450496706 state 0x0 ref 1 00000100:00100000:2.0:1789760744.886213:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2879c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512641536:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760744.886281:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24977408, mtime: 0x6aad94e8, atime: 0x0, ctime: 0x6aad94e8, blocks: 0xc007 00010000:00010000:2.0:1789760744.886291:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2e7e8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.886626:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=724s v=(0 0) 00010000:00010000:2.0:1789760744.886632:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760744.886639:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2e7e8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24977408->24981503] (req 24977408->24981503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.886653:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2e7e8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24977408->24981503] (req 24977408->24981503) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760744.925417:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973417 is committed 00002000:00010000:0.0:1789760744.925611:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24977408 -> 24981504 00010000:00010000:0.0:1789760744.925631:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2e7e8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24977408->24981503] (req 24977408->24981503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.925647:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2e7e8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24977408->24981503] (req 24977408->24981503) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.925667:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2e7e8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24977408->24981503] (req 24977408->24981503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.925894:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760744.925901:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2e7e8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24977408->24981503] (req 24977408->24981503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.925912:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2e7e8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24977408->24981503] (req 24977408->24981503) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.925939:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2e7e8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24977408->24981503] (req 24977408->24981503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.925953:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2e7e8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24977408->24981503] (req 24977408->24981503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760744.926074:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2879c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512641536:12345-192.168.203.56@tcp:4:dd.0 Request processed in 39858us (40561us total) trans 4294973417 rc 0/0 00000100:00100000:0.0:1789760744.926086:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6128 00000100:00100000:0.0:1789760744.926099:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 724406737195 time 724450496706 state 0x2 ref 0 00000100:00100000:2.0:1789760744.933123:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760744.933297:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512641664 00000100:00100000:1.0:1789760744.933321:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512641664 00000100:00100000:1.0:1789760744.933404:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 724406737195 time 724450496706 state 0x0 ref 1 00000100:00100000:1.0:1789760744.933423:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 724450757122 time 724450496706 state 0x1 ref 1 00000100:00100000:1.0:1789760744.933459:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 724450757122 time 724497758516 state 0x0 ref 1 00000100:00100000:1.0:1789760744.933473:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4132d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512641664:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760744.933534:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24981504, mtime: 0x6aad94e8, atime: 0x0, ctime: 0x6aad94e8, blocks: 0xc007 00010000:00010000:1.0:1789760744.933545:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2e7ef lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.933565:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=724s v=(0 0) 00010000:00010000:1.0:1789760744.933571:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760744.933578:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2e7ef lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24981504->24985599] (req 24981504->24985599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.933594:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2e7ef lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24981504->24985599] (req 24981504->24985599) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760744.980611:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24981504 -> 24985600 00010000:00010000:3.0:1789760744.980636:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2e7ef lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24981504->24985599] (req 24981504->24985599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.980652:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2e7ef lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24981504->24985599] (req 24981504->24985599) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.980663:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2e7ef lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24981504->24985599] (req 24981504->24985599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.980671:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760744.980677:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2e7ef lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24981504->24985599] (req 24981504->24985599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.980688:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2e7ef lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24981504->24985599] (req 24981504->24985599) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.980738:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2e7ef lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24981504->24985599] (req 24981504->24985599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.980747:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2e7ef lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24981504->24985599] (req 24981504->24985599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760744.981121:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973418 is committed 00000100:00100000:2.0:1789760744.981232:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4132d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512641664:12345-192.168.203.56@tcp:4:dd.0 Request processed in 47755us (48117us total) trans 4294973418 rc 0/0 00000100:00100000:2.0:1789760744.981245:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6129 00000100:00100000:2.0:1789760744.981258:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 724450757122 time 724497758516 state 0x2 ref 0 00000100:00100000:3.0:1789760744.987522:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760744.987679:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512641792 00000100:00100000:2.0:1789760744.987691:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512641792 00000100:00100000:2.0:1789760744.987905:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 724450757122 time 724497758516 state 0x0 ref 1 00000100:00100000:2.0:1789760744.987923:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 724498018932 time 724497758516 state 0x1 ref 1 00000100:00100000:2.0:1789760744.988078:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 724498018932 time 724552376161 state 0x0 ref 1 00000100:00100000:2.0:1789760744.988094:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5307480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512641792:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760744.988163:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24985600, mtime: 0x6aad94e8, atime: 0x0, ctime: 0x6aad94e8, blocks: 0xc007 00010000:00010000:2.0:1789760744.988174:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2e7f6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.988195:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=724s v=(0 0) 00010000:00010000:2.0:1789760744.988200:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760744.988214:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2e7f6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24985600->24989695] (req 24985600->24989695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.988232:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2e7f6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24985600->24989695] (req 24985600->24989695) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760745.018899:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973419 is committed 00002000:00010000:3.0:1789760745.019287:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24985600 -> 24989696 00010000:00010000:3.0:1789760745.019316:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2e7f6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24985600->24989695] (req 24985600->24989695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.019332:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2e7f6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24985600->24989695] (req 24985600->24989695) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.019342:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2e7f6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24985600->24989695] (req 24985600->24989695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.019421:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760745.019431:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2e7f6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24985600->24989695] (req 24985600->24989695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.019441:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2e7f6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24985600->24989695] (req 24985600->24989695) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.019466:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2e7f6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24985600->24989695] (req 24985600->24989695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.019473:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2e7f6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24985600->24989695] (req 24985600->24989695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760745.019640:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5307480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512641792:12345-192.168.203.56@tcp:4:dd.0 Request processed in 31542us (32131us total) trans 4294973419 rc 0/0 00000100:00100000:3.0:1789760745.019660:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6130 00000100:00100000:3.0:1789760745.019681:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 724498018932 time 724552376161 state 0x2 ref 0 00000100:00100000:1.0:1789760745.025347:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760745.025463:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512641920 00000100:00100000:0.0:1789760745.025479:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512641920 00000100:00100000:0.0:1789760745.025633:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 724498018932 time 724552376161 state 0x0 ref 1 00000100:00100000:0.0:1789760745.025654:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 724552636577 time 724552376161 state 0x1 ref 1 00000100:00100000:0.0:1789760745.025714:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 724552636577 time 724589994744 state 0x0 ref 1 00000100:00100000:0.0:1789760745.025731:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febfb89ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512641920:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760745.025964:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24989696, mtime: 0x6aad94e8, atime: 0x0, ctime: 0x6aad94e8, blocks: 0xc007 00010000:00010000:0.0:1789760745.025976:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2e7fd lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.026006:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=724s v=(0 0) 00010000:00010000:0.0:1789760745.026011:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760745.026019:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2e7fd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24989696->24993791] (req 24989696->24993791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.026043:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2e7fd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24989696->24993791] (req 24989696->24993791) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1789760745.026131:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:1.0:1789760745.063764:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973420 is committed 00002000:00010000:2.0:1789760745.064094:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24989696 -> 24993792 00010000:00010000:2.0:1789760745.064117:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2e7fd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24989696->24993791] (req 24989696->24993791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.064132:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2e7fd lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24989696->24993791] (req 24989696->24993791) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.064149:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2e7fd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24989696->24993791] (req 24989696->24993791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.064158:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760745.064164:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2e7fd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24989696->24993791] (req 24989696->24993791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.064175:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2e7fd lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24989696->24993791] (req 24989696->24993791) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.064200:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2e7fd lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24989696->24993791] (req 24989696->24993791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.064209:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2e7fd lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24989696->24993791] (req 24989696->24993791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760745.065128:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febfb89ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512641920:12345-192.168.203.56@tcp:4:dd.0 Request processed in 39389us (39784us total) trans 4294973420 rc 0/0 00000100:00100000:2.0:1789760745.065143:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6131 00000100:00100000:2.0:1789760745.065155:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 724552636577 time 724589994744 state 0x2 ref 0 00000100:00100000:2.0:1789760745.076767:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760745.077557:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512642048 00000100:00100000:0.0:1789760745.077572:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512642048 00000100:00100000:0.0:1789760745.077657:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 724552636577 time 724589994744 state 0x0 ref 1 00000100:00100000:0.0:1789760745.077679:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 724590255160 time 724589994744 state 0x1 ref 1 00000100:00100000:0.0:1789760745.077760:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 724590255160 time 724642059568 state 0x0 ref 1 00000100:00100000:0.0:1789760745.077776:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4132300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512642048:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760745.077856:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24993792, mtime: 0x6aad94e8, atime: 0x0, ctime: 0x6aad94e8, blocks: 0xc007 00010000:00010000:0.0:1789760745.077866:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2e804 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.077885:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=724s v=(0 0) 00010000:00010000:0.0:1789760745.077890:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760745.077898:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2e804 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24993792->24997887] (req 24993792->24997887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.077913:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2e804 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24993792->24997887] (req 24993792->24997887) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760745.126243:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973421 is committed 00002000:00010000:3.0:1789760745.126945:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24993792 -> 24997888 00010000:00010000:3.0:1789760745.126968:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2e804 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24993792->24997887] (req 24993792->24997887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.126994:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2e804 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24993792->24997887] (req 24993792->24997887) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.127004:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2e804 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24993792->24997887] (req 24993792->24997887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.127013:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760745.127019:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2e804 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24993792->24997887] (req 24993792->24997887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.127031:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2e804 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24993792->24997887] (req 24993792->24997887) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.127060:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2e804 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24993792->24997887] (req 24993792->24997887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.127070:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2e804 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24993792->24997887] (req 24993792->24997887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760745.127220:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4132300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512642048:12345-192.168.203.56@tcp:4:dd.0 Request processed in 49437us (50459us total) trans 4294973421 rc 0/0 00000100:00100000:3.0:1789760745.127233:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6132 00000100:00100000:3.0:1789760745.127247:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 724590255160 time 724642059568 state 0x2 ref 0 00000100:00100000:3.0:1789760745.134151:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760745.134537:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512642176 00000100:00100000:2.0:1789760745.134551:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512642176 00000100:00100000:2.0:1789760745.134636:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 724590255160 time 724642059568 state 0x0 ref 1 00000100:00100000:2.0:1789760745.134658:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 724642319984 time 724642059568 state 0x1 ref 1 00000100:00100000:2.0:1789760745.134852:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 724642319984 time 724699148179 state 0x0 ref 1 00000100:00100000:2.0:1789760745.134872:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5304700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512642176:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760745.134948:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24997888, mtime: 0x6aad94e8, atime: 0x0, ctime: 0x6aad94e8, blocks: 0xc007 00010000:00010000:2.0:1789760745.134961:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2e80b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.135859:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=724s v=(0 0) 00010000:00010000:2.0:1789760745.135872:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760745.135899:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2e80b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24997888->25001983] (req 24997888->25001983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.135924:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2e80b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24997888->25001983] (req 24997888->25001983) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760745.182412:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973422 is committed 00002000:00010000:3.0:1789760745.182586:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24997888 -> 25001984 00010000:00010000:3.0:1789760745.182607:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2e80b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24997888->25001983] (req 24997888->25001983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.182621:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2e80b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24997888->25001983] (req 24997888->25001983) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.182630:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2e80b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24997888->25001983] (req 24997888->25001983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.182638:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760745.182644:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2e80b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24997888->25001983] (req 24997888->25001983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.182653:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2e80b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24997888->25001983] (req 24997888->25001983) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.182677:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2e80b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24997888->25001983] (req 24997888->25001983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.182685:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2e80b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24997888->25001983] (req 24997888->25001983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760745.182864:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5304700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512642176:12345-192.168.203.56@tcp:4:dd.0 Request processed in 47991us (48876us total) trans 4294973422 rc 0/0 00000100:00100000:3.0:1789760745.182877:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6133 00000100:00100000:3.0:1789760745.182892:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 724642319984 time 724699148179 state 0x2 ref 0 00000100:00100000:2.0:1789760745.189426:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760745.189933:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512642304 00000100:00100000:0.0:1789760745.189947:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512642304 00000100:00100000:0.0:1789760745.190104:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 724642319984 time 724699148179 state 0x0 ref 1 00000100:00100000:0.0:1789760745.190124:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 724699408595 time 724699148179 state 0x1 ref 1 00000100:00100000:0.0:1789760745.190165:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 724699408595 time 724754463785 state 0x0 ref 1 00000100:00100000:0.0:1789760745.190179:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4131f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512642304:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760745.190388:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25001984, mtime: 0x6aad94e8, atime: 0x0, ctime: 0x6aad94e8, blocks: 0xc007 00010000:00010000:0.0:1789760745.190400:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2e812 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.190420:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=724s v=(0 0) 00010000:00010000:0.0:1789760745.190425:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760745.190433:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2e812 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25001984->25006079] (req 25001984->25006079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.190449:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2e812 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25001984->25006079] (req 25001984->25006079) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760745.237524:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973423 is committed 00002000:00010000:3.0:1789760745.237646:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25001984 -> 25006080 00010000:00010000:3.0:1789760745.237665:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2e812 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25001984->25006079] (req 25001984->25006079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.237679:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2e812 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25001984->25006079] (req 25001984->25006079) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.237824:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2e812 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25001984->25006079] (req 25001984->25006079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.237833:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760745.237839:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2e812 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25001984->25006079] (req 25001984->25006079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.237848:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2e812 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25001984->25006079] (req 25001984->25006079) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.237873:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2e812 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25001984->25006079] (req 25001984->25006079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.237882:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2e812 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25001984->25006079] (req 25001984->25006079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760745.238210:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4131f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512642304:12345-192.168.203.56@tcp:4:dd.0 Request processed in 48028us (48793us total) trans 4294973423 rc 0/0 00000100:00100000:3.0:1789760745.238224:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6134 00000100:00100000:3.0:1789760745.238235:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 724699408595 time 724754463785 state 0x2 ref 0 00000100:00100000:0.0:1789760745.245926:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760745.247186:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512642432 00000100:00100000:3.0:1789760745.247203:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512642432 00000100:00100000:2.0:1789760745.247374:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 724699408595 time 724754463785 state 0x0 ref 1 00000100:00100000:2.0:1789760745.247395:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 724754724201 time 724754463785 state 0x1 ref 1 00000100:00100000:2.0:1789760745.247476:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 724754724201 time 724811775169 state 0x0 ref 1 00000100:00100000:2.0:1789760745.247492:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2878e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512642432:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760745.247558:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25006080, mtime: 0x6aad94e8, atime: 0x0, ctime: 0x6aad94e8, blocks: 0xc007 00010000:00010000:2.0:1789760745.247569:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2e819 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.247589:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=724s v=(0 0) 00010000:00010000:2.0:1789760745.247594:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760745.247602:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2e819 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25006080->25010175] (req 25006080->25010175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.247619:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2e819 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25006080->25010175] (req 25006080->25010175) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760745.315270:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973424 is committed 00002000:00010000:3.0:1789760745.315609:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25006080 -> 25010176 00010000:00010000:3.0:1789760745.315631:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2e819 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25006080->25010175] (req 25006080->25010175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.315646:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2e819 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25006080->25010175] (req 25006080->25010175) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.315656:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2e819 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25006080->25010175] (req 25006080->25010175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.315663:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760745.315668:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2e819 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25006080->25010175] (req 25006080->25010175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.315677:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2e819 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25006080->25010175] (req 25006080->25010175) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.315734:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2e819 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25006080->25010175] (req 25006080->25010175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.315750:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2e819 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25006080->25010175] (req 25006080->25010175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760745.315871:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2878e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512642432:12345-192.168.203.56@tcp:4:dd.0 Request processed in 68374us (69952us total) trans 4294973424 rc 0/0 00000100:00100000:3.0:1789760745.315882:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6135 00000100:00100000:3.0:1789760745.315894:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 724754724201 time 724811775169 state 0x2 ref 0 00000100:00100000:1.0:1789760745.320242:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760745.320379:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512642560 00000100:00100000:3.0:1789760745.320393:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512642560 00000100:00100000:3.0:1789760745.320471:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 724754724201 time 724811775169 state 0x0 ref 1 00000100:00100000:3.0:1789760745.320491:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 724812035585 time 724811775169 state 0x1 ref 1 00000100:00100000:3.0:1789760745.320533:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 724812035585 time 724884831848 state 0x0 ref 1 00000100:00100000:3.0:1789760745.320549:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febfb89fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512642560:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760745.320624:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25010176, mtime: 0x6aad94e8, atime: 0x0, ctime: 0x6aad94e8, blocks: 0xc007 00010000:00010000:3.0:1789760745.320636:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2e820 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.320656:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=724s v=(0 0) 00010000:00010000:3.0:1789760745.320661:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760745.320669:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2e820 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25010176->25014271] (req 25010176->25014271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.320691:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2e820 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25010176->25014271] (req 25010176->25014271) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760745.380373:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973425 is committed 00002000:00010000:3.0:1789760745.381019:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25010176 -> 25014272 00010000:00010000:3.0:1789760745.381041:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2e820 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25010176->25014271] (req 25010176->25014271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.381055:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2e820 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25010176->25014271] (req 25010176->25014271) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.381065:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2e820 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25010176->25014271] (req 25010176->25014271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.381074:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760745.381081:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2e820 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25010176->25014271] (req 25010176->25014271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.381092:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2e820 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25010176->25014271] (req 25010176->25014271) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.381118:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2e820 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25010176->25014271] (req 25010176->25014271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.381126:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2e820 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25010176->25014271] (req 25010176->25014271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760745.381487:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febfb89fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512642560:12345-192.168.203.56@tcp:4:dd.0 Request processed in 60934us (61253us total) trans 4294973425 rc 0/0 00000100:00100000:3.0:1789760745.381500:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6136 00000100:00100000:3.0:1789760745.381511:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 724812035585 time 724884831848 state 0x2 ref 0 00000100:00100000:2.0:1789760745.388061:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760745.388270:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512642688 00000100:00100000:3.0:1789760745.388282:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512642688 00000100:00100000:3.0:1789760745.388370:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 724812035585 time 724884831848 state 0x0 ref 1 00000100:00100000:3.0:1789760745.388390:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 724885092264 time 724884831848 state 0x1 ref 1 00000100:00100000:3.0:1789760745.388849:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 724885092264 time 724953144765 state 0x0 ref 1 00000100:00100000:3.0:1789760745.388869:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4130380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512642688:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760745.388938:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25014272, mtime: 0x6aad94e8, atime: 0x0, ctime: 0x6aad94e8, blocks: 0xc007 00010000:00010000:3.0:1789760745.388948:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2e827 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.388969:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=724s v=(0 0) 00010000:00010000:3.0:1789760745.388974:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760745.388982:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2e827 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25014272->25018367] (req 25014272->25018367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.389000:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2e827 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25014272->25018367] (req 25014272->25018367) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760745.455656:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973426 is committed 00002000:00010000:3.0:1789760745.455845:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25014272 -> 25018368 00010000:00010000:3.0:1789760745.455865:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2e827 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25014272->25018367] (req 25014272->25018367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.455880:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2e827 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25014272->25018367] (req 25014272->25018367) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.455890:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2e827 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25014272->25018367] (req 25014272->25018367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.455898:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760745.455904:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2e827 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25014272->25018367] (req 25014272->25018367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.455915:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2e827 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25014272->25018367] (req 25014272->25018367) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.455944:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2e827 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25014272->25018367] (req 25014272->25018367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.455953:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2e827 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25014272->25018367] (req 25014272->25018367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760745.456239:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4130380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512642688:12345-192.168.203.56@tcp:4:dd.0 Request processed in 67367us (68184us total) trans 4294973426 rc 0/0 00000100:00100000:3.0:1789760745.456255:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6137 00000100:00100000:3.0:1789760745.456268:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 724885092264 time 724953144765 state 0x2 ref 0 00000100:00100000:0.0:1789760745.459624:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760745.459824:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512642816 00000100:00100000:3.0:1789760745.459838:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512642816 00000100:00100000:3.0:1789760745.459915:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 724885092264 time 724953144765 state 0x0 ref 1 00000100:00100000:3.0:1789760745.459935:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 724953405181 time 724953144765 state 0x1 ref 1 00000100:00100000:3.0:1789760745.459972:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 724953405181 time 725024271674 state 0x0 ref 1 00000100:00100000:3.0:1789760745.460102:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac287b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512642816:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760745.460172:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25018368, mtime: 0x6aad94e8, atime: 0x0, ctime: 0x6aad94e8, blocks: 0xc007 00010000:00010000:3.0:1789760745.460185:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2e82e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.460205:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=725s v=(0 0) 00010000:00010000:3.0:1789760745.460210:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760745.460218:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2e82e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25018368->25022463] (req 25018368->25022463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.460234:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2e82e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25018368->25022463] (req 25018368->25022463) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760745.516892:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973427 is committed 00002000:00010000:1.0:1789760745.517997:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25018368 -> 25022464 00010000:00010000:1.0:1789760745.518021:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2e82e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25018368->25022463] (req 25018368->25022463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.518035:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2e82e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25018368->25022463] (req 25018368->25022463) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.518046:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2e82e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25018368->25022463] (req 25018368->25022463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.518054:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760745.518060:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2e82e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25018368->25022463] (req 25018368->25022463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.518214:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2e82e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25018368->25022463] (req 25018368->25022463) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.518240:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2e82e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25018368->25022463] (req 25018368->25022463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.518248:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2e82e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25018368->25022463] (req 25018368->25022463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760745.518402:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac287b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512642816:12345-192.168.203.56@tcp:4:dd.0 Request processed in 58297us (58786us total) trans 4294973427 rc 0/0 00000100:00100000:1.0:1789760745.518415:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6138 00000100:00100000:1.0:1789760745.518427:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 724953405181 time 725024271674 state 0x2 ref 0 00000100:00100000:2.0:1789760745.523859:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760745.524888:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512642944 00000100:00100000:3.0:1789760745.524903:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512642944 00000100:00100000:3.0:1789760745.525056:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 724953405181 time 725024271674 state 0x0 ref 1 00000100:00100000:3.0:1789760745.525076:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 725024532090 time 725024271674 state 0x1 ref 1 00000100:00100000:3.0:1789760745.525114:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 725024532090 time 725089413361 state 0x0 ref 1 00000100:00100000:3.0:1789760745.525130:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4133b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512642944:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760745.525211:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25022464, mtime: 0x6aad94e8, atime: 0x0, ctime: 0x6aad94e8, blocks: 0xc007 00010000:00010000:3.0:1789760745.525221:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2e835 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.525241:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=725s v=(0 0) 00010000:00010000:3.0:1789760745.525246:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760745.525254:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2e835 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25022464->25026559] (req 25022464->25026559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.525270:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2e835 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25022464->25026559] (req 25022464->25026559) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760745.579385:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973428 is committed 00002000:00010000:3.0:1789760745.579520:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25022464 -> 25026560 00002000:00010000:3.0:1789760745.579526:0:13063:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760744 -> 1789760745 00002000:00010000:3.0:1789760745.579529:0:13063:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760744 -> 1789760745 00010000:00010000:3.0:1789760745.579546:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2e835 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25022464->25026559] (req 25022464->25026559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.579561:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2e835 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25022464->25026559] (req 25022464->25026559) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.579579:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2e835 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25022464->25026559] (req 25022464->25026559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.579588:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760745.579593:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2e835 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25022464->25026559] (req 25022464->25026559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.579602:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2e835 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25022464->25026559] (req 25022464->25026559) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.579627:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2e835 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25022464->25026559] (req 25022464->25026559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.579635:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2e835 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25022464->25026559] (req 25022464->25026559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760745.579802:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4133b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512642944:12345-192.168.203.56@tcp:4:dd.0 Request processed in 54669us (55955us total) trans 4294973428 rc 0/0 00000100:00100000:3.0:1789760745.579814:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6139 00000100:00100000:3.0:1789760745.579826:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 725024532090 time 725089413361 state 0x2 ref 0 00000100:00100000:2.0:1789760745.583365:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760745.583438:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512643072 00000100:00100000:3.0:1789760745.583454:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512643072 00000100:00100000:3.0:1789760745.583537:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 725024532090 time 725089413361 state 0x0 ref 1 00000100:00100000:3.0:1789760745.583558:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 725089673777 time 725089413361 state 0x1 ref 1 00000100:00100000:3.0:1789760745.583595:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 725089673777 time 725147894845 state 0x0 ref 1 00000100:00100000:3.0:1789760745.583610:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4133800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512643072:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760745.583677:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25026560, mtime: 0x6aad94e9, atime: 0x0, ctime: 0x6aad94e9, blocks: 0xc007 00010000:00010000:3.0:1789760745.583689:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2e83c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.584938:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=725s v=(0 0) 00010000:00010000:3.0:1789760745.584944:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760745.584952:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2e83c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25026560->25030655] (req 25026560->25030655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.584972:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2e83c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25026560->25030655] (req 25026560->25030655) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760745.658903:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973429 is committed 00002000:00010000:3.0:1789760745.659097:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25026560 -> 25030656 00010000:00010000:3.0:1789760745.659118:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2e83c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25026560->25030655] (req 25026560->25030655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.659133:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2e83c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25026560->25030655] (req 25026560->25030655) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.659143:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2e83c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25026560->25030655] (req 25026560->25030655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.659150:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760745.659156:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2e83c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25026560->25030655] (req 25026560->25030655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.659166:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2e83c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25026560->25030655] (req 25026560->25030655) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.659188:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2e83c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25026560->25030655] (req 25026560->25030655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.659196:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2e83c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25026560->25030655] (req 25026560->25030655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760745.659362:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4133800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512643072:12345-192.168.203.56@tcp:4:dd.0 Request processed in 75748us (75999us total) trans 4294973429 rc 0/0 00000100:00100000:3.0:1789760745.659375:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6140 00000100:00100000:3.0:1789760745.659387:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 725089673777 time 725147894845 state 0x2 ref 0 00000100:00100000:1.0:1789760745.663243:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760745.663366:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512643200 00000100:00100000:2.0:1789760745.663379:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512643200 00000100:00100000:2.0:1789760745.663450:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 725089673777 time 725147894845 state 0x0 ref 1 00000100:00100000:2.0:1789760745.663469:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 725148155261 time 725147894845 state 0x1 ref 1 00000100:00100000:2.0:1789760745.663502:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 725148155261 time 725227801349 state 0x0 ref 1 00000100:00100000:2.0:1789760745.663516:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4059180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512643200:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760745.663580:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25030656, mtime: 0x6aad94e9, atime: 0x0, ctime: 0x6aad94e9, blocks: 0xc007 00010000:00010000:2.0:1789760745.663590:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2e843 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.663677:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=725s v=(0 0) 00010000:00010000:2.0:1789760745.663682:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760745.663690:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2e843 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25030656->25034751] (req 25030656->25034751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.663865:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2e843 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25030656->25034751] (req 25030656->25034751) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760745.727982:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973430 is committed 00002000:00010000:1.0:1789760745.728284:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25030656 -> 25034752 00010000:00010000:1.0:1789760745.728313:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2e843 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25030656->25034751] (req 25030656->25034751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.728329:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2e843 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25030656->25034751] (req 25030656->25034751) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.728339:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2e843 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25030656->25034751] (req 25030656->25034751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.728347:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760745.728353:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2e843 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25030656->25034751] (req 25030656->25034751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.728363:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2e843 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25030656->25034751] (req 25030656->25034751) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.728389:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2e843 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25030656->25034751] (req 25030656->25034751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.728396:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2e843 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25030656->25034751] (req 25030656->25034751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760745.728519:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4059180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512643200:12345-192.168.203.56@tcp:4:dd.0 Request processed in 64999us (65284us total) trans 4294973430 rc 0/0 00000100:00100000:1.0:1789760745.728534:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6141 00000100:00100000:1.0:1789760745.728545:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 725148155261 time 725227801349 state 0x2 ref 0 00000100:00100000:1.0:1789760745.731671:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760745.731785:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512643328 00000100:00100000:3.0:1789760745.731797:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512643328 00000100:00100000:3.0:1789760745.731875:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 725148155261 time 725227801349 state 0x0 ref 1 00000100:00100000:3.0:1789760745.731894:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 725228061765 time 725227801349 state 0x1 ref 1 00000100:00100000:3.0:1789760745.731930:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 725228061765 time 725296229351 state 0x0 ref 1 00000100:00100000:3.0:1789760745.731947:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4059500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512643328:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760745.732201:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25034752, mtime: 0x6aad94e9, atime: 0x0, ctime: 0x6aad94e9, blocks: 0xc007 00010000:00010000:3.0:1789760745.732213:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2e84a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.732233:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=725s v=(0 0) 00010000:00010000:3.0:1789760745.732238:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760745.732247:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2e84a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25034752->25038847] (req 25034752->25038847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.732263:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2e84a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25034752->25038847] (req 25034752->25038847) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760745.764635:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973431 is committed 00002000:00010000:2.0:1789760745.764945:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25034752 -> 25038848 00010000:00010000:2.0:1789760745.764964:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2e84a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25034752->25038847] (req 25034752->25038847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.765038:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2e84a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25034752->25038847] (req 25034752->25038847) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.765048:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2e84a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25034752->25038847] (req 25034752->25038847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.765056:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760745.765062:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2e84a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25034752->25038847] (req 25034752->25038847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.765074:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2e84a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25034752->25038847] (req 25034752->25038847) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.765099:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2e84a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25034752->25038847] (req 25034752->25038847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.765108:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2e84a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25034752->25038847] (req 25034752->25038847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760745.765258:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4059500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512643328:12345-192.168.203.56@tcp:4:dd.0 Request processed in 33308us (33596us total) trans 4294973431 rc 0/0 00000100:00100000:2.0:1789760745.765274:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6142 00000100:00100000:2.0:1789760745.765286:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 725228061765 time 725296229351 state 0x2 ref 0 00000100:00100000:2.0:1789760745.769433:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760745.770401:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512643456 00000100:00100000:3.0:1789760745.770420:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512643456 00000100:00100000:3.0:1789760745.770512:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 725228061765 time 725296229351 state 0x0 ref 1 00000100:00100000:3.0:1789760745.770533:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 725296489767 time 725296229351 state 0x1 ref 1 00000100:00100000:3.0:1789760745.770576:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 725296489767 time 725334875362 state 0x0 ref 1 00000100:00100000:3.0:1789760745.770592:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4130000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512643456:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760745.770667:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25038848, mtime: 0x6aad94e9, atime: 0x0, ctime: 0x6aad94e9, blocks: 0xc007 00010000:00010000:3.0:1789760745.770679:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2e851 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.770735:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=725s v=(0 0) 00010000:00010000:3.0:1789760745.770742:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760745.770750:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2e851 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25038848->25042943] (req 25038848->25042943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.770770:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2e851 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25038848->25042943] (req 25038848->25042943) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.843238:0:5835:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:3.0:1789760745.867654:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973432 is committed 00002000:00010000:1.0:1789760745.867836:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25038848 -> 25042944 00010000:00010000:1.0:1789760745.867858:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2e851 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25038848->25042943] (req 25038848->25042943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.867874:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2e851 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25038848->25042943] (req 25038848->25042943) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.867884:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2e851 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25038848->25042943] (req 25038848->25042943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.867892:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760745.867898:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2e851 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25038848->25042943] (req 25038848->25042943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.867908:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2e851 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25038848->25042943] (req 25038848->25042943) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.867936:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2e851 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25038848->25042943] (req 25038848->25042943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.867951:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2e851 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25038848->25042943] (req 25038848->25042943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760745.868106:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4130000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512643456:12345-192.168.203.56@tcp:4:dd.0 Request processed in 97509us (98681us total) trans 4294973432 rc 0/0 00000100:00100000:1.0:1789760745.868118:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6143 00000100:00100000:1.0:1789760745.868131:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 725296489767 time 725334875362 state 0x2 ref 0 00000100:00100000:1.0:1789760745.873163:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760745.873337:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512643584 00000100:00100000:2.0:1789760745.873350:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512643584 00000100:00100000:2.0:1789760745.873418:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 725296489767 time 725334875362 state 0x0 ref 1 00000100:00100000:2.0:1789760745.873438:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 725335135778 time 725334875362 state 0x1 ref 1 00000100:00100000:2.0:1789760745.873478:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 725335135778 time 725437777249 state 0x0 ref 1 00000100:00100000:2.0:1789760745.873492:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4058a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512643584:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760745.873559:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25042944, mtime: 0x6aad94e9, atime: 0x0, ctime: 0x6aad94e9, blocks: 0xc007 00010000:00010000:2.0:1789760745.873570:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2e858 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.873590:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=725s v=(0 0) 00010000:00010000:2.0:1789760745.873595:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760745.873602:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2e858 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25042944->25047039] (req 25042944->25047039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.873618:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2e858 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25042944->25047039] (req 25042944->25047039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760745.917880:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25042944 -> 25047040 00010000:00010000:3.0:1789760745.917913:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2e858 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25042944->25047039] (req 25042944->25047039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.917930:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2e858 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25042944->25047039] (req 25042944->25047039) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.917952:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2e858 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25042944->25047039] (req 25042944->25047039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.917960:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760745.917965:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2e858 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25042944->25047039] (req 25042944->25047039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.917973:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2e858 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25042944->25047039] (req 25042944->25047039) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.917997:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2e858 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25042944->25047039] (req 25042944->25047039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.918005:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2e858 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25042944->25047039] (req 25042944->25047039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760745.918125:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4058a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2359:x1876699512643584:12345-192.168.203.56@tcp:4:dd.0 Request processed in 44628us (44969us total) trans 4294973433 rc 0/0 00000100:00100000:3.0:1789760745.918137:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6144 00000100:00100000:3.0:1789760745.918149:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 725335135778 time 725437777249 state 0x2 ref 0 00000001:00080000:3.0:1789760745.918200:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973433 is committed 00000100:00100000:3.0:1789760745.921740:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760745.921878:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512643712 00000100:00100000:2.0:1789760745.921891:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512643712 00000100:00100000:2.0:1789760745.921963:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 725335135778 time 725437777249 state 0x0 ref 1 00000100:00100000:2.0:1789760745.921983:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 725438037665 time 725437777249 state 0x1 ref 1 00000100:00100000:2.0:1789760745.922019:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 725438037665 time 725486318791 state 0x0 ref 1 00000100:00100000:2.0:1789760745.922033:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5306a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512643712:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760745.922094:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25047040, mtime: 0x6aad94e9, atime: 0x0, ctime: 0x6aad94e9, blocks: 0xc007 00010000:00010000:2.0:1789760745.922106:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2e85f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.922126:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=725s v=(0 0) 00010000:00010000:2.0:1789760745.922131:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760745.922139:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2e85f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25047040->25051135] (req 25047040->25051135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.922157:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2e85f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25047040->25051135] (req 25047040->25051135) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760745.986894:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973434 is committed 00002000:00010000:2.0:1789760745.987131:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25047040 -> 25051136 00010000:00010000:2.0:1789760745.987152:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2e85f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25047040->25051135] (req 25047040->25051135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.987167:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2e85f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25047040->25051135] (req 25047040->25051135) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.987178:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2e85f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25047040->25051135] (req 25047040->25051135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.987186:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760745.987192:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2e85f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25047040->25051135] (req 25047040->25051135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.987203:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2e85f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25047040->25051135] (req 25047040->25051135) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.987228:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2e85f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25047040->25051135] (req 25047040->25051135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.987235:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2e85f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25047040->25051135] (req 25047040->25051135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760745.987436:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5306a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512643712:12345-192.168.203.56@tcp:4:dd.0 Request processed in 65398us (65754us total) trans 4294973434 rc 0/0 00000100:00100000:2.0:1789760745.987450:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6145 00000100:00100000:2.0:1789760745.987463:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 725438037665 time 725486318791 state 0x2 ref 0 00000100:00100000:2.0:1789760745.994533:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760745.994680:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512643840 00000100:00100000:1.0:1789760745.994694:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512643840 00000100:00100000:1.0:1789760745.994807:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 725438037665 time 725486318791 state 0x0 ref 1 00000100:00100000:1.0:1789760745.994826:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 725486579207 time 725486318791 state 0x1 ref 1 00000100:00100000:1.0:1789760745.994958:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 725486579207 time 725559158535 state 0x0 ref 1 00000100:00100000:1.0:1789760745.994975:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4133480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512643840:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760745.995039:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25051136, mtime: 0x6aad94e9, atime: 0x0, ctime: 0x6aad94e9, blocks: 0xc007 00010000:00010000:1.0:1789760745.995050:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2e866 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.995070:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=725s v=(0 0) 00010000:00010000:1.0:1789760745.995076:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760745.995083:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2e866 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25051136->25055231] (req 25051136->25055231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.995100:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2e866 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25051136->25055231] (req 25051136->25055231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760746.063369:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973435 is committed 00002000:00010000:3.0:1789760746.063535:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25051136 -> 25055232 00010000:00010000:3.0:1789760746.063555:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2e866 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25051136->25055231] (req 25051136->25055231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.063570:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2e866 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25051136->25055231] (req 25051136->25055231) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.063580:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2e866 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25051136->25055231] (req 25051136->25055231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.063588:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760746.063594:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2e866 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25051136->25055231] (req 25051136->25055231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.063603:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2e866 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25051136->25055231] (req 25051136->25055231) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.063633:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2e866 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25051136->25055231] (req 25051136->25055231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.063641:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2e866 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25051136->25055231] (req 25051136->25055231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760746.063837:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4133480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512643840:12345-192.168.203.56@tcp:4:dd.0 Request processed in 68858us (69311us total) trans 4294973435 rc 0/0 00000100:00100000:3.0:1789760746.063849:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6146 00000100:00100000:3.0:1789760746.063869:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 725486579207 time 725559158535 state 0x2 ref 0 00000100:00100000:1.0:1789760746.075080:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760746.075188:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512643968 00000100:00100000:3.0:1789760746.075201:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512643968 00000100:00100000:3.0:1789760746.075291:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 725486579207 time 725559158535 state 0x0 ref 1 00000100:00100000:3.0:1789760746.075320:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 725559418951 time 725559158535 state 0x1 ref 1 00000100:00100000:3.0:1789760746.075352:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 725559418951 time 725639651700 state 0x0 ref 1 00000100:00100000:3.0:1789760746.075366:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac405ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512643968:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760746.075430:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25055232, mtime: 0x6aad94e9, atime: 0x0, ctime: 0x6aad94e9, blocks: 0xc007 00010000:00010000:3.0:1789760746.075446:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2e86d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.075466:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=725s v=(0 0) 00010000:00010000:3.0:1789760746.075472:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760746.075480:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2e86d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25055232->25059327] (req 25055232->25059327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.075496:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2e86d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25055232->25059327] (req 25055232->25059327) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760746.139627:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25055232 -> 25059328 00010000:00010000:3.0:1789760746.139654:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2e86d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25055232->25059327] (req 25055232->25059327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.139670:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2e86d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25055232->25059327] (req 25055232->25059327) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.139681:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2e86d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25055232->25059327] (req 25055232->25059327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.139689:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760746.139732:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2e86d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25055232->25059327] (req 25055232->25059327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.139742:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2e86d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25055232->25059327] (req 25055232->25059327) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.139764:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2e86d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25055232->25059327] (req 25055232->25059327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.139771:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2e86d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25055232->25059327] (req 25055232->25059327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760746.139879:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac405ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2359:x1876699512643968:12345-192.168.203.56@tcp:4:dd.0 Request processed in 64509us (64808us total) trans 4294973436 rc 0/0 00000100:00100000:3.0:1789760746.139892:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6147 00000100:00100000:3.0:1789760746.139905:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 725559418951 time 725639651700 state 0x2 ref 0 00000001:00080000:1.0:1789760746.140739:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973436 is committed 00000100:00100000:0.0:1789760746.151225:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760746.151467:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512644096 00000100:00100000:2.0:1789760746.151481:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512644096 00000100:00100000:2.0:1789760746.151560:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 725559418951 time 725639651700 state 0x0 ref 1 00000100:00100000:2.0:1789760746.151580:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 725639912116 time 725639651700 state 0x1 ref 1 00000100:00100000:2.0:1789760746.151619:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 725639912116 time 725715918452 state 0x0 ref 1 00000100:00100000:2.0:1789760746.151634:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2878380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512644096:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760746.151750:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25059328, mtime: 0x6aad94e9, atime: 0x0, ctime: 0x6aad94e9, blocks: 0xc007 00010000:00010000:2.0:1789760746.151762:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2e874 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.151782:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=725s v=(0 0) 00010000:00010000:2.0:1789760746.151787:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760746.151795:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2e874 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25059328->25063423] (req 25059328->25063423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.151814:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2e874 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25059328->25063423] (req 25059328->25063423) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760746.221357:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973437 is committed 00002000:00010000:0.0:1789760746.221773:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25059328 -> 25063424 00010000:00010000:0.0:1789760746.221800:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2e874 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25059328->25063423] (req 25059328->25063423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.221817:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2e874 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25059328->25063423] (req 25059328->25063423) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.221829:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2e874 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25059328->25063423] (req 25059328->25063423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.221840:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760746.221847:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2e874 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25059328->25063423] (req 25059328->25063423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.221860:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2e874 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25059328->25063423] (req 25059328->25063423) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.221887:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2e874 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25059328->25063423] (req 25059328->25063423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.221897:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2e874 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25059328->25063423] (req 25059328->25063423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760746.233378:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2878380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512644096:12345-192.168.203.56@tcp:4:dd.0 Request processed in 81726us (82149us total) trans 4294973437 rc 0/0 00000100:00100000:1.0:1789760746.233397:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6148 00000100:00100000:1.0:1789760746.233410:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 725639912116 time 725715918452 state 0x2 ref 0 00000100:00100000:2.0:1789760746.240918:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760746.241260:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512644224 00000100:00100000:3.0:1789760746.241274:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512644224 00000100:00100000:3.0:1789760746.241477:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 725639912116 time 725715918452 state 0x0 ref 1 00000100:00100000:3.0:1789760746.241496:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 725716178868 time 725715918452 state 0x1 ref 1 00000100:00100000:3.0:1789760746.241590:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 725716178868 time 725805888548 state 0x0 ref 1 00000100:00100000:3.0:1789760746.241607:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4130700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512644224:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760746.241675:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25063424, mtime: 0x6aad94e9, atime: 0x0, ctime: 0x6aad94e9, blocks: 0xc007 00010000:00010000:3.0:1789760746.241686:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2e87b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.241739:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=725s v=(0 0) 00010000:00010000:3.0:1789760746.241744:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760746.241752:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2e87b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25063424->25067519] (req 25063424->25067519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.241769:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2e87b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25063424->25067519] (req 25063424->25067519) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760746.292107:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973438 is committed 00002000:00010000:3.0:1789760746.292448:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25063424 -> 25067520 00010000:00010000:3.0:1789760746.292469:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2e87b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25063424->25067519] (req 25063424->25067519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.292485:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2e87b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25063424->25067519] (req 25063424->25067519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.292494:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2e87b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25063424->25067519] (req 25063424->25067519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.292502:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760746.292508:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2e87b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25063424->25067519] (req 25063424->25067519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.292519:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2e87b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25063424->25067519] (req 25063424->25067519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.292544:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2e87b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25063424->25067519] (req 25063424->25067519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.292553:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2e87b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25063424->25067519] (req 25063424->25067519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760746.293147:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4130700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512644224:12345-192.168.203.56@tcp:4:dd.0 Request processed in 51535us (52237us total) trans 4294973438 rc 0/0 00000100:00100000:3.0:1789760746.293164:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6149 00000100:00100000:3.0:1789760746.293176:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 725716178868 time 725805888548 state 0x2 ref 0 00000100:00100000:2.0:1789760746.297440:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760746.297599:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512644352 00000100:00100000:3.0:1789760746.297622:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512644352 00000100:00100000:3.0:1789760746.297731:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 725716178868 time 725805888548 state 0x0 ref 1 00000100:00100000:3.0:1789760746.297750:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 725806148964 time 725805888548 state 0x1 ref 1 00000100:00100000:3.0:1789760746.297776:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 725806148964 time 725862075368 state 0x0 ref 1 00000100:00100000:3.0:1789760746.297789:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4131180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512644352:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760746.297865:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25067520, mtime: 0x6aad94e9, atime: 0x0, ctime: 0x6aad94e9, blocks: 0xc007 00010000:00010000:3.0:1789760746.297877:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2e882 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.297898:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=725s v=(0 0) 00010000:00010000:3.0:1789760746.297913:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760746.297922:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2e882 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25067520->25071615] (req 25067520->25071615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.297937:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2e882 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25067520->25071615] (req 25067520->25071615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760746.358388:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973439 is committed 00002000:00010000:0.0:1789760746.358517:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25067520 -> 25071616 00010000:00010000:0.0:1789760746.358537:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2e882 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25067520->25071615] (req 25067520->25071615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.358553:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2e882 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25067520->25071615] (req 25067520->25071615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.358563:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2e882 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25067520->25071615] (req 25067520->25071615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.358571:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760746.358577:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2e882 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25067520->25071615] (req 25067520->25071615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.358586:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2e882 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25067520->25071615] (req 25067520->25071615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.358610:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2e882 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25067520->25071615] (req 25067520->25071615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.358619:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2e882 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25067520->25071615] (req 25067520->25071615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760746.358774:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4131180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512644352:12345-192.168.203.56@tcp:4:dd.0 Request processed in 60980us (61342us total) trans 4294973439 rc 0/0 00000100:00100000:0.0:1789760746.358786:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6150 00000100:00100000:0.0:1789760746.358798:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 725806148964 time 725862075368 state 0x2 ref 0 00000100:00100000:1.0:1789760746.364660:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760746.364900:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512644480 00000100:00100000:0.0:1789760746.364914:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512644480 00000100:00100000:0.0:1789760746.364983:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 725806148964 time 725862075368 state 0x0 ref 1 00000100:00100000:0.0:1789760746.365002:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 725862335784 time 725862075368 state 0x1 ref 1 00000100:00100000:0.0:1789760746.365038:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 725862335784 time 725929337102 state 0x0 ref 1 00000100:00100000:0.0:1789760746.365053:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4058e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512644480:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760746.365121:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25071616, mtime: 0x6aad94e9, atime: 0x0, ctime: 0x6aad94e9, blocks: 0xc007 00010000:00010000:0.0:1789760746.365132:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2e889 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.365152:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=725s v=(0 0) 00010000:00010000:0.0:1789760746.365157:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760746.365165:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2e889 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25071616->25075711] (req 25071616->25075711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.365192:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2e889 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25071616->25075711] (req 25071616->25075711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760746.413943:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25071616 -> 25075712 00010000:00010000:0.0:1789760746.413978:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2e889 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25071616->25075711] (req 25071616->25075711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.413994:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2e889 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25071616->25075711] (req 25071616->25075711) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.414003:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2e889 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25071616->25075711] (req 25071616->25075711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.414011:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760746.414017:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2e889 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25071616->25075711] (req 25071616->25075711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.414034:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2e889 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25071616->25075711] (req 25071616->25075711) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.414057:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2e889 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25071616->25075711] (req 25071616->25075711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.414065:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2e889 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25071616->25075711] (req 25071616->25075711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760746.414185:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4058e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2359:x1876699512644480:12345-192.168.203.56@tcp:4:dd.0 Request processed in 49128us (49533us total) trans 4294973440 rc 0/0 00000100:00100000:0.0:1789760746.414197:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6151 00000100:00100000:0.0:1789760746.414208:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 725862335784 time 725929337102 state 0x2 ref 0 00000001:00080000:2.0:1789760746.415003:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973440 is committed 00000100:00100000:1.0:1789760746.418998:0:3309:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1789760746.419037:0:3309:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8feac4058380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3309:1876699540224640:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1789760746.419132:0:3309:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760746.419227:0:6707:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540224640 00000100:00100000:1.0:1789760746.419237:0:6707:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540224640 00000100:00100000:1.0:1789760746.419323:0:6707:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 198 00000100:00100000:1.0:1789760746.419348:0:6707:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4217100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3309:x1876699540224640:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1789760746.419378:0:6707:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 526843904 pending 0 free 3833593856 avail 3795845120 00000100:00100000:1.0:1789760746.419429:0:6707:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4217100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3309:x1876699540224640:12345-0@lo:13:osp-pre-0-0.0 Request processed in 83us (296us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760746.419437:0:6707:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 198 00000100:00100000:1.0:1789760746.419495:0:3309:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8feac4058380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3309:1876699540224640:0@lo:13:osp-pre-0-0.0 00000100:00080000:2.0:1789760746.419990:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1789760746.420560:0:3309:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8feac5306d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3309:1876699540225024:0@lo:400:kworker.0 00000100:00100000:3.0:1789760746.420623:0:3309:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760746.420871:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760746.421115:0:3312:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8feac5304a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3312:1876699540224896:0@lo:400:kworker.0 00000100:00100000:0.0:1789760746.421154:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760746.421189:0:3312:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [2->3] 00000100:00100000:0.0:1789760746.421195:0:3312:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8feac5305880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:6df98d54-eb52-4c5c-8fc9-ee9d74f90f98:3312:1876699540224768:0@lo:400:kworker.0 00000100:00100000:0.0:1789760746.421224:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760746.421247:0:3312:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8feac5307800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3312:1876699540225280:0@lo:400:kworker.0 00000100:00100000:0.0:1789760746.421274:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760746.421318:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540224768 00000100:00100000:0.0:1789760746.421326:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540224768 00000100:00100000:2.0:1789760746.421439:0:8229:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540225024 00000100:00100000:2.0:1789760746.421444:0:8229:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540225024 00000100:00100000:2.0:1789760746.421500:0:8229:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540224896 00000100:00100000:2.0:1789760746.421503:0:8229:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540224896 00000100:00100000:2.0:1789760746.421529:0:8229:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 476 00000100:00100000:2.0:1789760746.421538:0:8229:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5304000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3309:x1876699540225024:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760746.421595:0:8229:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5304000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3309:x1876699540225024:12345-0@lo:400:kworker.0 Request processed in 58us (972us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760746.421603:0:8229:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 476 00000100:00100000:2.0:1789760746.421624:0:8229:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 477 00000100:00100000:2.0:1789760746.421630:0:8229:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac287a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3312:x1876699540224896:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760746.421772:0:3309:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8feac5306d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3309:1876699540225024:0@lo:400:kworker.0 00000100:00100000:3.0:1789760746.421794:0:3309:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1789760746.421802:0:3309:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8feac5306300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3309:1876699540225152:0@lo:400:kworker.0 00000100:00100000:0.0:1789760746.421931:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 399 00000100:00100000:0.0:1789760746.421939:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febe0501850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:6df98d54-eb52-4c5c-8fc9-ee9d74f90f98+19:3312:x1876699540224768:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760746.422200:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febe0501850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:6df98d54-eb52-4c5c-8fc9-ee9d74f90f98+19:3312:x1876699540224768:12345-0@lo:400:kworker.0 Request processed in 261us (974us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760746.422208:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 399 00000100:00100000:0.0:1789760746.422251:0:10102:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540225280 00000100:00100000:0.0:1789760746.422256:0:10102:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540225280 00000100:00100000:0.0:1789760746.422291:0:10102:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 183 00000100:00100000:0.0:1789760746.422315:0:10102:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac287b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3312:x1876699540225280:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760746.422515:0:3309:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760746.422549:0:5848:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540225152 00000100:00100000:3.0:1789760746.422553:0:5848:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540225152 00000100:00100000:3.0:1789760746.422581:0:5848:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 478 00000100:00100000:3.0:1789760746.422587:0:5848:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5306d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3309:x1876699540225152:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760746.422634:0:5848:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5306d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3309:x1876699540225152:12345-0@lo:400:kworker.0 Request processed in 47us (119us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760746.422641:0:5848:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 478 00000100:00100000:2.0:1789760746.422749:0:8229:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac287a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3312:x1876699540224896:12345-0@lo:400:kworker.0 Request processed in 1117us (1592us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760746.422757:0:8229:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 477 00000100:00080000:2.0:1789760746.422790:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:2.0:1789760746.422796:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1789760746.423159:0:3309:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8feac5306300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3309:1876699540225152:0@lo:400:kworker.0 00000100:00100000:0.0:1789760746.423263:0:10102:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac287b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3312:x1876699540225280:12345-0@lo:400:kworker.0 Request processed in 964us (1987us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760746.423272:0:10102:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 183 00000100:00100000:3.0:1789760746.423544:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512644608 00000100:00100000:3.0:1789760746.423550:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512644608 00000100:00100000:3.0:1789760746.423597:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 725862335784 time 725929337102 state 0x0 ref 1 00000100:00100000:3.0:1789760746.423613:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 725929597518 time 725929337102 state 0x1 ref 1 00000100:00100000:0.0:1789760746.423682:0:3312:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8feac5305880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:6df98d54-eb52-4c5c-8fc9-ee9d74f90f98:3312:1876699540224768:0@lo:400:kworker.0 00000100:00100000:0.0:1789760746.423714:0:3312:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8feac5307800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3312:1876699540225280:0@lo:400:kworker.0 00000100:00100000:0.0:1789760746.423728:0:3312:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8feac5304a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3312:1876699540224896:0@lo:400:kworker.0 00000100:00100000:3.0:1789760746.423798:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 725929597518 time 725988096218 state 0x0 ref 1 00000100:00100000:3.0:1789760746.423876:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4130a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512644608:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760746.423934:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25075712, mtime: 0x6aad94e9, atime: 0x0, ctime: 0x6aad94e9, blocks: 0xc007 00010000:00010000:3.0:1789760746.423945:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2e890 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.423966:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=725s v=(0 0) 00010000:00010000:3.0:1789760746.423972:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760746.423990:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2e890 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25075712->25079807] (req 25075712->25079807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.424007:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2e890 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25075712->25079807] (req 25075712->25079807) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760746.491994:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973441 is committed 00002000:00010000:3.0:1789760746.492044:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25075712 -> 25079808 00010000:00010000:3.0:1789760746.492067:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2e890 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25075712->25079807] (req 25075712->25079807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.492082:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2e890 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25075712->25079807] (req 25075712->25079807) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.492093:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2e890 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25075712->25079807] (req 25075712->25079807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.492101:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760746.492107:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2e890 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25075712->25079807] (req 25075712->25079807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.492117:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2e890 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25075712->25079807] (req 25075712->25079807) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.492279:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2e890 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25075712->25079807] (req 25075712->25079807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.492295:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2e890 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25075712->25079807] (req 25075712->25079807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760746.492584:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4130a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512644608:12345-192.168.203.56@tcp:4:dd.0 Request processed in 68703us (71772us total) trans 4294973441 rc 0/0 00000100:00100000:3.0:1789760746.492599:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6152 00000100:00100000:3.0:1789760746.492673:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 725929597518 time 725988096218 state 0x2 ref 0 00000100:00100000:0.0:1789760746.495554:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760746.495649:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512644736 00000100:00100000:3.0:1789760746.495661:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512644736 00000100:00100000:3.0:1789760746.495917:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 725929597518 time 725988096218 state 0x0 ref 1 00000100:00100000:3.0:1789760746.495936:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 725988356634 time 725988096218 state 0x1 ref 1 00000100:00100000:3.0:1789760746.496033:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 725988356634 time 726060331331 state 0x0 ref 1 00000100:00100000:3.0:1789760746.496049:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2878000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512644736:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760746.496116:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25079808, mtime: 0x6aad94e9, atime: 0x0, ctime: 0x6aad94e9, blocks: 0xc007 00010000:00010000:3.0:1789760746.496127:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2e897 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.496147:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=726s v=(0 0) 00010000:00010000:3.0:1789760746.496153:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760746.496161:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2e897 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25079808->25083903] (req 25079808->25083903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.496185:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2e897 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25079808->25083903] (req 25079808->25083903) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760746.533407:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973442 is committed 00002000:00010000:3.0:1789760746.533530:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25079808 -> 25083904 00002000:00010000:3.0:1789760746.533536:0:6709:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760745 -> 1789760746 00002000:00010000:3.0:1789760746.533539:0:6709:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760745 -> 1789760746 00010000:00010000:3.0:1789760746.533557:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2e897 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25079808->25083903] (req 25079808->25083903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.533572:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2e897 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25079808->25083903] (req 25079808->25083903) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.533583:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2e897 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25079808->25083903] (req 25079808->25083903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.533592:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760746.533598:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2e897 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25079808->25083903] (req 25079808->25083903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.533609:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2e897 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25079808->25083903] (req 25079808->25083903) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.533633:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2e897 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25079808->25083903] (req 25079808->25083903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.533649:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2e897 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25079808->25083903] (req 25079808->25083903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760746.534049:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2878000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512644736:12345-192.168.203.56@tcp:4:dd.0 Request processed in 37994us (38501us total) trans 4294973442 rc 0/0 00000100:00100000:3.0:1789760746.534063:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6153 00000100:00100000:3.0:1789760746.534076:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 725988356634 time 726060331331 state 0x2 ref 0 00000100:00100000:0.0:1789760746.538572:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760746.538666:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512644864 00000100:00100000:2.0:1789760746.538680:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512644864 00000100:00100000:2.0:1789760746.538846:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 725988356634 time 726060331331 state 0x0 ref 1 00000100:00100000:2.0:1789760746.538871:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 726060591747 time 726060331331 state 0x1 ref 1 00000100:00100000:2.0:1789760746.538903:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 726060591747 time 726103202688 state 0x0 ref 1 00000100:00100000:2.0:1789760746.538917:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac287b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512644864:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760746.538995:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25083904, mtime: 0x6aad94ea, atime: 0x0, ctime: 0x6aad94ea, blocks: 0xc007 00010000:00010000:2.0:1789760746.539008:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2e89e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.539028:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=726s v=(0 0) 00010000:00010000:2.0:1789760746.539034:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760746.539042:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2e89e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25083904->25087999] (req 25083904->25087999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.539058:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2e89e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25083904->25087999] (req 25083904->25087999) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760746.577531:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973443 is committed 00002000:00010000:3.0:1789760746.577668:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25083904 -> 25088000 00010000:00010000:3.0:1789760746.577720:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2e89e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25083904->25087999] (req 25083904->25087999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.577737:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2e89e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25083904->25087999] (req 25083904->25087999) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.577747:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2e89e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25083904->25087999] (req 25083904->25087999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.577756:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760746.577762:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2e89e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25083904->25087999] (req 25083904->25087999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.577773:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2e89e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25083904->25087999] (req 25083904->25087999) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.577801:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2e89e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25083904->25087999] (req 25083904->25087999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.577810:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2e89e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25083904->25087999] (req 25083904->25087999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760746.577927:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac287b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512644864:12345-192.168.203.56@tcp:4:dd.0 Request processed in 39004us (39362us total) trans 4294973443 rc 0/0 00000100:00100000:3.0:1789760746.577940:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6154 00000100:00100000:3.0:1789760746.577952:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 726060591747 time 726103202688 state 0x2 ref 0 00000100:00100000:0.0:1789760746.582452:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760746.582545:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512644992 00000100:00100000:3.0:1789760746.582558:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512644992 00000100:00100000:3.0:1789760746.582632:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 726060591747 time 726103202688 state 0x0 ref 1 00000100:00100000:3.0:1789760746.582650:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 726103463104 time 726103202688 state 0x1 ref 1 00000100:00100000:3.0:1789760746.582685:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 726103463104 time 726146983572 state 0x0 ref 1 00000100:00100000:3.0:1789760746.582728:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2878700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512644992:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760746.582791:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25088000, mtime: 0x6aad94ea, atime: 0x0, ctime: 0x6aad94ea, blocks: 0xc007 00010000:00010000:3.0:1789760746.582801:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2e8a5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.582821:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=726s v=(0 0) 00010000:00010000:3.0:1789760746.582827:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760746.582835:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2e8a5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25088000->25092095] (req 25088000->25092095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.582852:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2e8a5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25088000->25092095] (req 25088000->25092095) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760746.619038:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973444 is committed 00002000:00010000:1.0:1789760746.619260:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25088000 -> 25092096 00010000:00010000:1.0:1789760746.619280:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2e8a5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25088000->25092095] (req 25088000->25092095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.619307:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2e8a5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25088000->25092095] (req 25088000->25092095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.619319:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2e8a5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25088000->25092095] (req 25088000->25092095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.619328:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760746.619333:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2e8a5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25088000->25092095] (req 25088000->25092095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.619344:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2e8a5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25088000->25092095] (req 25088000->25092095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.619368:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2e8a5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25088000->25092095] (req 25088000->25092095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.619377:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2e8a5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25088000->25092095] (req 25088000->25092095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760746.619507:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2878700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512644992:12345-192.168.203.56@tcp:4:dd.0 Request processed in 36803us (37063us total) trans 4294973444 rc 0/0 00000100:00100000:1.0:1789760746.619519:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6155 00000100:00100000:1.0:1789760746.619531:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 726103463104 time 726146983572 state 0x2 ref 0 00000100:00100000:2.0:1789760746.625289:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760746.625396:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512645120 00000100:00100000:3.0:1789760746.625409:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512645120 00000100:00100000:3.0:1789760746.625485:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 726103463104 time 726146983572 state 0x0 ref 1 00000100:00100000:3.0:1789760746.625504:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 726147243988 time 726146983572 state 0x1 ref 1 00000100:00100000:3.0:1789760746.625541:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 726147243988 time 726189840662 state 0x0 ref 1 00000100:00100000:3.0:1789760746.625554:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2747b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512645120:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760746.625621:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25092096, mtime: 0x6aad94ea, atime: 0x0, ctime: 0x6aad94ea, blocks: 0xc007 00010000:00010000:3.0:1789760746.625632:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2e8ac lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.625652:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=726s v=(0 0) 00010000:00010000:3.0:1789760746.625657:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760746.625664:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2e8ac lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25092096->25096191] (req 25092096->25096191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.625682:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2e8ac lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25092096->25096191] (req 25092096->25096191) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760746.661395:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25092096 -> 25096192 00010000:00010000:3.0:1789760746.661424:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2e8ac lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25092096->25096191] (req 25092096->25096191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.661441:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2e8ac lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25092096->25096191] (req 25092096->25096191) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.661452:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2e8ac lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25092096->25096191] (req 25092096->25096191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.661461:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760746.661468:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2e8ac lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25092096->25096191] (req 25092096->25096191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.661478:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2e8ac lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25092096->25096191] (req 25092096->25096191) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.661503:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2e8ac lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25092096->25096191] (req 25092096->25096191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.661510:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2e8ac lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25092096->25096191] (req 25092096->25096191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760746.661639:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2747b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2360:x1876699512645120:12345-192.168.203.56@tcp:4:dd.0 Request processed in 36079us (36356us total) trans 4294973445 rc 0/0 00000100:00100000:3.0:1789760746.661656:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6156 00000100:00100000:3.0:1789760746.661668:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 726147243988 time 726189840662 state 0x2 ref 0 00000001:00080000:1.0:1789760746.662004:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973445 is committed 00000100:00100000:2.0:1789760746.665148:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760746.665259:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512645248 00000100:00100000:3.0:1789760746.665276:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512645248 00000100:00100000:3.0:1789760746.665358:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 726147243988 time 726189840662 state 0x0 ref 1 00000100:00100000:3.0:1789760746.665377:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 726190101078 time 726189840662 state 0x1 ref 1 00000100:00100000:3.0:1789760746.665413:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 726190101078 time 726229711797 state 0x0 ref 1 00000100:00100000:3.0:1789760746.665428:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2747800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512645248:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760746.665492:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25096192, mtime: 0x6aad94ea, atime: 0x0, ctime: 0x6aad94ea, blocks: 0xc007 00010000:00010000:3.0:1789760746.665503:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2e8b3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.665523:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=726s v=(0 0) 00010000:00010000:3.0:1789760746.665528:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760746.665536:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2e8b3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25096192->25100287] (req 25096192->25100287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.665554:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2e8b3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25096192->25100287] (req 25096192->25100287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760746.705414:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973446 is committed 00002000:00010000:3.0:1789760746.705653:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25096192 -> 25100288 00010000:00010000:3.0:1789760746.705674:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2e8b3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25096192->25100287] (req 25096192->25100287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.705823:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2e8b3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25096192->25100287] (req 25096192->25100287) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.705834:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2e8b3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25096192->25100287] (req 25096192->25100287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.705843:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760746.705849:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2e8b3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25096192->25100287] (req 25096192->25100287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.705859:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2e8b3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25096192->25100287] (req 25096192->25100287) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.705883:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2e8b3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25096192->25100287] (req 25096192->25100287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.705892:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2e8b3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25096192->25100287] (req 25096192->25100287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760746.706268:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2747800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512645248:12345-192.168.203.56@tcp:4:dd.0 Request processed in 40835us (41128us total) trans 4294973446 rc 0/0 00000100:00100000:3.0:1789760746.706282:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6157 00000100:00100000:3.0:1789760746.706304:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 726190101078 time 726229711797 state 0x2 ref 0 00000100:00100000:2.0:1789760746.715267:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760746.715457:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512645376 00000100:00100000:1.0:1789760746.715471:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512645376 00000100:00100000:1.0:1789760746.715621:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 726190101078 time 726229711797 state 0x0 ref 1 00000100:00100000:1.0:1789760746.715642:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 726229972213 time 726229711797 state 0x1 ref 1 00000100:00100000:2.0:1789760746.716291:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 726229972213 time 726280584381 state 0x0 ref 1 00000100:00100000:2.0:1789760746.716326:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2745500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512645376:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760746.716395:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25100288, mtime: 0x6aad94ea, atime: 0x0, ctime: 0x6aad94ea, blocks: 0xc007 00010000:00010000:2.0:1789760746.716406:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2e8ba lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.716427:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=726s v=(0 0) 00010000:00010000:2.0:1789760746.716433:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760746.716441:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2e8ba lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25100288->25104383] (req 25100288->25104383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.716458:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2e8ba lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25100288->25104383] (req 25100288->25104383) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760746.750144:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25100288 -> 25104384 00010000:00010000:2.0:1789760746.750172:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2e8ba lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25100288->25104383] (req 25100288->25104383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.750188:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2e8ba lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25100288->25104383] (req 25100288->25104383) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.750199:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2e8ba lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25100288->25104383] (req 25100288->25104383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.750208:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760746.750214:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2e8ba lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25100288->25104383] (req 25100288->25104383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.750226:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2e8ba lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25100288->25104383] (req 25100288->25104383) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.750258:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2e8ba lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25100288->25104383] (req 25100288->25104383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.750267:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2e8ba lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25100288->25104383] (req 25100288->25104383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760746.750400:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2745500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2359:x1876699512645376:12345-192.168.203.56@tcp:4:dd.0 Request processed in 34074us (35141us total) trans 4294973447 rc 0/0 00000100:00100000:2.0:1789760746.750413:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6158 00000100:00100000:2.0:1789760746.750425:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 726229972213 time 726280584381 state 0x2 ref 0 00000001:00080000:2.0:1789760746.750830:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973447 is committed 00000100:00100000:2.0:1789760746.755836:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760746.755945:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512645504 00000100:00100000:3.0:1789760746.755960:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512645504 00000100:00100000:3.0:1789760746.756103:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 726229972213 time 726280584381 state 0x0 ref 1 00000100:00100000:3.0:1789760746.756123:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 726280844797 time 726280584381 state 0x1 ref 1 00000100:00100000:3.0:1789760746.756201:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 726280844797 time 726320499779 state 0x0 ref 1 00000100:00100000:3.0:1789760746.756217:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2746a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512645504:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760746.756283:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25104384, mtime: 0x6aad94ea, atime: 0x0, ctime: 0x6aad94ea, blocks: 0xc007 00010000:00010000:3.0:1789760746.756294:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2e8c1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.756324:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=726s v=(0 0) 00010000:00010000:3.0:1789760746.756330:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760746.756338:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2e8c1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25104384->25108479] (req 25104384->25108479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.756354:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2e8c1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25104384->25108479] (req 25104384->25108479) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760746.793053:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25104384 -> 25108480 00010000:00010000:3.0:1789760746.793082:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2e8c1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25104384->25108479] (req 25104384->25108479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.793097:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2e8c1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25104384->25108479] (req 25104384->25108479) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.793107:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2e8c1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25104384->25108479] (req 25104384->25108479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.793115:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760746.793120:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2e8c1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25104384->25108479] (req 25104384->25108479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.793131:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2e8c1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25104384->25108479] (req 25104384->25108479) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.793157:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2e8c1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25104384->25108479] (req 25104384->25108479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.793170:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2e8c1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25104384->25108479] (req 25104384->25108479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760746.793184:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973448 is committed 00000100:00100000:3.0:1789760746.793306:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2746a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512645504:12345-192.168.203.56@tcp:4:dd.0 Request processed in 37075us (37468us total) trans 4294973448 rc 0/0 00000100:00100000:3.0:1789760746.793320:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6159 00000100:00100000:3.0:1789760746.793333:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 726280844797 time 726320499779 state 0x2 ref 0 00000100:00100000:3.0:1789760746.796255:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760746.796358:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512645632 00000100:00100000:1.0:1789760746.796371:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512645632 00000100:00100000:1.0:1789760746.796442:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 726280844797 time 726320499779 state 0x0 ref 1 00000100:00100000:1.0:1789760746.796462:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 726320760195 time 726320499779 state 0x1 ref 1 00000100:00100000:3.0:1789760746.796578:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 726320760195 time 726360873896 state 0x0 ref 1 00000100:00100000:3.0:1789760746.796597:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5304380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512645632:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760746.796664:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25108480, mtime: 0x6aad94ea, atime: 0x0, ctime: 0x6aad94ea, blocks: 0xc007 00010000:00010000:3.0:1789760746.796676:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2e8c8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.796730:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=726s v=(0 0) 00010000:00010000:3.0:1789760746.796737:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760746.796745:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2e8c8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25108480->25112575] (req 25108480->25112575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.796763:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2e8c8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25108480->25112575] (req 25108480->25112575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760746.834479:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25108480 -> 25112576 00000001:00080000:2.0:1789760746.834510:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973449 is committed 00010000:00010000:3.0:1789760746.834569:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2e8c8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25108480->25112575] (req 25108480->25112575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.834584:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2e8c8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25108480->25112575] (req 25108480->25112575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.834594:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2e8c8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25108480->25112575] (req 25108480->25112575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.834602:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760746.834607:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2e8c8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25108480->25112575] (req 25108480->25112575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.834618:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2e8c8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25108480->25112575] (req 25108480->25112575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.834646:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2e8c8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25108480->25112575] (req 25108480->25112575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.834655:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2e8c8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25108480->25112575] (req 25108480->25112575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760746.834832:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5304380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512645632:12345-192.168.203.56@tcp:4:dd.0 Request processed in 38233us (38583us total) trans 4294973449 rc 0/0 00000100:00100000:3.0:1789760746.834843:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6160 00000100:00100000:3.0:1789760746.834855:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 726320760195 time 726360873896 state 0x2 ref 0 00000100:00100000:2.0:1789760746.843624:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760746.843817:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512645760 00000100:00100000:0.0:1789760746.843831:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512645760 00000100:00100000:0.0:1789760746.843898:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 726320760195 time 726360873896 state 0x0 ref 1 00000100:00100000:0.0:1789760746.843917:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 726361134312 time 726360873896 state 0x1 ref 1 00000100:00100000:0.0:1789760746.843949:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 726361134312 time 726408248379 state 0x0 ref 1 00000100:00100000:0.0:1789760746.843963:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2745880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512645760:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760746.844038:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25112576, mtime: 0x6aad94ea, atime: 0x0, ctime: 0x6aad94ea, blocks: 0xc007 00010000:00010000:0.0:1789760746.844049:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2e8cf lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.844070:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=726s v=(0 0) 00010000:00010000:0.0:1789760746.844076:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760746.844084:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2e8cf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25112576->25116671] (req 25112576->25116671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.844101:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2e8cf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25112576->25116671] (req 25112576->25116671) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.866976:0:7647:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:2.0:1789760746.894533:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973450 is committed 00002000:00010000:1.0:1789760746.894827:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25112576 -> 25116672 00010000:00010000:1.0:1789760746.894848:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2e8cf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25112576->25116671] (req 25112576->25116671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.894863:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2e8cf lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25112576->25116671] (req 25112576->25116671) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.894872:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2e8cf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25112576->25116671] (req 25112576->25116671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.894881:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760746.894887:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2e8cf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25112576->25116671] (req 25112576->25116671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.894896:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2e8cf lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25112576->25116671] (req 25112576->25116671) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.894921:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2e8cf lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25112576->25116671] (req 25112576->25116671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.894929:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2e8cf lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25112576->25116671] (req 25112576->25116671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760746.895168:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2745880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512645760:12345-192.168.203.56@tcp:4:dd.0 Request processed in 51199us (51552us total) trans 4294973450 rc 0/0 00000100:00100000:1.0:1789760746.895180:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6161 00000100:00100000:1.0:1789760746.895192:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 726361134312 time 726408248379 state 0x2 ref 0 00000100:00100000:3.0:1789760746.899120:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760746.899202:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512645888 00000100:00100000:0.0:1789760746.899216:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512645888 00000100:00100000:0.0:1789760746.899290:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 726361134312 time 726408248379 state 0x0 ref 1 00000100:00100000:0.0:1789760746.899322:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 726408508795 time 726408248379 state 0x1 ref 1 00000100:00100000:1.0:1789760746.899446:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 726408508795 time 726463741537 state 0x0 ref 1 00000100:00100000:1.0:1789760746.899463:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5306680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512645888:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760746.899536:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25116672, mtime: 0x6aad94ea, atime: 0x0, ctime: 0x6aad94ea, blocks: 0xc007 00010000:00010000:1.0:1789760746.899547:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2e8d6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.899573:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=726s v=(0 0) 00010000:00010000:1.0:1789760746.899578:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760746.899587:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2e8d6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25116672->25120767] (req 25116672->25120767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.899604:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2e8d6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25116672->25120767] (req 25116672->25120767) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760746.932265:0:3312:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8feac2746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3312:1876699540225408:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1789760746.932569:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760746.932665:0:6707:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540225408 00000100:00100000:1.0:1789760746.932675:0:6707:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540225408 00000100:00100000:1.0:1789760746.932875:0:6707:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 199 00000100:00100000:1.0:1789760746.932887:0:6707:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4058380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3312:x1876699540225408:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1789760746.932935:0:6707:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3859808256 avail 3857711104 00000100:00100000:1.0:1789760746.933002:0:6707:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4058380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3312:x1876699540225408:12345-0@lo:13:osp-pre-1-0.0 Request processed in 117us (434us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760746.933010:0:6707:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 199 00000100:00100000:1.0:1789760746.933067:0:3312:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8feac2746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3312:1876699540225408:0@lo:13:osp-pre-1-0.0 00002000:00010000:1.0:1789760746.947773:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25116672 -> 25120768 00010000:00010000:1.0:1789760746.947799:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2e8d6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25116672->25120767] (req 25116672->25120767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.947813:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2e8d6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25116672->25120767] (req 25116672->25120767) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.947828:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2e8d6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25116672->25120767] (req 25116672->25120767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.947836:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760746.947842:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2e8d6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25116672->25120767] (req 25116672->25120767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.947852:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2e8d6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25116672->25120767] (req 25116672->25120767) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.947876:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2e8d6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25116672->25120767] (req 25116672->25120767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.947885:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2e8d6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25116672->25120767] (req 25116672->25120767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760746.948004:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5306680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2360:x1876699512645888:12345-192.168.203.56@tcp:4:dd.0 Request processed in 48537us (48892us total) trans 4294973451 rc 0/0 00000100:00100000:1.0:1789760746.948016:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6162 00000100:00100000:1.0:1789760746.948029:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 726408508795 time 726463741537 state 0x2 ref 0 00000001:00080000:3.0:1789760746.948736:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973451 is committed 00000100:00100000:2.0:1789760746.951341:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760746.951878:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512646016 00000100:00100000:3.0:1789760746.951893:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512646016 00000100:00100000:3.0:1789760746.952042:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 726408508795 time 726463741537 state 0x0 ref 1 00000100:00100000:3.0:1789760746.952064:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 726464001953 time 726463741537 state 0x1 ref 1 00000100:00100000:3.0:1789760746.952142:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 726464001953 time 726516440006 state 0x0 ref 1 00000100:00100000:3.0:1789760746.952166:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2745180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512646016:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760746.952247:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25120768, mtime: 0x6aad94ea, atime: 0x0, ctime: 0x6aad94ea, blocks: 0xc007 00010000:00010000:3.0:1789760746.952265:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2e8dd lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.952307:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=726s v=(0 0) 00010000:00010000:3.0:1789760746.952313:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760746.952321:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2e8dd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25120768->25124863] (req 25120768->25124863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.952338:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2e8dd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25120768->25124863] (req 25120768->25124863) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760746.992595:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973452 is committed 00002000:00010000:1.0:1789760746.992932:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25120768 -> 25124864 00010000:00010000:1.0:1789760746.992953:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2e8dd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25120768->25124863] (req 25120768->25124863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.992974:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2e8dd lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25120768->25124863] (req 25120768->25124863) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.992984:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2e8dd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25120768->25124863] (req 25120768->25124863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.992991:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760746.992997:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2e8dd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25120768->25124863] (req 25120768->25124863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.993008:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2e8dd lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25120768->25124863] (req 25120768->25124863) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.993039:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2e8dd lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25120768->25124863] (req 25120768->25124863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.993049:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2e8dd lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25120768->25124863] (req 25120768->25124863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760746.993183:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2745180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512646016:12345-192.168.203.56@tcp:4:dd.0 Request processed in 41021us (41854us total) trans 4294973452 rc 0/0 00000100:00100000:1.0:1789760746.993195:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6163 00000100:00100000:1.0:1789760746.993206:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 726464001953 time 726516440006 state 0x2 ref 0 00000100:00100000:0.0:1789760746.996233:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760746.996341:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512646144 00000100:00100000:1.0:1789760746.996420:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512646144 00000100:00100000:1.0:1789760746.996495:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 726464001953 time 726516440006 state 0x0 ref 1 00000100:00100000:1.0:1789760746.996519:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 726516700422 time 726516440006 state 0x1 ref 1 00000100:00100000:1.0:1789760746.996601:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 726516700422 time 726560900315 state 0x0 ref 1 00000100:00100000:1.0:1789760746.996616:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac287aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512646144:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760746.996679:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25124864, mtime: 0x6aad94ea, atime: 0x0, ctime: 0x6aad94ea, blocks: 0xc007 00010000:00010000:1.0:1789760746.996735:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2e8e4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.996757:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=726s v=(0 0) 00010000:00010000:1.0:1789760746.996762:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760746.996769:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2e8e4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25124864->25128959] (req 25124864->25128959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.996786:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2e8e4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25124864->25128959] (req 25124864->25128959) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760747.024543:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973453 is committed 00002000:00010000:0.0:1789760747.024679:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25124864 -> 25128960 00010000:00010000:0.0:1789760747.024724:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2e8e4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25124864->25128959] (req 25124864->25128959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.024740:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2e8e4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25124864->25128959] (req 25124864->25128959) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.024756:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2e8e4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25124864->25128959] (req 25124864->25128959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.024764:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760747.024770:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2e8e4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25124864->25128959] (req 25124864->25128959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.024782:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2e8e4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25124864->25128959] (req 25124864->25128959) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.024806:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2e8e4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25124864->25128959] (req 25124864->25128959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.024816:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2e8e4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25124864->25128959] (req 25124864->25128959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760747.024925:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac287aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512646144:12345-192.168.203.56@tcp:4:dd.0 Request processed in 28305us (28698us total) trans 4294973453 rc 0/0 00000100:00100000:0.0:1789760747.024937:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6164 00000100:00100000:0.0:1789760747.024949:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 726516700422 time 726560900315 state 0x2 ref 0 00000100:00100000:3.0:1789760747.028650:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760747.029124:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512646272 00000100:00100000:2.0:1789760747.029139:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512646272 00000100:00100000:2.0:1789760747.029235:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 726516700422 time 726560900315 state 0x0 ref 1 00000100:00100000:2.0:1789760747.029255:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 726561160731 time 726560900315 state 0x1 ref 1 00000100:00100000:2.0:1789760747.029303:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 726561160731 time 726593592886 state 0x0 ref 1 00000100:00100000:2.0:1789760747.029319:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5307b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512646272:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760747.029386:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25128960, mtime: 0x6aad94ea, atime: 0x0, ctime: 0x6aad94ea, blocks: 0xc007 00010000:00010000:2.0:1789760747.029398:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2e8eb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.029419:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=726s v=(0 0) 00010000:00010000:2.0:1789760747.029424:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760747.029432:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2e8eb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25128960->25133055] (req 25128960->25133055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.029449:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2e8eb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25128960->25133055] (req 25128960->25133055) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1789760747.030160:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:2.0:1789760747.059616:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973454 is committed 00002000:00010000:2.0:1789760747.059783:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25128960 -> 25133056 00010000:00010000:2.0:1789760747.059804:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2e8eb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25128960->25133055] (req 25128960->25133055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.059818:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2e8eb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25128960->25133055] (req 25128960->25133055) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.059825:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2e8eb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25128960->25133055] (req 25128960->25133055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.059831:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760747.059835:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2e8eb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25128960->25133055] (req 25128960->25133055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.059843:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2e8eb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25128960->25133055] (req 25128960->25133055) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.059864:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2e8eb lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25128960->25133055] (req 25128960->25133055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.059869:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2e8eb lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25128960->25133055] (req 25128960->25133055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760747.060029:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5307b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512646272:12345-192.168.203.56@tcp:4:dd.0 Request processed in 30650us (31331us total) trans 4294973454 rc 0/0 00000100:00100000:2.0:1789760747.060040:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6165 00000100:00100000:2.0:1789760747.060050:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 726561160731 time 726593592886 state 0x2 ref 0 00000100:00100000:2.0:1789760747.063067:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760747.063271:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512646400 00000100:00100000:3.0:1789760747.063284:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512646400 00000100:00100000:3.0:1789760747.063364:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 726561160731 time 726593592886 state 0x0 ref 1 00000100:00100000:3.0:1789760747.063382:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 726593853302 time 726593592886 state 0x1 ref 1 00000100:00100000:3.0:1789760747.063455:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 726593853302 time 726627753117 state 0x0 ref 1 00000100:00100000:3.0:1789760747.063471:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2744a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512646400:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760747.063536:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25133056, mtime: 0x6aad94ea, atime: 0x0, ctime: 0x6aad94ea, blocks: 0xc007 00010000:00010000:3.0:1789760747.063546:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2e8f2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.063568:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=726s v=(0 0) 00010000:00010000:3.0:1789760747.063573:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760747.063580:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2e8f2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25133056->25137151] (req 25133056->25137151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.063597:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2e8f2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25133056->25137151] (req 25133056->25137151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760747.088216:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973455 is committed 00002000:00010000:3.0:1789760747.088339:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25133056 -> 25137152 00010000:00010000:3.0:1789760747.088361:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2e8f2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25133056->25137151] (req 25133056->25137151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.088377:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2e8f2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25133056->25137151] (req 25133056->25137151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.088388:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2e8f2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25133056->25137151] (req 25133056->25137151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.088396:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760747.088403:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2e8f2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25133056->25137151] (req 25133056->25137151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.088415:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2e8f2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25133056->25137151] (req 25133056->25137151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.088442:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2e8f2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25133056->25137151] (req 25133056->25137151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.088452:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2e8f2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25133056->25137151] (req 25133056->25137151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760747.088614:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2744a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512646400:12345-192.168.203.56@tcp:4:dd.0 Request processed in 25139us (25552us total) trans 4294973455 rc 0/0 00000100:00100000:3.0:1789760747.088626:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6166 00000100:00100000:3.0:1789760747.088639:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 726593853302 time 726627753117 state 0x2 ref 0 00000100:00100000:0.0:1789760747.092489:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760747.092602:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512646528 00000100:00100000:1.0:1789760747.092615:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512646528 00000100:00100000:1.0:1789760747.092726:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 726593853302 time 726627753117 state 0x0 ref 1 00000100:00100000:1.0:1789760747.092746:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 726628013533 time 726627753117 state 0x1 ref 1 00000100:00100000:1.0:1789760747.092776:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 726628013533 time 726657075262 state 0x0 ref 1 00000100:00100000:1.0:1789760747.092794:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2879880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512646528:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760747.092853:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25137152, mtime: 0x6aad94ea, atime: 0x0, ctime: 0x6aad94ea, blocks: 0xc007 00010000:00010000:1.0:1789760747.092863:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2e8f9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760747.092883:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=726s v=(0 0) 00010000:00010000:1.0:1789760747.092888:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760747.092895:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2e8f9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25137152->25141247] (req 25137152->25141247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760747.092911:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2e8f9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25137152->25141247] (req 25137152->25141247) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760747.120140:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973456 is committed 00002000:00010000:3.0:1789760747.120342:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25137152 -> 25141248 00010000:00010000:3.0:1789760747.120362:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2e8f9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25137152->25141247] (req 25137152->25141247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.120376:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2e8f9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25137152->25141247] (req 25137152->25141247) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.120387:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2e8f9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25137152->25141247] (req 25137152->25141247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.120396:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760747.120402:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2e8f9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25137152->25141247] (req 25137152->25141247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.120412:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2e8f9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25137152->25141247] (req 25137152->25141247) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.120437:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2e8f9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25137152->25141247] (req 25137152->25141247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.120451:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2e8f9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25137152->25141247] (req 25137152->25141247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760747.120562:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2879880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512646528:12345-192.168.203.56@tcp:4:dd.0 Request processed in 27763us (28081us total) trans 4294973456 rc 0/0 00000100:00100000:3.0:1789760747.120574:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6167 00000100:00100000:3.0:1789760747.120587:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 726628013533 time 726657075262 state 0x2 ref 0 00000100:00100000:3.0:1789760747.124428:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760747.124627:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512646656 00000100:00100000:2.0:1789760747.124640:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512646656 00000100:00100000:2.0:1789760747.124757:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 726628013533 time 726657075262 state 0x0 ref 1 00000100:00100000:2.0:1789760747.124777:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 726657335678 time 726657075262 state 0x1 ref 1 00000100:00100000:2.0:1789760747.124804:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 726657335678 time 726689104068 state 0x0 ref 1 00000100:00100000:2.0:1789760747.124819:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5307100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512646656:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760747.124888:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25141248, mtime: 0x6aad94ea, atime: 0x0, ctime: 0x6aad94ea, blocks: 0xc007 00010000:00010000:2.0:1789760747.124899:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2e900 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.124920:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=726s v=(0 0) 00010000:00010000:2.0:1789760747.124925:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760747.124932:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2e900 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25141248->25145343] (req 25141248->25145343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.124948:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2e900 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25141248->25145343] (req 25141248->25145343) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760747.163146:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25141248 -> 25145344 00010000:00010000:0.0:1789760747.163180:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2e900 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25141248->25145343] (req 25141248->25145343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.163197:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2e900 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25141248->25145343] (req 25141248->25145343) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.163208:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2e900 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25141248->25145343] (req 25141248->25145343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.163217:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760747.163222:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2e900 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25141248->25145343] (req 25141248->25145343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.163233:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2e900 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25141248->25145343] (req 25141248->25145343) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.163258:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2e900 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25141248->25145343] (req 25141248->25145343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.163267:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2e900 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25141248->25145343] (req 25141248->25145343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760747.163390:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5307100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2358:x1876699512646656:12345-192.168.203.56@tcp:4:dd.0 Request processed in 38567us (38970us total) trans 4294973457 rc 0/0 00000100:00100000:0.0:1789760747.163403:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6168 00000100:00100000:0.0:1789760747.163415:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 726657335678 time 726689104068 state 0x2 ref 0 00000001:00080000:1.0:1789760747.163782:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973457 is committed 00000100:00100000:2.0:1789760747.167657:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760747.167833:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512646784 00000100:00100000:3.0:1789760747.167848:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512646784 00000100:00100000:3.0:1789760747.167944:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 726657335678 time 726689104068 state 0x0 ref 1 00000100:00100000:3.0:1789760747.168029:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 726689364484 time 726689104068 state 0x1 ref 1 00000100:00100000:3.0:1789760747.168113:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 726689364484 time 726732410176 state 0x0 ref 1 00000100:00100000:3.0:1789760747.168128:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2746680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512646784:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760747.168196:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25145344, mtime: 0x6aad94ea, atime: 0x0, ctime: 0x6aad94ea, blocks: 0xc007 00010000:00010000:3.0:1789760747.168207:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2e907 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.168228:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=726s v=(0 0) 00010000:00010000:3.0:1789760747.168234:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760747.168241:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2e907 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25145344->25149439] (req 25145344->25149439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.168258:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2e907 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25145344->25149439] (req 25145344->25149439) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760747.212559:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973458 is committed 00002000:00010000:1.0:1789760747.212668:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25145344 -> 25149440 00010000:00010000:1.0:1789760747.212691:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2e907 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25145344->25149439] (req 25145344->25149439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760747.212777:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2e907 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25145344->25149439] (req 25145344->25149439) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760747.212786:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2e907 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25145344->25149439] (req 25145344->25149439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760747.212793:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760747.212799:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2e907 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25145344->25149439] (req 25145344->25149439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760747.212879:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2e907 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25145344->25149439] (req 25145344->25149439) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760747.212904:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2e907 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25145344->25149439] (req 25145344->25149439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760747.212911:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2e907 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25145344->25149439] (req 25145344->25149439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760747.213149:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2746680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512646784:12345-192.168.203.56@tcp:4:dd.0 Request processed in 45016us (45500us total) trans 4294973458 rc 0/0 00000100:00100000:1.0:1789760747.213161:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6169 00000100:00100000:1.0:1789760747.213174:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 726689364484 time 726732410176 state 0x2 ref 0 00000100:00100000:2.0:1789760747.217771:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760747.218019:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512646912 00000100:00100000:3.0:1789760747.218034:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512646912 00000100:00100000:3.0:1789760747.218124:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 726689364484 time 726732410176 state 0x0 ref 1 00000100:00100000:3.0:1789760747.218142:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 726732670592 time 726732410176 state 0x1 ref 1 00000100:00100000:3.0:1789760747.218217:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 726732670592 time 726782516277 state 0x0 ref 1 00000100:00100000:3.0:1789760747.218233:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2746300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512646912:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760747.218311:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25149440, mtime: 0x6aad94ea, atime: 0x0, ctime: 0x6aad94ea, blocks: 0xc007 00010000:00010000:3.0:1789760747.218322:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2e90e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.218343:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=726s v=(0 0) 00010000:00010000:3.0:1789760747.218348:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760747.218356:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2e90e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25149440->25153535] (req 25149440->25153535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.218373:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2e90e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25149440->25153535] (req 25149440->25153535) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760747.259219:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973459 is committed 00002000:00010000:0.0:1789760747.259779:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25149440 -> 25153536 00010000:00010000:0.0:1789760747.259867:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2e90e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25149440->25153535] (req 25149440->25153535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.259884:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2e90e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25149440->25153535] (req 25149440->25153535) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.259895:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2e90e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25149440->25153535] (req 25149440->25153535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.259903:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760747.259909:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2e90e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25149440->25153535] (req 25149440->25153535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.259921:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2e90e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25149440->25153535] (req 25149440->25153535) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.259950:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2e90e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25149440->25153535] (req 25149440->25153535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.259959:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2e90e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25149440->25153535] (req 25149440->25153535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760747.260201:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2746300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512646912:12345-192.168.203.56@tcp:4:dd.0 Request processed in 41965us (42438us total) trans 4294973459 rc 0/0 00000100:00100000:0.0:1789760747.260214:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6170 00000100:00100000:0.0:1789760747.260228:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 726732670592 time 726782516277 state 0x2 ref 0 00000100:00100000:0.0:1789760747.263862:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760747.264043:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512647040 00000100:00100000:2.0:1789760747.264058:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512647040 00000100:00100000:2.0:1789760747.264141:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 726732670592 time 726782516277 state 0x0 ref 1 00000100:00100000:2.0:1789760747.264162:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 726782776693 time 726782516277 state 0x1 ref 1 00000100:00100000:2.0:1789760747.264227:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 726782776693 time 726828525410 state 0x0 ref 1 00000100:00100000:2.0:1789760747.264243:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2879180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512647040:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760747.264319:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25153536, mtime: 0x6aad94ea, atime: 0x0, ctime: 0x6aad94ea, blocks: 0xc007 00010000:00010000:2.0:1789760747.264331:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2e915 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.264354:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=726s v=(0 0) 00010000:00010000:2.0:1789760747.264360:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760747.264368:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2e915 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25153536->25157631] (req 25153536->25157631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.264385:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2e915 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25153536->25157631] (req 25153536->25157631) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760747.310244:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973460 is committed 00002000:00010000:2.0:1789760747.310363:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25153536 -> 25157632 00010000:00010000:2.0:1789760747.310386:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2e915 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25153536->25157631] (req 25153536->25157631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.310402:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2e915 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25153536->25157631] (req 25153536->25157631) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.310412:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2e915 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25153536->25157631] (req 25153536->25157631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.310494:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760747.310500:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2e915 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25153536->25157631] (req 25153536->25157631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.310511:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2e915 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25153536->25157631] (req 25153536->25157631) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.310537:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2e915 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25153536->25157631] (req 25153536->25157631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.310547:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2e915 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25153536->25157631] (req 25153536->25157631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760747.310995:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2879180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512647040:12345-192.168.203.56@tcp:4:dd.0 Request processed in 46747us (47202us total) trans 4294973460 rc 0/0 00000100:00100000:2.0:1789760747.311010:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6171 00000100:00100000:2.0:1789760747.311025:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 726782776693 time 726828525410 state 0x2 ref 0 00000100:00100000:2.0:1789760747.314734:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760747.314937:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512647168 00000100:00100000:1.0:1789760747.314952:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512647168 00000100:00100000:1.0:1789760747.315036:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 726782776693 time 726828525410 state 0x0 ref 1 00000100:00100000:1.0:1789760747.315056:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 726828785826 time 726828525410 state 0x1 ref 1 00000100:00100000:1.0:1789760747.315081:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 726828785826 time 726879380852 state 0x0 ref 1 00000100:00100000:1.0:1789760747.315096:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febea651500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512647168:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760747.315162:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25157632, mtime: 0x6aad94ea, atime: 0x0, ctime: 0x6aad94ea, blocks: 0xc007 00010000:00010000:1.0:1789760747.315173:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2e91c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760747.315194:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=726s v=(0 0) 00010000:00010000:1.0:1789760747.315200:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760747.315207:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2e91c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25157632->25161727] (req 25157632->25161727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760747.315223:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2e91c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25157632->25161727] (req 25157632->25161727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760747.378340:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973461 is committed 00002000:00010000:1.0:1789760747.378604:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25157632 -> 25161728 00010000:00010000:1.0:1789760747.378625:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2e91c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25157632->25161727] (req 25157632->25161727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760747.378641:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2e91c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25157632->25161727] (req 25157632->25161727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760747.378652:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2e91c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25157632->25161727] (req 25157632->25161727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760747.378661:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760747.378667:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2e91c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25157632->25161727] (req 25157632->25161727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760747.378679:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2e91c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25157632->25161727] (req 25157632->25161727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760747.378977:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2e91c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25157632->25161727] (req 25157632->25161727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760747.378986:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2e91c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25157632->25161727] (req 25157632->25161727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760747.379104:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febea651500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512647168:12345-192.168.203.56@tcp:4:dd.0 Request processed in 64004us (64425us total) trans 4294973461 rc 0/0 00000100:00100000:1.0:1789760747.379116:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6172 00000100:00100000:1.0:1789760747.379128:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 726828785826 time 726879380852 state 0x2 ref 0 00000100:00100000:0.0:1789760747.382557:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760747.383391:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512647296 00000100:00100000:2.0:1789760747.383408:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512647296 00000100:00100000:2.0:1789760747.383639:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 726828785826 time 726879380852 state 0x0 ref 1 00000100:00100000:2.0:1789760747.383660:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 726879641268 time 726879380852 state 0x1 ref 1 00000100:00100000:2.0:1789760747.383769:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 726879641268 time 726948067400 state 0x0 ref 1 00000100:00100000:2.0:1789760747.383785:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2879500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512647296:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760747.384100:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25161728, mtime: 0x6aad94ea, atime: 0x0, ctime: 0x6aad94ea, blocks: 0xc007 00010000:00010000:2.0:1789760747.384111:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2e923 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.384132:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=726s v=(0 0) 00010000:00010000:2.0:1789760747.384137:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760747.384145:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2e923 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25161728->25165823] (req 25161728->25165823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.384165:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2e923 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25161728->25165823] (req 25161728->25165823) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760747.460242:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973462 is committed 00002000:00010000:0.0:1789760747.460564:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25161728 -> 25165824 00010000:00010000:0.0:1789760747.460585:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2e923 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25161728->25165823] (req 25161728->25165823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.460600:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2e923 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25161728->25165823] (req 25161728->25165823) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.460610:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2e923 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25161728->25165823] (req 25161728->25165823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.460619:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760747.460624:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2e923 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25161728->25165823] (req 25161728->25165823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.460635:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2e923 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25161728->25165823] (req 25161728->25165823) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.460662:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2e923 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25161728->25165823] (req 25161728->25165823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.460672:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2e923 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25161728->25165823] (req 25161728->25165823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760747.460939:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2879500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512647296:12345-192.168.203.56@tcp:4:dd.0 Request processed in 77150us (78390us total) trans 4294973462 rc 0/0 00000100:00100000:0.0:1789760747.460953:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6173 00000100:00100000:0.0:1789760747.460973:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 726879641268 time 726948067400 state 0x2 ref 0 00000100:00100000:3.0:1789760747.465804:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760747.466091:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512647424 00000100:00100000:2.0:1789760747.466105:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512647424 00000100:00100000:2.0:1789760747.466179:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 726879641268 time 726948067400 state 0x0 ref 1 00000100:00100000:2.0:1789760747.466198:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 726948327816 time 726948067400 state 0x1 ref 1 00000100:00100000:2.0:1789760747.466242:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 726948327816 time 727030541005 state 0x0 ref 1 00000100:00100000:2.0:1789760747.466255:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5304a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512647424:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760747.466335:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25165824, mtime: 0x6aad94ea, atime: 0x0, ctime: 0x6aad94ea, blocks: 0xc007 00010000:00010000:2.0:1789760747.466347:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2e92a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.466369:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=727s v=(0 0) 00010000:00010000:2.0:1789760747.466374:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760747.466382:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2e92a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25165824->25169919] (req 25165824->25169919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.466398:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2e92a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25165824->25169919] (req 25165824->25169919) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760747.537025:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25165824 -> 25169920 00002000:00010000:0.0:1789760747.537038:0:13063:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb blocks from disk: 49159 -> 51207 00010000:00010000:0.0:1789760747.537058:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2e92a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25165824->25169919] (req 25165824->25169919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.537074:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2e92a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25165824->25169919] (req 25165824->25169919) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.537085:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2e92a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25165824->25169919] (req 25165824->25169919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.537094:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760747.537100:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2e92a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25165824->25169919] (req 25165824->25169919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.537110:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2e92a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25165824->25169919] (req 25165824->25169919) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.537135:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2e92a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25165824->25169919] (req 25165824->25169919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.537142:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2e92a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25165824->25169919] (req 25165824->25169919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760747.537282:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5304a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2361:x1876699512647424:12345-192.168.203.56@tcp:4:dd.0 Request processed in 71021us (71502us total) trans 4294973463 rc 0/0 00000100:00100000:0.0:1789760747.537303:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6174 00000100:00100000:0.0:1789760747.537316:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 726948327816 time 727030541005 state 0x2 ref 0 00000001:00080000:3.0:1789760747.537935:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973463 is committed 00000100:00100000:3.0:1789760747.542444:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760747.542555:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512647552 00000100:00100000:1.0:1789760747.542569:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512647552 00000100:00100000:1.0:1789760747.545172:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 726948327816 time 727030541005 state 0x0 ref 1 00000100:00100000:1.0:1789760747.545207:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 727030801421 time 727030541005 state 0x1 ref 1 00000100:00100000:1.0:1789760747.545336:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 727030801421 time 727109633467 state 0x0 ref 1 00000100:00100000:1.0:1789760747.545366:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5307800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512647552:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760747.545442:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25169920, mtime: 0x6aad94ea, atime: 0x0, ctime: 0x6aad94ea, blocks: 0xc807 00010000:00010000:1.0:1789760747.545454:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2e931 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760747.545475:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=727s v=(0 0) 00010000:00010000:1.0:1789760747.545493:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760747.545502:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2e931 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25169920->25174015] (req 25169920->25174015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760747.545525:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2e931 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25169920->25174015] (req 25169920->25174015) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760747.583921:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973464 is committed 00002000:00010000:3.0:1789760747.584138:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25169920 -> 25174016 00002000:00010000:3.0:1789760747.584144:0:6708:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760746 -> 1789760747 00002000:00010000:3.0:1789760747.584147:0:6708:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760746 -> 1789760747 00010000:00010000:3.0:1789760747.584163:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2e931 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25169920->25174015] (req 25169920->25174015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.584178:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2e931 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25169920->25174015] (req 25169920->25174015) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.584259:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2e931 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25169920->25174015] (req 25169920->25174015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.584270:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760747.584277:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2e931 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25169920->25174015] (req 25169920->25174015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.584287:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2e931 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25169920->25174015] (req 25169920->25174015) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.584324:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2e931 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25169920->25174015] (req 25169920->25174015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.584336:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2e931 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25169920->25174015] (req 25169920->25174015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760747.586896:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5307800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512647552:12345-192.168.203.56@tcp:4:dd.0 Request processed in 41531us (44451us total) trans 4294973464 rc 0/0 00000100:00100000:3.0:1789760747.586913:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6175 00000100:00100000:3.0:1789760747.586925:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 727030801421 time 727109633467 state 0x2 ref 0 00000100:00100000:3.0:1789760747.591181:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760747.591690:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512647680 00000100:00100000:0.0:1789760747.591806:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512647680 00000100:00100000:0.0:1789760747.591884:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 727030801421 time 727109633467 state 0x0 ref 1 00000100:00100000:0.0:1789760747.591905:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 727109893883 time 727109633467 state 0x1 ref 1 00000100:00100000:0.0:1789760747.592010:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 727109893883 time 727156307660 state 0x0 ref 1 00000100:00100000:0.0:1789760747.592026:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5305880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512647680:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760747.592094:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25174016, mtime: 0x6aad94eb, atime: 0x0, ctime: 0x6aad94eb, blocks: 0xc807 00010000:00010000:0.0:1789760747.592106:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2e938 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.592126:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=727s v=(0 0) 00010000:00010000:0.0:1789760747.592132:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760747.592140:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2e938 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25174016->25178111] (req 25174016->25178111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.592157:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2e938 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25174016->25178111] (req 25174016->25178111) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760747.651114:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973465 is committed 00002000:00010000:3.0:1789760747.651492:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25174016 -> 25178112 00010000:00010000:3.0:1789760747.651513:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2e938 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25174016->25178111] (req 25174016->25178111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.651529:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2e938 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25174016->25178111] (req 25174016->25178111) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.651539:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2e938 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25174016->25178111] (req 25174016->25178111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.651546:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760747.651553:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2e938 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25174016->25178111] (req 25174016->25178111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.651563:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2e938 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25174016->25178111] (req 25174016->25178111) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.654034:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2e938 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25174016->25178111] (req 25174016->25178111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.654060:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2e938 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25174016->25178111] (req 25174016->25178111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760747.654640:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5305880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512647680:12345-192.168.203.56@tcp:4:dd.0 Request processed in 62608us (63462us total) trans 4294973465 rc 0/0 00000100:00100000:3.0:1789760747.654655:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6176 00000100:00100000:3.0:1789760747.654668:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 727109893883 time 727156307660 state 0x2 ref 0 00000100:00100000:0.0:1789760747.658959:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760747.659142:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512647808 00000100:00100000:2.0:1789760747.659156:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512647808 00000100:00100000:2.0:1789760747.659255:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 727109893883 time 727156307660 state 0x0 ref 1 00000100:00100000:2.0:1789760747.659276:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 727156568076 time 727156307660 state 0x1 ref 1 00000100:00100000:2.0:1789760747.659317:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 727156568076 time 727223615228 state 0x0 ref 1 00000100:00100000:2.0:1789760747.659332:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac3352300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512647808:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760747.659404:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25178112, mtime: 0x6aad94eb, atime: 0x0, ctime: 0x6aad94eb, blocks: 0xc807 00010000:00010000:2.0:1789760747.659416:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2e93f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.659436:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=727s v=(0 0) 00010000:00010000:2.0:1789760747.659441:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760747.659450:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2e93f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25178112->25182207] (req 25178112->25182207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.659467:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2e93f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25178112->25182207] (req 25178112->25182207) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760747.707504:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973466 is committed 00002000:00010000:2.0:1789760747.707637:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25178112 -> 25182208 00010000:00010000:2.0:1789760747.707658:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2e93f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25178112->25182207] (req 25178112->25182207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.707674:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2e93f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25178112->25182207] (req 25178112->25182207) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.707690:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2e93f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25178112->25182207] (req 25178112->25182207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.707729:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760747.707735:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2e93f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25178112->25182207] (req 25178112->25182207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.707746:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2e93f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25178112->25182207] (req 25178112->25182207) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.707770:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2e93f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25178112->25182207] (req 25178112->25182207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.707778:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2e93f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25178112->25182207] (req 25178112->25182207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760747.707902:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac3352300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512647808:12345-192.168.203.56@tcp:4:dd.0 Request processed in 48566us (48951us total) trans 4294973466 rc 0/0 00000100:00100000:2.0:1789760747.707915:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6177 00000100:00100000:2.0:1789760747.707927:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 727156568076 time 727223615228 state 0x2 ref 0 00000100:00100000:2.0:1789760747.714161:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760747.714262:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512647936 00000100:00100000:1.0:1789760747.714278:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512647936 00000100:00100000:1.0:1789760747.714378:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 727156568076 time 727223615228 state 0x0 ref 1 00000100:00100000:1.0:1789760747.714399:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 727223875644 time 727223615228 state 0x1 ref 1 00000100:00100000:1.0:1789760747.714434:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 727223875644 time 727278733361 state 0x0 ref 1 00000100:00100000:1.0:1789760747.714449:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febea650380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512647936:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760747.714516:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25182208, mtime: 0x6aad94eb, atime: 0x0, ctime: 0x6aad94eb, blocks: 0xc807 00010000:00010000:1.0:1789760747.714529:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2e946 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760747.714549:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=727s v=(0 0) 00010000:00010000:1.0:1789760747.714554:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760747.714562:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2e946 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25182208->25186303] (req 25182208->25186303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760747.714579:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2e946 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25182208->25186303] (req 25182208->25186303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760747.758624:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973467 is committed 00002000:00010000:1.0:1789760747.758815:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25182208 -> 25186304 00010000:00010000:1.0:1789760747.758836:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2e946 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25182208->25186303] (req 25182208->25186303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760747.758850:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2e946 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25182208->25186303] (req 25182208->25186303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760747.758860:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2e946 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25182208->25186303] (req 25182208->25186303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760747.758868:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760747.758874:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2e946 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25182208->25186303] (req 25182208->25186303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760747.758884:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2e946 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25182208->25186303] (req 25182208->25186303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760747.758909:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2e946 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25182208->25186303] (req 25182208->25186303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760747.758919:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2e946 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25182208->25186303] (req 25182208->25186303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760747.759153:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febea650380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512647936:12345-192.168.203.56@tcp:4:dd.0 Request processed in 44699us (45003us total) trans 4294973467 rc 0/0 00000100:00100000:1.0:1789760747.759170:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6178 00000100:00100000:1.0:1789760747.759182:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 727223875644 time 727278733361 state 0x2 ref 0 00000100:00100000:3.0:1789760747.762906:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760747.763063:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512648064 00000100:00100000:0.0:1789760747.763078:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512648064 00000100:00100000:0.0:1789760747.763151:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 727223875644 time 727278733361 state 0x0 ref 1 00000100:00100000:0.0:1789760747.763170:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 727278993777 time 727278733361 state 0x1 ref 1 00000100:00100000:0.0:1789760747.763241:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 727278993777 time 727327538788 state 0x0 ref 1 00000100:00100000:0.0:1789760747.763256:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5306300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512648064:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760747.763409:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25186304, mtime: 0x6aad94eb, atime: 0x0, ctime: 0x6aad94eb, blocks: 0xc807 00010000:00010000:0.0:1789760747.763420:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2e94d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.763439:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=727s v=(0 0) 00010000:00010000:0.0:1789760747.763445:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760747.763454:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2e94d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25186304->25190399] (req 25186304->25190399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.763469:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2e94d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25186304->25190399] (req 25186304->25190399) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760747.810461:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973468 is committed 00002000:00010000:3.0:1789760747.810596:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25186304 -> 25190400 00010000:00010000:3.0:1789760747.810616:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2e94d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25186304->25190399] (req 25186304->25190399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.810634:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2e94d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25186304->25190399] (req 25186304->25190399) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.810645:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2e94d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25186304->25190399] (req 25186304->25190399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.810654:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760747.810660:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2e94d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25186304->25190399] (req 25186304->25190399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.810671:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2e94d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25186304->25190399] (req 25186304->25190399) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.810766:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2e94d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25186304->25190399] (req 25186304->25190399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.810777:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2e94d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25186304->25190399] (req 25186304->25190399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760747.810906:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5306300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512648064:12345-192.168.203.56@tcp:4:dd.0 Request processed in 47645us (48008us total) trans 4294973468 rc 0/0 00000100:00100000:3.0:1789760747.810919:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6179 00000100:00100000:3.0:1789760747.810932:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 727278993777 time 727327538788 state 0x2 ref 0 00000100:00100000:3.0:1789760747.817624:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760747.817892:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512648192 00000100:00100000:2.0:1789760747.817906:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512648192 00000100:00100000:2.0:1789760747.817985:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 727278993777 time 727327538788 state 0x0 ref 1 00000100:00100000:2.0:1789760747.818005:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 727327799204 time 727327538788 state 0x1 ref 1 00000100:00100000:2.0:1789760747.818032:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 727327799204 time 727382331389 state 0x0 ref 1 00000100:00100000:2.0:1789760747.818046:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2875c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512648192:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760747.818110:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25190400, mtime: 0x6aad94eb, atime: 0x0, ctime: 0x6aad94eb, blocks: 0xc807 00010000:00010000:2.0:1789760747.818122:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2e954 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.818147:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=727s v=(0 0) 00010000:00010000:2.0:1789760747.818152:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760747.818160:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2e954 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25190400->25194495] (req 25190400->25194495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.818177:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2e954 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25190400->25194495] (req 25190400->25194495) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760747.873215:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973469 is committed 00002000:00010000:2.0:1789760747.873419:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25190400 -> 25194496 00010000:00010000:2.0:1789760747.873441:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2e954 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25190400->25194495] (req 25190400->25194495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.873456:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2e954 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25190400->25194495] (req 25190400->25194495) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.873466:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2e954 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25190400->25194495] (req 25190400->25194495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.873475:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760747.873481:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2e954 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25190400->25194495] (req 25190400->25194495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.873491:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2e954 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25190400->25194495] (req 25190400->25194495) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.873516:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2e954 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25190400->25194495] (req 25190400->25194495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.873525:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2e954 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25190400->25194495] (req 25190400->25194495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760747.873736:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2875c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512648192:12345-192.168.203.56@tcp:4:dd.0 Request processed in 55635us (56069us total) trans 4294973469 rc 0/0 00000100:00100000:2.0:1789760747.873750:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6180 00000100:00100000:2.0:1789760747.873762:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 727327799204 time 727382331389 state 0x2 ref 0 00000100:00100000:2.0:1789760747.884889:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760747.887504:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512648320 00000100:00100000:1.0:1789760747.887525:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512648320 00000100:00100000:1.0:1789760747.887614:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 727327799204 time 727382331389 state 0x0 ref 1 00000100:00100000:1.0:1789760747.887635:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 727382591805 time 727382331389 state 0x1 ref 1 00000100:00100000:1.0:1789760747.888154:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 727382591805 time 727452452750 state 0x0 ref 1 00000100:00100000:1.0:1789760747.888172:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febfc0a3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512648320:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760747.888358:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25194496, mtime: 0x6aad94eb, atime: 0x0, ctime: 0x6aad94eb, blocks: 0xc807 00010000:00010000:1.0:1789760747.888369:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2e95b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760747.888389:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=727s v=(0 0) 00010000:00010000:1.0:1789760747.888394:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760747.888401:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2e95b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25194496->25198591] (req 25194496->25198591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760747.888419:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2e95b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25194496->25198591] (req 25194496->25198591) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.896910:0:5836:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:1.0:1789760747.946083:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25194496 -> 25198592 00010000:00010000:1.0:1789760747.946113:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2e95b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25194496->25198591] (req 25194496->25198591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760747.946128:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2e95b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25194496->25198591] (req 25194496->25198591) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760747.946139:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2e95b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25194496->25198591] (req 25194496->25198591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760747.946147:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760747.946153:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2e95b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25194496->25198591] (req 25194496->25198591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760747.946163:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2e95b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25194496->25198591] (req 25194496->25198591) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760747.946192:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2e95b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25194496->25198591] (req 25194496->25198591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760747.946213:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2e95b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25194496->25198591] (req 25194496->25198591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760747.946348:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febfc0a3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2358:x1876699512648320:12345-192.168.203.56@tcp:4:dd.0 Request processed in 58175us (61468us total) trans 4294973470 rc 0/0 00000100:00100000:1.0:1789760747.946361:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6181 00000100:00100000:1.0:1789760747.946374:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 727382591805 time 727452452750 state 0x2 ref 0 00000001:00080000:0.0:1789760747.947470:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973470 is committed 00000100:00100000:3.0:1789760747.951934:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760747.952175:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512648448 00000100:00100000:0.0:1789760747.952190:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512648448 00000100:00100000:0.0:1789760747.952275:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 727382591805 time 727452452750 state 0x0 ref 1 00000100:00100000:0.0:1789760747.952307:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 727452713166 time 727452452750 state 0x1 ref 1 00000100:00100000:0.0:1789760747.952402:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 727452713166 time 727516700122 state 0x0 ref 1 00000100:00100000:0.0:1789760747.952418:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febea4b5450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512648448:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760747.952486:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25198592, mtime: 0x6aad94eb, atime: 0x0, ctime: 0x6aad94eb, blocks: 0xc807 00010000:00010000:0.0:1789760747.952497:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2e962 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.952518:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=727s v=(0 0) 00010000:00010000:0.0:1789760747.952523:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760747.952532:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2e962 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25198592->25202687] (req 25198592->25202687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.952549:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2e962 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25198592->25202687] (req 25198592->25202687) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760747.995386:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973471 is committed 00002000:00010000:1.0:1789760747.995587:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25198592 -> 25202688 00010000:00010000:1.0:1789760747.995608:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2e962 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25198592->25202687] (req 25198592->25202687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760747.995624:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2e962 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25198592->25202687] (req 25198592->25202687) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760747.995635:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2e962 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25198592->25202687] (req 25198592->25202687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760747.995645:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760747.995651:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2e962 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25198592->25202687] (req 25198592->25202687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760747.995661:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2e962 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25198592->25202687] (req 25198592->25202687) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760747.995723:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2e962 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25198592->25202687] (req 25198592->25202687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760747.995739:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2e962 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25198592->25202687] (req 25198592->25202687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760747.995862:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febea4b5450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512648448:12345-192.168.203.56@tcp:4:dd.0 Request processed in 43440us (43935us total) trans 4294973471 rc 0/0 00000100:00100000:1.0:1789760747.995876:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6182 00000100:00100000:1.0:1789760747.995889:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 727452713166 time 727516700122 state 0x2 ref 0 00000100:00100000:0.0:1789760748.000414:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760748.000621:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512648576 00000100:00100000:2.0:1789760748.000640:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512648576 00000100:00100000:2.0:1789760748.000771:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 727452713166 time 727516700122 state 0x0 ref 1 00000100:00100000:2.0:1789760748.000790:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 727516960538 time 727516700122 state 0x1 ref 1 00000100:00100000:2.0:1789760748.000825:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 727516960538 time 727565124991 state 0x0 ref 1 00000100:00100000:2.0:1789760748.000842:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac3353800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512648576:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760748.000911:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25202688, mtime: 0x6aad94eb, atime: 0x0, ctime: 0x6aad94eb, blocks: 0xc807 00010000:00010000:2.0:1789760748.000922:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2e969 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.000945:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=727s v=(0 0) 00010000:00010000:2.0:1789760748.000950:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760748.000967:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2e969 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25202688->25206783] (req 25202688->25206783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.000983:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2e969 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25202688->25206783] (req 25202688->25206783) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760748.033914:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973472 is committed 00002000:00010000:0.0:1789760748.034116:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25202688 -> 25206784 00010000:00010000:0.0:1789760748.034137:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2e969 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25202688->25206783] (req 25202688->25206783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.034153:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2e969 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25202688->25206783] (req 25202688->25206783) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.034167:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2e969 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25202688->25206783] (req 25202688->25206783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.034177:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760748.034183:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2e969 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25202688->25206783] (req 25202688->25206783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.034193:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2e969 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25202688->25206783] (req 25202688->25206783) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.034219:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2e969 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25202688->25206783] (req 25202688->25206783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.034228:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2e969 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25202688->25206783] (req 25202688->25206783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760748.034394:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac3353800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512648576:12345-192.168.203.56@tcp:4:dd.0 Request processed in 33548us (33980us total) trans 4294973472 rc 0/0 00000100:00100000:0.0:1789760748.034407:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6183 00000100:00100000:0.0:1789760748.034419:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 727516960538 time 727565124991 state 0x2 ref 0 00000100:00100000:0.0:1789760748.039538:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760748.039663:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512648704 00000100:00100000:1.0:1789760748.039677:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512648704 00000100:00100000:1.0:1789760748.040043:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 727516960538 time 727565124991 state 0x0 ref 1 00000100:00100000:1.0:1789760748.040062:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 727565385407 time 727565124991 state 0x1 ref 1 00000100:00100000:1.0:1789760748.040143:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 727565385407 time 727604442011 state 0x0 ref 1 00000100:00100000:1.0:1789760748.040157:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac3352a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512648704:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760748.040220:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25206784, mtime: 0x6aad94eb, atime: 0x0, ctime: 0x6aad94eb, blocks: 0xc807 00010000:00010000:1.0:1789760748.040231:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2e970 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.040250:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=727s v=(0 0) 00010000:00010000:1.0:1789760748.040256:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760748.040263:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2e970 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25206784->25210879] (req 25206784->25210879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.040277:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2e970 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25206784->25210879] (req 25206784->25210879) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760748.100058:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973473 is committed 00002000:00010000:0.0:1789760748.100104:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25206784 -> 25210880 00010000:00010000:0.0:1789760748.100125:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2e970 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25206784->25210879] (req 25206784->25210879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.100145:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2e970 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25206784->25210879] (req 25206784->25210879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.100155:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2e970 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25206784->25210879] (req 25206784->25210879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.100162:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760748.100167:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2e970 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25206784->25210879] (req 25206784->25210879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.100176:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2e970 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25206784->25210879] (req 25206784->25210879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.100195:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2e970 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25206784->25210879] (req 25206784->25210879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.100204:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2e970 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25206784->25210879] (req 25206784->25210879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760748.100363:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac3352a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512648704:12345-192.168.203.56@tcp:4:dd.0 Request processed in 60203us (60835us total) trans 4294973473 rc 0/0 00000100:00100000:0.0:1789760748.100374:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6184 00000100:00100000:0.0:1789760748.100383:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 727565385407 time 727604442011 state 0x2 ref 0 00000100:00100000:3.0:1789760748.107081:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760748.107192:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512648832 00000100:00100000:1.0:1789760748.107210:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512648832 00000100:00100000:1.0:1789760748.107320:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 727565385407 time 727604442011 state 0x0 ref 1 00000100:00100000:1.0:1789760748.107339:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 727604702427 time 727604442011 state 0x1 ref 1 00000100:00100000:1.0:1789760748.107379:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 727604702427 time 727671678152 state 0x0 ref 1 00000100:00100000:1.0:1789760748.107397:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2877100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512648832:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760748.107466:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25210880, mtime: 0x6aad94eb, atime: 0x0, ctime: 0x6aad94eb, blocks: 0xc807 00010000:00010000:1.0:1789760748.107478:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2e977 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.107499:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=727s v=(0 0) 00010000:00010000:1.0:1789760748.107503:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760748.107512:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2e977 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25210880->25214975] (req 25210880->25214975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.107529:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2e977 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25210880->25214975] (req 25210880->25214975) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760748.146121:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973474 is committed 00002000:00010000:1.0:1789760748.146224:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25210880 -> 25214976 00010000:00010000:1.0:1789760748.146240:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2e977 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25210880->25214975] (req 25210880->25214975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.146253:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2e977 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25210880->25214975] (req 25210880->25214975) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.146262:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2e977 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25210880->25214975] (req 25210880->25214975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.146269:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760748.146274:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2e977 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25210880->25214975] (req 25210880->25214975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.146282:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2e977 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25210880->25214975] (req 25210880->25214975) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.146313:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2e977 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25210880->25214975] (req 25210880->25214975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.146321:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2e977 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25210880->25214975] (req 25210880->25214975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760748.146418:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2877100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512648832:12345-192.168.203.56@tcp:4:dd.0 Request processed in 39022us (39341us total) trans 4294973474 rc 0/0 00000100:00100000:1.0:1789760748.146429:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6185 00000100:00100000:1.0:1789760748.146438:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 727604702427 time 727671678152 state 0x2 ref 0 00000100:00100000:0.0:1789760748.150196:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760748.150269:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512648960 00000100:00100000:1.0:1789760748.150279:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512648960 00000100:00100000:1.0:1789760748.150340:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 727604702427 time 727671678152 state 0x0 ref 1 00000100:00100000:1.0:1789760748.150365:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 727671938568 time 727671678152 state 0x1 ref 1 00000100:00100000:1.0:1789760748.150395:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 727671938568 time 727714694798 state 0x0 ref 1 00000100:00100000:1.0:1789760748.150408:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac3353480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512648960:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760748.150458:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25214976, mtime: 0x6aad94eb, atime: 0x0, ctime: 0x6aad94eb, blocks: 0xc807 00010000:00010000:1.0:1789760748.150468:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2e97e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.150482:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=727s v=(0 0) 00010000:00010000:1.0:1789760748.150486:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760748.150492:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2e97e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25214976->25219071] (req 25214976->25219071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.150507:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2e97e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25214976->25219071] (req 25214976->25219071) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760748.187186:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973475 is committed 00002000:00010000:3.0:1789760748.187420:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25214976 -> 25219072 00010000:00010000:3.0:1789760748.187437:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2e97e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25214976->25219071] (req 25214976->25219071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.187449:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2e97e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25214976->25219071] (req 25214976->25219071) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.187464:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2e97e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25214976->25219071] (req 25214976->25219071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.187472:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760748.187477:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2e97e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25214976->25219071] (req 25214976->25219071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.187486:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2e97e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25214976->25219071] (req 25214976->25219071) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.187506:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2e97e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25214976->25219071] (req 25214976->25219071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.187513:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2e97e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25214976->25219071] (req 25214976->25219071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760748.187610:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac3353480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512648960:12345-192.168.203.56@tcp:4:dd.0 Request processed in 37200us (37418us total) trans 4294973475 rc 0/0 00000100:00100000:3.0:1789760748.187620:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6186 00000100:00100000:3.0:1789760748.187629:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 727671938568 time 727714694798 state 0x2 ref 0 00000100:00100000:0.0:1789760748.191912:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760748.191995:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512649088 00000100:00100000:1.0:1789760748.192006:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512649088 00000100:00100000:1.0:1789760748.192059:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 727671938568 time 727714694798 state 0x0 ref 1 00000100:00100000:1.0:1789760748.192075:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 727714955214 time 727714694798 state 0x1 ref 1 00000100:00100000:1.0:1789760748.192105:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 727714955214 time 727756404427 state 0x0 ref 1 00000100:00100000:1.0:1789760748.192117:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac3352300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512649088:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760748.192166:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25219072, mtime: 0x6aad94eb, atime: 0x0, ctime: 0x6aad94eb, blocks: 0xc807 00010000:00010000:1.0:1789760748.192175:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2e985 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.192192:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=727s v=(0 0) 00010000:00010000:1.0:1789760748.192197:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760748.192204:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2e985 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25219072->25223167] (req 25219072->25223167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.192219:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2e985 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25219072->25223167] (req 25219072->25223167) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760748.223176:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973476 is committed 00002000:00010000:0.0:1789760748.223473:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25219072 -> 25223168 00010000:00010000:0.0:1789760748.223488:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2e985 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25219072->25223167] (req 25219072->25223167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.223502:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2e985 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25219072->25223167] (req 25219072->25223167) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.223511:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2e985 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25219072->25223167] (req 25219072->25223167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.223519:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760748.223524:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2e985 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25219072->25223167] (req 25219072->25223167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.223532:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2e985 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25219072->25223167] (req 25219072->25223167) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.223552:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2e985 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25219072->25223167] (req 25219072->25223167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.223560:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2e985 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25219072->25223167] (req 25219072->25223167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760748.223687:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac3352300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512649088:12345-192.168.203.56@tcp:4:dd.0 Request processed in 31566us (31779us total) trans 4294973476 rc 0/0 00000100:00100000:0.0:1789760748.223740:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6187 00000100:00100000:0.0:1789760748.223750:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 727714955214 time 727756404427 state 0x2 ref 0 00000100:00100000:2.0:1789760748.228254:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760748.228430:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512649216 00000100:00100000:1.0:1789760748.228442:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512649216 00000100:00100000:1.0:1789760748.228498:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 727714955214 time 727756404427 state 0x0 ref 1 00000100:00100000:1.0:1789760748.228512:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 727756664843 time 727756404427 state 0x1 ref 1 00000100:00100000:1.0:1789760748.228543:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 727756664843 time 727792842865 state 0x0 ref 1 00000100:00100000:1.0:1789760748.228554:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febfc0a3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512649216:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760748.228604:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25223168, mtime: 0x6aad94eb, atime: 0x0, ctime: 0x6aad94eb, blocks: 0xc807 00010000:00010000:1.0:1789760748.228612:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2e98c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.228626:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=727s v=(0 0) 00010000:00010000:1.0:1789760748.228630:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760748.228636:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2e98c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25223168->25227263] (req 25223168->25227263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.228650:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2e98c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25223168->25227263] (req 25223168->25227263) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760748.276795:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973477 is committed 00002000:00010000:2.0:1789760748.277076:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25223168 -> 25227264 00010000:00010000:2.0:1789760748.277093:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2e98c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25223168->25227263] (req 25223168->25227263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.277105:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2e98c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25223168->25227263] (req 25223168->25227263) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.277115:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2e98c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25223168->25227263] (req 25223168->25227263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.277123:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760748.277129:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2e98c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25223168->25227263] (req 25223168->25227263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.277138:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2e98c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25223168->25227263] (req 25223168->25227263) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.277157:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2e98c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25223168->25227263] (req 25223168->25227263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.277164:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2e98c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25223168->25227263] (req 25223168->25227263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760748.277363:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febfc0a3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512649216:12345-192.168.203.56@tcp:4:dd.0 Request processed in 48805us (49113us total) trans 4294973477 rc 0/0 00000100:00100000:2.0:1789760748.277373:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6188 00000100:00100000:2.0:1789760748.277383:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 727756664843 time 727792842865 state 0x2 ref 0 00000100:00100000:2.0:1789760748.280667:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760748.280765:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512649344 00000100:00100000:3.0:1789760748.280774:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512649344 00000100:00100000:3.0:1789760748.280828:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 727756664843 time 727792842865 state 0x0 ref 1 00000100:00100000:3.0:1789760748.280854:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 727793103281 time 727792842865 state 0x1 ref 1 00000100:00100000:3.0:1789760748.280885:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 727793103281 time 727845184440 state 0x0 ref 1 00000100:00100000:3.0:1789760748.280898:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febfc0a3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512649344:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760748.280942:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25227264, mtime: 0x6aad94eb, atime: 0x0, ctime: 0x6aad94eb, blocks: 0xc807 00010000:00010000:3.0:1789760748.280950:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2e993 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.280976:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=727s v=(0 0) 00010000:00010000:3.0:1789760748.280981:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760748.280988:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2e993 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25227264->25231359] (req 25227264->25231359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.281002:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2e993 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25227264->25231359] (req 25227264->25231359) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760748.312401:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25227264 -> 25231360 00010000:00010000:1.0:1789760748.312422:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2e993 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25227264->25231359] (req 25227264->25231359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.312436:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2e993 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25227264->25231359] (req 25227264->25231359) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.312446:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2e993 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25227264->25231359] (req 25227264->25231359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.312453:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760748.312458:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2e993 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25227264->25231359] (req 25227264->25231359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.312465:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2e993 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25227264->25231359] (req 25227264->25231359) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.312483:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2e993 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25227264->25231359] (req 25227264->25231359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.312489:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2e993 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25227264->25231359] (req 25227264->25231359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760748.312585:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febfc0a3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2360:x1876699512649344:12345-192.168.203.56@tcp:4:dd.0 Request processed in 31684us (31922us total) trans 4294973478 rc 0/0 00000100:00100000:1.0:1789760748.312595:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6189 00000100:00100000:1.0:1789760748.312604:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 727793103281 time 727845184440 state 0x2 ref 0 00000001:00080000:2.0:1789760748.312692:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973478 is committed 00000100:00100000:0.0:1789760748.320523:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760748.320601:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512649472 00000100:00100000:1.0:1789760748.320612:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512649472 00000100:00100000:1.0:1789760748.320665:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 727793103281 time 727845184440 state 0x0 ref 1 00000100:00100000:1.0:1789760748.320691:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 727845444856 time 727845184440 state 0x1 ref 1 00000100:00100000:1.0:1789760748.320762:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 727845444856 time 727885060638 state 0x0 ref 1 00000100:00100000:1.0:1789760748.320776:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febfb894700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512649472:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760748.320822:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25231360, mtime: 0x6aad94eb, atime: 0x0, ctime: 0x6aad94eb, blocks: 0xc807 00010000:00010000:1.0:1789760748.320831:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2e99a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.320856:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=727s v=(0 0) 00010000:00010000:1.0:1789760748.320860:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760748.320867:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2e99a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25231360->25235455] (req 25231360->25235455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.320881:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2e99a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25231360->25235455] (req 25231360->25235455) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760748.359109:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973479 is committed 00002000:00010000:0.0:1789760748.359162:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25231360 -> 25235456 00010000:00010000:0.0:1789760748.359177:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2e99a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25231360->25235455] (req 25231360->25235455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.359190:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2e99a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25231360->25235455] (req 25231360->25235455) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.359199:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2e99a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25231360->25235455] (req 25231360->25235455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.359204:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760748.359208:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2e99a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25231360->25235455] (req 25231360->25235455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.359214:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2e99a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25231360->25235455] (req 25231360->25235455) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.359227:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2e99a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25231360->25235455] (req 25231360->25235455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.359231:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2e99a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25231360->25235455] (req 25231360->25235455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760748.359318:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febfb894700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512649472:12345-192.168.203.56@tcp:4:dd.0 Request processed in 38540us (38800us total) trans 4294973479 rc 0/0 00000100:00100000:0.0:1789760748.359329:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6190 00000100:00100000:0.0:1789760748.359338:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 727845444856 time 727885060638 state 0x2 ref 0 00000100:00100000:3.0:1789760748.361753:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760748.363482:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512649600 00000100:00100000:2.0:1789760748.363495:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512649600 00000100:00100000:2.0:1789760748.363569:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 727845444856 time 727885060638 state 0x0 ref 1 00000100:00100000:2.0:1789760748.363585:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 727885321054 time 727885060638 state 0x1 ref 1 00000100:00100000:2.0:1789760748.363664:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 727885321054 time 727927962793 state 0x0 ref 1 00000100:00100000:2.0:1789760748.363684:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2877b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512649600:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760748.363743:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25235456, mtime: 0x6aad94eb, atime: 0x0, ctime: 0x6aad94eb, blocks: 0xc807 00010000:00010000:2.0:1789760748.363753:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2e9a1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.363768:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=727s v=(0 0) 00010000:00010000:2.0:1789760748.363784:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760748.363791:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2e9a1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25235456->25239551] (req 25235456->25239551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.363812:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2e9a1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25235456->25239551] (req 25235456->25239551) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760748.403911:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973480 is committed 00002000:00010000:0.0:1789760748.404020:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25235456 -> 25239552 00010000:00010000:0.0:1789760748.404036:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2e9a1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25235456->25239551] (req 25235456->25239551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.404049:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2e9a1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25235456->25239551] (req 25235456->25239551) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.404058:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2e9a1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25235456->25239551] (req 25235456->25239551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.404065:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760748.404070:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2e9a1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25235456->25239551] (req 25235456->25239551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.404080:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2e9a1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25235456->25239551] (req 25235456->25239551) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.404099:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2e9a1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25235456->25239551] (req 25235456->25239551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.404107:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2e9a1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25235456->25239551] (req 25235456->25239551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760748.404205:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2877b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512649600:12345-192.168.203.56@tcp:4:dd.0 Request processed in 40520us (42456us total) trans 4294973480 rc 0/0 00000100:00100000:0.0:1789760748.404215:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6191 00000100:00100000:0.0:1789760748.404224:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 727885321054 time 727927962793 state 0x2 ref 0 00000100:00100000:2.0:1789760748.410194:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760748.410274:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512649728 00000100:00100000:1.0:1789760748.410292:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512649728 00000100:00100000:1.0:1789760748.410347:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 727885321054 time 727927962793 state 0x0 ref 1 00000100:00100000:1.0:1789760748.410362:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 727928223209 time 727927962793 state 0x1 ref 1 00000100:00100000:1.0:1789760748.410444:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 727928223209 time 727974743276 state 0x0 ref 1 00000100:00100000:1.0:1789760748.410458:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac437f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512649728:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760748.410507:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25239552, mtime: 0x6aad94eb, atime: 0x0, ctime: 0x6aad94eb, blocks: 0xc807 00010000:00010000:1.0:1789760748.410516:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2e9a8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.410531:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=727s v=(0 0) 00010000:00010000:1.0:1789760748.410535:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760748.410540:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2e9a8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25239552->25243647] (req 25239552->25243647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.410554:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2e9a8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25239552->25243647] (req 25239552->25243647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760748.469214:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973481 is committed 00002000:00010000:3.0:1789760748.469479:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25239552 -> 25243648 00010000:00010000:3.0:1789760748.469504:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2e9a8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25239552->25243647] (req 25239552->25243647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.469521:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2e9a8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25239552->25243647] (req 25239552->25243647) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.469540:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2e9a8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25239552->25243647] (req 25239552->25243647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.469548:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760748.469554:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2e9a8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25239552->25243647] (req 25239552->25243647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.469566:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2e9a8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25239552->25243647] (req 25239552->25243647) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.469667:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2e9a8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25239552->25243647] (req 25239552->25243647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.469677:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2e9a8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25239552->25243647] (req 25239552->25243647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760748.469841:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac437f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512649728:12345-192.168.203.56@tcp:4:dd.0 Request processed in 59378us (59647us total) trans 4294973481 rc 0/0 00000100:00100000:3.0:1789760748.469853:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6192 00000100:00100000:3.0:1789760748.469865:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 727928223209 time 727974743276 state 0x2 ref 0 00000100:00100000:3.0:1789760748.473988:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760748.474096:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512649856 00000100:00100000:0.0:1789760748.474110:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512649856 00000100:00100000:0.0:1789760748.474184:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 727928223209 time 727974743276 state 0x0 ref 1 00000100:00100000:0.0:1789760748.474204:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 727975003692 time 727974743276 state 0x1 ref 1 00000100:00100000:0.0:1789760748.474256:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 727975003692 time 728038546603 state 0x0 ref 1 00000100:00100000:0.0:1789760748.474271:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2874700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512649856:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760748.474346:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25243648, mtime: 0x6aad94eb, atime: 0x0, ctime: 0x6aad94eb, blocks: 0xc807 00010000:00010000:0.0:1789760748.474357:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2e9af lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.474378:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=728s v=(0 0) 00010000:00010000:0.0:1789760748.474383:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760748.474391:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2e9af lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25243648->25247743] (req 25243648->25247743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.474408:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2e9af lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25243648->25247743] (req 25243648->25247743) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760748.525246:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973482 is committed 00002000:00010000:2.0:1789760748.525396:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25243648 -> 25247744 00010000:00010000:2.0:1789760748.525416:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2e9af lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25243648->25247743] (req 25243648->25247743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.525429:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2e9af lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25243648->25247743] (req 25243648->25247743) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.525439:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2e9af lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25243648->25247743] (req 25243648->25247743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.525452:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760748.525458:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2e9af lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25243648->25247743] (req 25243648->25247743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.525467:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2e9af lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25243648->25247743] (req 25243648->25247743) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.525490:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2e9af lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25243648->25247743] (req 25243648->25247743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.525499:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2e9af lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25243648->25247743] (req 25243648->25247743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760748.525635:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2874700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512649856:12345-192.168.203.56@tcp:4:dd.0 Request processed in 51360us (51656us total) trans 4294973482 rc 0/0 00000100:00100000:2.0:1789760748.525649:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6193 00000100:00100000:2.0:1789760748.525662:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 727975003692 time 728038546603 state 0x2 ref 0 00000100:00100000:2.0:1789760748.531292:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760748.531441:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512649984 00000100:00100000:0.0:1789760748.531451:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512649984 00000100:00100000:0.0:1789760748.531507:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 727975003692 time 728038546603 state 0x0 ref 1 00000100:00100000:0.0:1789760748.531521:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 728038807019 time 728038546603 state 0x1 ref 1 00000100:00100000:0.0:1789760748.531591:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 728038807019 time 728095889242 state 0x0 ref 1 00000100:00100000:0.0:1789760748.531606:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac437fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512649984:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760748.531655:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25247744, mtime: 0x6aad94eb, atime: 0x0, ctime: 0x6aad94eb, blocks: 0xc807 00010000:00010000:0.0:1789760748.531664:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2e9b6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.531680:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=728s v=(0 0) 00010000:00010000:0.0:1789760748.531685:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760748.531691:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2e9b6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25247744->25251839] (req 25247744->25251839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.531737:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2e9b6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25247744->25251839] (req 25247744->25251839) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760748.584037:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973483 is committed 00002000:00010000:3.0:1789760748.584135:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25247744 -> 25251840 00002000:00010000:3.0:1789760748.584140:0:13062:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760747 -> 1789760748 00002000:00010000:3.0:1789760748.584143:0:13062:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760747 -> 1789760748 00010000:00010000:3.0:1789760748.584156:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2e9b6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25247744->25251839] (req 25247744->25251839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.584170:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2e9b6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25247744->25251839] (req 25247744->25251839) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.584180:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2e9b6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25247744->25251839] (req 25247744->25251839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.584188:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760748.584194:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2e9b6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25247744->25251839] (req 25247744->25251839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.584203:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2e9b6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25247744->25251839] (req 25247744->25251839) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.584222:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2e9b6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25247744->25251839] (req 25247744->25251839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.584230:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2e9b6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25247744->25251839] (req 25247744->25251839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760748.584362:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac437fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512649984:12345-192.168.203.56@tcp:4:dd.0 Request processed in 52753us (53083us total) trans 4294973483 rc 0/0 00000100:00100000:3.0:1789760748.584373:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6194 00000100:00100000:3.0:1789760748.584383:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 728038807019 time 728095889242 state 0x2 ref 0 00000100:00100000:0.0:1789760748.587464:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760748.587566:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512650112 00000100:00100000:3.0:1789760748.587576:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512650112 00000100:00100000:3.0:1789760748.587637:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 728038807019 time 728095889242 state 0x0 ref 1 00000100:00100000:3.0:1789760748.587653:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 728096149658 time 728095889242 state 0x1 ref 1 00000100:00100000:3.0:1789760748.587682:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 728096149658 time 728151981907 state 0x0 ref 1 00000100:00100000:3.0:1789760748.587721:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febfb897100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512650112:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760748.587778:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25251840, mtime: 0x6aad94ec, atime: 0x0, ctime: 0x6aad94ec, blocks: 0xc807 00010000:00010000:3.0:1789760748.587788:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2e9bd lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.587804:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=728s v=(0 0) 00010000:00010000:3.0:1789760748.587808:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760748.587814:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2e9bd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25251840->25255935] (req 25251840->25255935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.587828:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2e9bd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25251840->25255935] (req 25251840->25255935) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760748.614478:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25251840 -> 25255936 00010000:00010000:3.0:1789760748.614497:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2e9bd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25251840->25255935] (req 25251840->25255935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.614511:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2e9bd lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25251840->25255935] (req 25251840->25255935) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.614520:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2e9bd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25251840->25255935] (req 25251840->25255935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.614528:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760748.614532:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2e9bd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25251840->25255935] (req 25251840->25255935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.614542:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2e9bd lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25251840->25255935] (req 25251840->25255935) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.614560:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2e9bd lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25251840->25255935] (req 25251840->25255935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.614567:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2e9bd lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25251840->25255935] (req 25251840->25255935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760748.614653:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febfb897100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2358:x1876699512650112:12345-192.168.203.56@tcp:4:dd.0 Request processed in 26956us (27339us total) trans 4294973484 rc 0/0 00000100:00100000:3.0:1789760748.614663:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6195 00000100:00100000:3.0:1789760748.615156:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 728096149658 time 728151981907 state 0x2 ref 0 00000001:00080000:0.0:1789760748.615277:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973484 is committed 00000100:00100000:0.0:1789760748.618028:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760748.620819:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512650240 00000100:00100000:2.0:1789760748.620835:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512650240 00000100:00100000:2.0:1789760748.620898:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 728096149658 time 728151981907 state 0x0 ref 1 00000100:00100000:2.0:1789760748.620917:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 728152242323 time 728151981907 state 0x1 ref 1 00000100:00100000:2.0:1789760748.620953:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 728152242323 time 728185252545 state 0x0 ref 1 00000100:00100000:2.0:1789760748.620968:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febfb895880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512650240:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760748.621015:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25255936, mtime: 0x6aad94ec, atime: 0x0, ctime: 0x6aad94ec, blocks: 0xc807 00010000:00010000:2.0:1789760748.621024:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2e9c4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.621039:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=728s v=(0 0) 00010000:00010000:2.0:1789760748.621043:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760748.621049:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2e9c4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25255936->25260031] (req 25255936->25260031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.621062:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2e9c4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25255936->25260031] (req 25255936->25260031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760748.648936:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973485 is committed 00002000:00010000:0.0:1789760748.649068:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25255936 -> 25260032 00010000:00010000:0.0:1789760748.649084:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2e9c4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25255936->25260031] (req 25255936->25260031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.649097:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2e9c4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25255936->25260031] (req 25255936->25260031) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.649106:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2e9c4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25255936->25260031] (req 25255936->25260031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.649114:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760748.649119:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2e9c4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25255936->25260031] (req 25255936->25260031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.649127:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2e9c4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25255936->25260031] (req 25255936->25260031) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.649147:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2e9c4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25255936->25260031] (req 25255936->25260031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.649155:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2e9c4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25255936->25260031] (req 25255936->25260031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760748.649278:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febfb895880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512650240:12345-192.168.203.56@tcp:4:dd.0 Request processed in 28308us (31253us total) trans 4294973485 rc 0/0 00000100:00100000:0.0:1789760748.649298:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6196 00000100:00100000:0.0:1789760748.649308:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 728152242323 time 728185252545 state 0x2 ref 0 00000100:00100000:2.0:1789760748.652072:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760748.652170:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512650368 00000100:00100000:3.0:1789760748.652179:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512650368 00000100:00100000:3.0:1789760748.652233:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 728152242323 time 728185252545 state 0x0 ref 1 00000100:00100000:3.0:1789760748.652253:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 728185512961 time 728185252545 state 0x1 ref 1 00000100:00100000:1.0:1789760748.652342:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 728185512961 time 728216639112 state 0x0 ref 1 00000100:00100000:1.0:1789760748.652356:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac437d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512650368:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760748.652400:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25260032, mtime: 0x6aad94ec, atime: 0x0, ctime: 0x6aad94ec, blocks: 0xc807 00010000:00010000:1.0:1789760748.652408:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2e9cb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.652423:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=728s v=(0 0) 00010000:00010000:1.0:1789760748.652428:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760748.652433:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2e9cb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25260032->25264127] (req 25260032->25264127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.652444:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2e9cb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25260032->25264127] (req 25260032->25264127) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760748.690485:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973486 is committed 00002000:00010000:2.0:1789760748.690588:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25260032 -> 25264128 00010000:00010000:2.0:1789760748.690604:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2e9cb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25260032->25264127] (req 25260032->25264127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.690617:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2e9cb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25260032->25264127] (req 25260032->25264127) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.690625:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2e9cb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25260032->25264127] (req 25260032->25264127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.690633:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760748.690637:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2e9cb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25260032->25264127] (req 25260032->25264127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.690647:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2e9cb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25260032->25264127] (req 25260032->25264127) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.690667:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2e9cb lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25260032->25264127] (req 25260032->25264127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.690685:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2e9cb lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25260032->25264127] (req 25260032->25264127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760748.690792:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac437d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512650368:12345-192.168.203.56@tcp:4:dd.0 Request processed in 38434us (38725us total) trans 4294973486 rc 0/0 00000100:00100000:2.0:1789760748.690802:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6197 00000100:00100000:2.0:1789760748.690824:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 728185512961 time 728216639112 state 0x2 ref 0 00000100:00100000:2.0:1789760748.694170:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760748.694274:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512650496 00000100:00100000:3.0:1789760748.694294:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512650496 00000100:00100000:3.0:1789760748.694349:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 728185512961 time 728216639112 state 0x0 ref 1 00000100:00100000:3.0:1789760748.694363:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 728216899528 time 728216639112 state 0x1 ref 1 00000100:00100000:3.0:1789760748.694390:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 728216899528 time 728258689916 state 0x0 ref 1 00000100:00100000:3.0:1789760748.694403:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac437df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512650496:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760748.694447:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25264128, mtime: 0x6aad94ec, atime: 0x0, ctime: 0x6aad94ec, blocks: 0xc807 00010000:00010000:3.0:1789760748.694456:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2e9d2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.694472:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=728s v=(0 0) 00010000:00010000:3.0:1789760748.694477:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760748.694482:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2e9d2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25264128->25268223] (req 25264128->25268223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.694495:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2e9d2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25264128->25268223] (req 25264128->25268223) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760748.741294:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25264128 -> 25268224 00010000:00010000:3.0:1789760748.741315:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2e9d2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25264128->25268223] (req 25264128->25268223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.741330:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2e9d2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25264128->25268223] (req 25264128->25268223) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.741340:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2e9d2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25264128->25268223] (req 25264128->25268223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.741347:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760748.741352:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2e9d2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25264128->25268223] (req 25264128->25268223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.741361:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2e9d2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25264128->25268223] (req 25264128->25268223) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.741382:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2e9d2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25264128->25268223] (req 25264128->25268223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.741390:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2e9d2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25264128->25268223] (req 25264128->25268223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760748.741496:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac437df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2361:x1876699512650496:12345-192.168.203.56@tcp:4:dd.0 Request processed in 47091us (47328us total) trans 4294973487 rc 0/0 00000100:00100000:3.0:1789760748.741507:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6198 00000100:00100000:3.0:1789760748.741516:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 728216899528 time 728258689916 state 0x2 ref 0 00000001:00080000:1.0:1789760748.741650:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973487 is committed 00000100:00100000:0.0:1789760748.746854:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760748.746947:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512650624 00000100:00100000:2.0:1789760748.746956:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512650624 00000100:00100000:2.0:1789760748.747011:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 728216899528 time 728258689916 state 0x0 ref 1 00000100:00100000:2.0:1789760748.747028:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 728258950332 time 728258689916 state 0x1 ref 1 00000100:00100000:2.0:1789760748.747076:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 728258950332 time 728311367896 state 0x0 ref 1 00000100:00100000:2.0:1789760748.747089:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febe7907100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512650624:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760748.747146:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25268224, mtime: 0x6aad94ec, atime: 0x0, ctime: 0x6aad94ec, blocks: 0xc807 00010000:00010000:2.0:1789760748.747156:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2e9d9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.747171:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=728s v=(0 0) 00010000:00010000:2.0:1789760748.747176:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760748.747182:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2e9d9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25268224->25272319] (req 25268224->25272319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.747196:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2e9d9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25268224->25272319] (req 25268224->25272319) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760748.784973:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973488 is committed 00002000:00010000:3.0:1789760748.785411:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25268224 -> 25272320 00010000:00010000:3.0:1789760748.785428:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2e9d9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25268224->25272319] (req 25268224->25272319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.785443:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2e9d9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25268224->25272319] (req 25268224->25272319) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.785451:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2e9d9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25268224->25272319] (req 25268224->25272319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.785458:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760748.785463:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2e9d9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25268224->25272319] (req 25268224->25272319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.785471:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2e9d9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25268224->25272319] (req 25268224->25272319) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.785493:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2e9d9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25268224->25272319] (req 25268224->25272319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.785501:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2e9d9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25268224->25272319] (req 25268224->25272319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760748.785631:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febe7907100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512650624:12345-192.168.203.56@tcp:4:dd.0 Request processed in 38540us (38782us total) trans 4294973488 rc 0/0 00000100:00100000:3.0:1789760748.785641:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6199 00000100:00100000:3.0:1789760748.785652:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 728258950332 time 728311367896 state 0x2 ref 0 00000100:00100000:2.0:1789760748.788534:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760748.789183:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512650752 00000100:00100000:1.0:1789760748.789194:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512650752 00000100:00100000:0.0:1789760748.789339:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 728258950332 time 728311367896 state 0x0 ref 1 00000100:00100000:0.0:1789760748.789354:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 728311628312 time 728311367896 state 0x1 ref 1 00000100:00100000:0.0:1789760748.789418:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 728311628312 time 728353717573 state 0x0 ref 1 00000100:00100000:0.0:1789760748.789432:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac437e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512650752:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760748.789478:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25272320, mtime: 0x6aad94ec, atime: 0x0, ctime: 0x6aad94ec, blocks: 0xc807 00010000:00010000:0.0:1789760748.789486:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2e9e0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.789502:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=728s v=(0 0) 00010000:00010000:0.0:1789760748.789506:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760748.789512:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2e9e0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25272320->25276415] (req 25272320->25276415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.789526:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2e9e0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25272320->25276415] (req 25272320->25276415) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760748.832300:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973489 is committed 00002000:00010000:3.0:1789760748.832460:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25272320 -> 25276416 00010000:00010000:3.0:1789760748.832474:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2e9e0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25272320->25276415] (req 25272320->25276415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.832486:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2e9e0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25272320->25276415] (req 25272320->25276415) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.832506:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2e9e0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25272320->25276415] (req 25272320->25276415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.832514:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760748.832519:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2e9e0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25272320->25276415] (req 25272320->25276415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.832528:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2e9e0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25272320->25276415] (req 25272320->25276415) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.832554:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2e9e0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25272320->25276415] (req 25272320->25276415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.832562:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2e9e0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25272320->25276415] (req 25272320->25276415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760748.832647:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac437e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512650752:12345-192.168.203.56@tcp:4:dd.0 Request processed in 43213us (44117us total) trans 4294973489 rc 0/0 00000100:00100000:3.0:1789760748.832657:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6200 00000100:00100000:3.0:1789760748.832667:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 728311628312 time 728353717573 state 0x2 ref 0 00000100:00100000:2.0:1789760748.836664:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760748.836921:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512650880 00000100:00100000:3.0:1789760748.836930:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512650880 00000100:00100000:3.0:1789760748.837209:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 728311628312 time 728353717573 state 0x0 ref 1 00000100:00100000:3.0:1789760748.837226:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 728353977989 time 728353717573 state 0x1 ref 1 00000100:00100000:3.0:1789760748.837303:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 728353977989 time 728401601913 state 0x0 ref 1 00000100:00100000:3.0:1789760748.837317:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac28a2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512650880:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760748.837364:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25276416, mtime: 0x6aad94ec, atime: 0x0, ctime: 0x6aad94ec, blocks: 0xc807 00010000:00010000:3.0:1789760748.837373:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2e9e7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.837389:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=728s v=(0 0) 00010000:00010000:3.0:1789760748.837394:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760748.837400:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2e9e7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25276416->25280511] (req 25276416->25280511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.837413:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2e9e7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25276416->25280511] (req 25276416->25280511) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760748.870752:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973490 is committed 00002000:00010000:3.0:1789760748.870908:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25276416 -> 25280512 00010000:00010000:3.0:1789760748.870923:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2e9e7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25276416->25280511] (req 25276416->25280511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.870944:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2e9e7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25276416->25280511] (req 25276416->25280511) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.870954:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2e9e7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25276416->25280511] (req 25276416->25280511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.870961:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760748.870966:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2e9e7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25276416->25280511] (req 25276416->25280511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.870976:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2e9e7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25276416->25280511] (req 25276416->25280511) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.870994:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2e9e7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25276416->25280511] (req 25276416->25280511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.871002:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2e9e7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25276416->25280511] (req 25276416->25280511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760748.871095:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac28a2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512650880:12345-192.168.203.56@tcp:4:dd.0 Request processed in 33777us (34438us total) trans 4294973490 rc 0/0 00000100:00100000:3.0:1789760748.871104:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6201 00000100:00100000:3.0:1789760748.871115:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 728353977989 time 728401601913 state 0x2 ref 0 00000100:00100000:3.0:1789760748.873776:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760748.873969:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512651008 00000100:00100000:0.0:1789760748.873978:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512651008 00000100:00100000:0.0:1789760748.874030:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 728353977989 time 728401601913 state 0x0 ref 1 00000100:00100000:0.0:1789760748.874047:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 728401862329 time 728401601913 state 0x1 ref 1 00000100:00100000:0.0:1789760748.874086:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 728401862329 time 728438375786 state 0x0 ref 1 00000100:00100000:0.0:1789760748.874099:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2876a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512651008:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760748.874146:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25280512, mtime: 0x6aad94ec, atime: 0x0, ctime: 0x6aad94ec, blocks: 0xc807 00010000:00010000:0.0:1789760748.874155:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2e9ee lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.874170:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=728s v=(0 0) 00010000:00010000:0.0:1789760748.874174:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760748.874181:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2e9ee lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25280512->25284607] (req 25280512->25284607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.874195:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2e9ee lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25280512->25284607] (req 25280512->25284607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760748.903354:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25280512 -> 25284608 00010000:00010000:1.0:1789760748.903374:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2e9ee lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25280512->25284607] (req 25280512->25284607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.903388:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2e9ee lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25280512->25284607] (req 25280512->25284607) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.903397:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2e9ee lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25280512->25284607] (req 25280512->25284607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.903405:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760748.903409:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2e9ee lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25280512->25284607] (req 25280512->25284607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.903419:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2e9ee lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25280512->25284607] (req 25280512->25284607) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.903439:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2e9ee lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25280512->25284607] (req 25280512->25284607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.903451:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2e9ee lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25280512->25284607] (req 25280512->25284607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760748.903546:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2876a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2360:x1876699512651008:12345-192.168.203.56@tcp:4:dd.0 Request processed in 29444us (29773us total) trans 4294973491 rc 0/0 00000100:00100000:1.0:1789760748.903556:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6202 00000100:00100000:1.0:1789760748.903566:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 728401862329 time 728438375786 state 0x2 ref 0 00000001:00080000:2.0:1789760748.906067:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973491 is committed 00000100:00100000:0.0:1789760748.906861:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760748.906946:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512651136 00000100:00100000:3.0:1789760748.907015:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512651136 00000100:00100000:3.0:1789760748.907073:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 728401862329 time 728438375786 state 0x0 ref 1 00000100:00100000:3.0:1789760748.908164:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 728438636202 time 728438375786 state 0x1 ref 1 00000100:00100000:3.0:1789760748.908199:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 728438636202 time 728472498655 state 0x0 ref 1 00000100:00100000:3.0:1789760748.908217:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2747b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512651136:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760748.908268:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25284608, mtime: 0x6aad94ec, atime: 0x0, ctime: 0x6aad94ec, blocks: 0xc807 00010000:00010000:3.0:1789760748.908277:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2e9f5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.908305:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=728s v=(0 0) 00010000:00010000:3.0:1789760748.908310:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760748.908317:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2e9f5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25284608->25288703] (req 25284608->25288703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.908332:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2e9f5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25284608->25288703] (req 25284608->25288703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.916265:0:5835:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:1.0:1789760748.952970:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973492 is committed 00002000:00010000:0.0:1789760748.953218:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25284608 -> 25288704 00010000:00010000:0.0:1789760748.953233:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2e9f5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25284608->25288703] (req 25284608->25288703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.953246:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2e9f5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25284608->25288703] (req 25284608->25288703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.953255:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2e9f5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25284608->25288703] (req 25284608->25288703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.953264:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760748.953269:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2e9f5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25284608->25288703] (req 25284608->25288703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.953277:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2e9f5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25284608->25288703] (req 25284608->25288703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.953309:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2e9f5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25284608->25288703] (req 25284608->25288703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.953317:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2e9f5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25284608->25288703] (req 25284608->25288703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760748.953440:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2747b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512651136:12345-192.168.203.56@tcp:4:dd.0 Request processed in 45226us (46580us total) trans 4294973492 rc 0/0 00000100:00100000:0.0:1789760748.953450:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6203 00000100:00100000:0.0:1789760748.953459:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 728438636202 time 728472498655 state 0x2 ref 0 00000100:00100000:0.0:1789760748.955917:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760748.956056:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512651264 00000100:00100000:3.0:1789760748.956065:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512651264 00000100:00100000:3.0:1789760748.956116:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 728438636202 time 728472498655 state 0x0 ref 1 00000100:00100000:3.0:1789760748.956130:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 728472759071 time 728472498655 state 0x1 ref 1 00000100:00100000:3.0:1789760748.956160:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 728472759071 time 728520460750 state 0x0 ref 1 00000100:00100000:3.0:1789760748.956172:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2747800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512651264:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760748.956217:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25288704, mtime: 0x6aad94ec, atime: 0x0, ctime: 0x6aad94ec, blocks: 0xc807 00010000:00010000:3.0:1789760748.956228:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2e9fc lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.956242:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=728s v=(0 0) 00010000:00010000:3.0:1789760748.956246:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760748.956252:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2e9fc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25288704->25292799] (req 25288704->25292799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.956266:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2e9fc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25288704->25292799] (req 25288704->25292799) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760749.040625:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973493 is committed 00002000:00010000:1.0:1789760749.040898:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25288704 -> 25292800 00010000:00010000:1.0:1789760749.040916:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2e9fc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25288704->25292799] (req 25288704->25292799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.040930:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2e9fc lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25288704->25292799] (req 25288704->25292799) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.040940:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2e9fc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25288704->25292799] (req 25288704->25292799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.040948:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760749.041021:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2e9fc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25288704->25292799] (req 25288704->25292799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.041031:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2e9fc lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25288704->25292799] (req 25288704->25292799) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.041055:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2e9fc lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25288704->25292799] (req 25288704->25292799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.041063:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2e9fc lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25288704->25292799] (req 25288704->25292799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760749.041156:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2747800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512651264:12345-192.168.203.56@tcp:4:dd.0 Request processed in 84979us (85241us total) trans 4294973493 rc 0/0 00000100:00100000:1.0:1789760749.041166:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6204 00000100:00100000:1.0:1789760749.041176:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 728472759071 time 728520460750 state 0x2 ref 0 00000100:00100000:2.0:1789760749.049189:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760749.049274:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512651392 00000100:00100000:1.0:1789760749.049294:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512651392 00000100:00100000:1.0:1789760749.049359:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 728472759071 time 728520460750 state 0x0 ref 1 00000100:00100000:1.0:1789760749.049373:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 728520721166 time 728520460750 state 0x1 ref 1 00000100:00100000:1.0:1789760749.049402:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 728520721166 time 728613702472 state 0x0 ref 1 00000100:00100000:1.0:1789760749.049414:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac28a1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512651392:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760749.049472:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25292800, mtime: 0x6aad94ec, atime: 0x0, ctime: 0x6aad94ec, blocks: 0xc807 00010000:00010000:1.0:1789760749.049480:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2ea03 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.049494:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=728s v=(0 0) 00010000:00010000:1.0:1789760749.049499:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760749.049506:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2ea03 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25292800->25296895] (req 25292800->25296895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.049520:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2ea03 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25292800->25296895] (req 25292800->25296895) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1789760749.049563:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:0.0:1789760749.093447:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973494 is committed 00002000:00010000:1.0:1789760749.093636:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25292800 -> 25296896 00010000:00010000:1.0:1789760749.093651:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2ea03 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25292800->25296895] (req 25292800->25296895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.093663:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2ea03 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25292800->25296895] (req 25292800->25296895) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.093683:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2ea03 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25292800->25296895] (req 25292800->25296895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.093690:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760749.093712:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2ea03 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25292800->25296895] (req 25292800->25296895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.093721:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2ea03 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25292800->25296895] (req 25292800->25296895) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.093740:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2ea03 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25292800->25296895] (req 25292800->25296895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.093749:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2ea03 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25292800->25296895] (req 25292800->25296895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760749.093837:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac28a1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512651392:12345-192.168.203.56@tcp:4:dd.0 Request processed in 44420us (44652us total) trans 4294973494 rc 0/0 00000100:00100000:1.0:1789760749.093847:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6205 00000100:00100000:1.0:1789760749.093856:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 728520721166 time 728613702472 state 0x2 ref 0 00000100:00100000:1.0:1789760749.105268:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760749.106028:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512651520 00000100:00100000:0.0:1789760749.106039:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512651520 00000100:00100000:0.0:1789760749.106100:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 728520721166 time 728613702472 state 0x0 ref 1 00000100:00100000:0.0:1789760749.106114:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 728613962888 time 728613702472 state 0x1 ref 1 00000100:00100000:0.0:1789760749.106146:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 728613962888 time 728670445968 state 0x0 ref 1 00000100:00100000:0.0:1789760749.106159:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4059180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512651520:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760749.106205:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25296896, mtime: 0x6aad94ec, atime: 0x0, ctime: 0x6aad94ec, blocks: 0xc807 00010000:00010000:0.0:1789760749.106215:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2ea0a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.106230:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=728s v=(0 0) 00010000:00010000:0.0:1789760749.106235:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760749.106242:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2ea0a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25296896->25300991] (req 25296896->25300991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.106256:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2ea0a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25296896->25300991] (req 25296896->25300991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760749.169487:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973495 is committed 00002000:00010000:0.0:1789760749.169557:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25296896 -> 25300992 00010000:00010000:0.0:1789760749.169571:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2ea0a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25296896->25300991] (req 25296896->25300991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.169584:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2ea0a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25296896->25300991] (req 25296896->25300991) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.169593:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2ea0a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25296896->25300991] (req 25296896->25300991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.169607:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760749.169612:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2ea0a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25296896->25300991] (req 25296896->25300991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.169621:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2ea0a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25296896->25300991] (req 25296896->25300991) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.169639:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2ea0a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25296896->25300991] (req 25296896->25300991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.169646:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2ea0a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25296896->25300991] (req 25296896->25300991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760749.169904:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4059180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512651520:12345-192.168.203.56@tcp:4:dd.0 Request processed in 63742us (64640us total) trans 4294973495 rc 0/0 00000100:00100000:0.0:1789760749.169914:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6206 00000100:00100000:0.0:1789760749.169926:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 728613962888 time 728670445968 state 0x2 ref 0 00000100:00100000:0.0:1789760749.178558:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760749.178667:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512651648 00000100:00100000:1.0:1789760749.178677:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512651648 00000100:00100000:1.0:1789760749.178765:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 728613962888 time 728670445968 state 0x0 ref 1 00000100:00100000:1.0:1789760749.178780:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 728670706384 time 728670445968 state 0x1 ref 1 00000100:00100000:1.0:1789760749.178896:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 728670706384 time 728743195128 state 0x0 ref 1 00000100:00100000:1.0:1789760749.178910:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2745500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512651648:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760749.178959:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25300992, mtime: 0x6aad94ec, atime: 0x0, ctime: 0x6aad94ec, blocks: 0xc807 00010000:00010000:1.0:1789760749.178968:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2ea11 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.178982:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=728s v=(0 0) 00010000:00010000:1.0:1789760749.178987:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760749.178993:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2ea11 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25300992->25305087] (req 25300992->25305087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.179007:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2ea11 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25300992->25305087] (req 25300992->25305087) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760749.244674:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25300992 -> 25305088 00010000:00010000:0.0:1789760749.244728:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2ea11 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25300992->25305087] (req 25300992->25305087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.244742:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2ea11 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25300992->25305087] (req 25300992->25305087) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.244757:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2ea11 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25300992->25305087] (req 25300992->25305087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.244765:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760749.244770:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2ea11 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25300992->25305087] (req 25300992->25305087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.244779:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2ea11 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25300992->25305087] (req 25300992->25305087) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.244798:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2ea11 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25300992->25305087] (req 25300992->25305087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.244806:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2ea11 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25300992->25305087] (req 25300992->25305087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760749.244885:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2745500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2361:x1876699512651648:12345-192.168.203.56@tcp:4:dd.0 Request processed in 65974us (66333us total) trans 4294973496 rc 0/0 00000100:00100000:0.0:1789760749.244897:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6207 00000100:00100000:0.0:1789760749.244907:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 728670706384 time 728743195128 state 0x2 ref 0 00000001:00080000:0.0:1789760749.244956:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973496 is committed 00000100:00100000:0.0:1789760749.247621:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760749.247745:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512651776 00000100:00100000:1.0:1789760749.247756:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512651776 00000100:00100000:1.0:1789760749.247826:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 728670706384 time 728743195128 state 0x0 ref 1 00000100:00100000:1.0:1789760749.247842:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 728743455544 time 728743195128 state 0x1 ref 1 00000100:00100000:1.0:1789760749.247873:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 728743455544 time 728812173257 state 0x0 ref 1 00000100:00100000:1.0:1789760749.247886:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2746a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512651776:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760749.247934:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25305088, mtime: 0x6aad94ec, atime: 0x0, ctime: 0x6aad94ec, blocks: 0xc807 00010000:00010000:1.0:1789760749.247943:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2ea18 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.248065:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=728s v=(0 0) 00010000:00010000:1.0:1789760749.248070:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760749.248077:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2ea18 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25305088->25309183] (req 25305088->25309183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.248092:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2ea18 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25305088->25309183] (req 25305088->25309183) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760749.297434:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25305088 -> 25309184 00010000:00010000:0.0:1789760749.297458:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2ea18 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25305088->25309183] (req 25305088->25309183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.297473:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2ea18 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25305088->25309183] (req 25305088->25309183) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.297482:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2ea18 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25305088->25309183] (req 25305088->25309183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.297490:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760749.297495:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2ea18 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25305088->25309183] (req 25305088->25309183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.297505:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2ea18 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25305088->25309183] (req 25305088->25309183) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.297526:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2ea18 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25305088->25309183] (req 25305088->25309183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.297535:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2ea18 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25305088->25309183] (req 25305088->25309183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760749.297632:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2746a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2359:x1876699512651776:12345-192.168.203.56@tcp:4:dd.0 Request processed in 49744us (50018us total) trans 4294973497 rc 0/0 00000100:00100000:0.0:1789760749.297642:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6208 00000100:00100000:0.0:1789760749.297652:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 728743455544 time 728812173257 state 0x2 ref 0 00000001:00080000:1.0:1789760749.297833:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973497 is committed 00000100:00100000:1.0:1789760749.302560:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760749.302635:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512651904 00000100:00100000:0.0:1789760749.302645:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512651904 00000100:00100000:0.0:1789760749.302732:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 728743455544 time 728812173257 state 0x0 ref 1 00000100:00100000:0.0:1789760749.302748:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 728812433673 time 728812173257 state 0x1 ref 1 00000100:00100000:0.0:1789760749.302779:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 728812433673 time 728867078405 state 0x0 ref 1 00000100:00100000:0.0:1789760749.302792:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4059500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512651904:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760749.302844:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25309184, mtime: 0x6aad94ec, atime: 0x0, ctime: 0x6aad94ec, blocks: 0xc807 00010000:00010000:0.0:1789760749.302852:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2ea1f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.302868:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=728s v=(0 0) 00010000:00010000:0.0:1789760749.302872:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760749.302878:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2ea1f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25309184->25313279] (req 25309184->25313279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.302892:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2ea1f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25309184->25313279] (req 25309184->25313279) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760749.360189:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25309184 -> 25313280 00010000:00010000:0.0:1789760749.360212:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2ea1f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25309184->25313279] (req 25309184->25313279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.360225:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2ea1f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25309184->25313279] (req 25309184->25313279) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.360240:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2ea1f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25309184->25313279] (req 25309184->25313279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.360249:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760749.360254:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2ea1f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25309184->25313279] (req 25309184->25313279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.360272:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2ea1f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25309184->25313279] (req 25309184->25313279) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.360305:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2ea1f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25309184->25313279] (req 25309184->25313279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.360314:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2ea1f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25309184->25313279] (req 25309184->25313279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760749.360416:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4059500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2358:x1876699512651904:12345-192.168.203.56@tcp:4:dd.0 Request processed in 57620us (57860us total) trans 4294973498 rc 0/0 00000100:00100000:0.0:1789760749.360429:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6209 00000100:00100000:0.0:1789760749.360438:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 728812433673 time 728867078405 state 0x2 ref 0 00000001:00080000:1.0:1789760749.363999:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973498 is committed 00000100:00100000:3.0:1789760749.368175:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760749.368415:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512652032 00000100:00100000:0.0:1789760749.368425:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512652032 00000100:00100000:0.0:1789760749.368484:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 728812433673 time 728867078405 state 0x0 ref 1 00000100:00100000:0.0:1789760749.368499:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 728867338821 time 728867078405 state 0x1 ref 1 00000100:00100000:0.0:1789760749.368525:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 728867338821 time 728932824634 state 0x0 ref 1 00000100:00100000:0.0:1789760749.368537:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2875880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512652032:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760749.368585:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25313280, mtime: 0x6aad94ec, atime: 0x0, ctime: 0x6aad94ec, blocks: 0xc807 00010000:00010000:0.0:1789760749.368594:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2ea26 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.368611:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=728s v=(0 0) 00010000:00010000:0.0:1789760749.368615:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760749.368621:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2ea26 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25313280->25317375] (req 25313280->25317375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.368635:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2ea26 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25313280->25317375] (req 25313280->25317375) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760749.401251:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973499 is committed 00002000:00010000:1.0:1789760749.401386:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25313280 -> 25317376 00010000:00010000:1.0:1789760749.401401:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2ea26 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25313280->25317375] (req 25313280->25317375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.401413:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2ea26 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25313280->25317375] (req 25313280->25317375) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.401422:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2ea26 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25313280->25317375] (req 25313280->25317375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.401429:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760749.401434:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2ea26 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25313280->25317375] (req 25313280->25317375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.401443:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2ea26 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25313280->25317375] (req 25313280->25317375) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.401461:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2ea26 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25313280->25317375] (req 25313280->25317375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.401468:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2ea26 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25313280->25317375] (req 25313280->25317375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760749.401601:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2875880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512652032:12345-192.168.203.56@tcp:4:dd.0 Request processed in 33061us (33434us total) trans 4294973499 rc 0/0 00000100:00100000:1.0:1789760749.401610:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6210 00000100:00100000:1.0:1789760749.401620:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 728867338821 time 728932824634 state 0x2 ref 0 00000100:00100000:3.0:1789760749.406549:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760749.406606:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512652160 00000100:00100000:0.0:1789760749.406616:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512652160 00000100:00100000:0.0:1789760749.406675:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 728867338821 time 728932824634 state 0x0 ref 1 00000100:00100000:0.0:1789760749.406693:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 728933085050 time 728932824634 state 0x1 ref 1 00000100:00100000:0.0:1789760749.406763:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 728933085050 time 728971061584 state 0x0 ref 1 00000100:00100000:0.0:1789760749.406776:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2875c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512652160:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760749.406823:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25317376, mtime: 0x6aad94ec, atime: 0x0, ctime: 0x6aad94ec, blocks: 0xc807 00010000:00010000:0.0:1789760749.406837:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2ea2d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.406856:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=728s v=(0 0) 00010000:00010000:0.0:1789760749.406861:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760749.406867:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2ea2d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25317376->25321471] (req 25317376->25321471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.406881:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2ea2d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25317376->25321471] (req 25317376->25321471) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760749.434273:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973500 is committed 00002000:00010000:2.0:1789760749.434370:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25317376 -> 25321472 00010000:00010000:2.0:1789760749.434386:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2ea2d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25317376->25321471] (req 25317376->25321471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.434398:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2ea2d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25317376->25321471] (req 25317376->25321471) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.434407:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2ea2d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25317376->25321471] (req 25317376->25321471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.434414:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760749.434420:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2ea2d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25317376->25321471] (req 25317376->25321471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.434430:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2ea2d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25317376->25321471] (req 25317376->25321471) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.434450:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2ea2d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25317376->25321471] (req 25317376->25321471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.434458:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2ea2d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25317376->25321471] (req 25317376->25321471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760749.434586:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2875c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512652160:12345-192.168.203.56@tcp:4:dd.0 Request processed in 27808us (28044us total) trans 4294973500 rc 0/0 00000100:00100000:2.0:1789760749.434598:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6211 00000100:00100000:2.0:1789760749.434609:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 728933085050 time 728971061584 state 0x2 ref 0 00000100:00100000:3.0:1789760749.437430:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760749.437563:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512652288 00000100:00100000:0.0:1789760749.437574:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512652288 00000100:00100000:0.0:1789760749.437630:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 728933085050 time 728971061584 state 0x0 ref 1 00000100:00100000:0.0:1789760749.437646:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 728971322000 time 728971061584 state 0x1 ref 1 00000100:00100000:0.0:1789760749.437849:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 728971322000 time 729002147541 state 0x0 ref 1 00000100:00100000:0.0:1789760749.437863:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2fb6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512652288:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760749.437910:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25321472, mtime: 0x6aad94ec, atime: 0x0, ctime: 0x6aad94ec, blocks: 0xc807 00010000:00010000:0.0:1789760749.437919:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2ea34 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.437935:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=729s v=(0 0) 00010000:00010000:0.0:1789760749.437939:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760749.438010:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2ea34 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25321472->25325567] (req 25321472->25325567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.438031:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2ea34 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25321472->25325567] (req 25321472->25325567) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760749.464245:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973501 is committed 00002000:00010000:1.0:1789760749.464455:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25321472 -> 25325568 00010000:00010000:1.0:1789760749.464471:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2ea34 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25321472->25325567] (req 25321472->25325567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.464484:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2ea34 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25321472->25325567] (req 25321472->25325567) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.464493:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2ea34 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25321472->25325567] (req 25321472->25325567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.464500:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760749.464504:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2ea34 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25321472->25325567] (req 25321472->25325567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.464513:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2ea34 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25321472->25325567] (req 25321472->25325567) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.464532:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2ea34 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25321472->25325567] (req 25321472->25325567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.464539:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2ea34 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25321472->25325567] (req 25321472->25325567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760749.464633:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2fb6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512652288:12345-192.168.203.56@tcp:4:dd.0 Request processed in 26768us (27209us total) trans 4294973501 rc 0/0 00000100:00100000:1.0:1789760749.464643:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6212 00000100:00100000:1.0:1789760749.464654:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 728971322000 time 729002147541 state 0x2 ref 0 00000100:00100000:2.0:1789760749.467301:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760749.467386:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512652416 00000100:00100000:1.0:1789760749.467397:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512652416 00000100:00100000:1.0:1789760749.467449:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 728971322000 time 729002147541 state 0x0 ref 1 00000100:00100000:1.0:1789760749.467463:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 729002407957 time 729002147541 state 0x1 ref 1 00000100:00100000:1.0:1789760749.467492:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 729002407957 time 729031792414 state 0x0 ref 1 00000100:00100000:1.0:1789760749.467505:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac28a3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512652416:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760749.467554:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25325568, mtime: 0x6aad94ec, atime: 0x0, ctime: 0x6aad94ec, blocks: 0xc807 00010000:00010000:1.0:1789760749.467562:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2ea3b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.467578:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=729s v=(0 0) 00010000:00010000:1.0:1789760749.467583:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760749.467589:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2ea3b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25325568->25329663] (req 25325568->25329663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.467602:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2ea3b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25325568->25329663] (req 25325568->25329663) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760749.509075:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25325568 -> 25329664 00010000:00010000:0.0:1789760749.509094:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2ea3b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25325568->25329663] (req 25325568->25329663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.509108:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2ea3b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25325568->25329663] (req 25325568->25329663) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.509118:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2ea3b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25325568->25329663] (req 25325568->25329663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.509126:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760749.509132:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2ea3b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25325568->25329663] (req 25325568->25329663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.509142:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2ea3b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25325568->25329663] (req 25325568->25329663) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.509160:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2ea3b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25325568->25329663] (req 25325568->25329663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.509169:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2ea3b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25325568->25329663] (req 25325568->25329663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760749.509261:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac28a3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2361:x1876699512652416:12345-192.168.203.56@tcp:4:dd.0 Request processed in 41753us (41964us total) trans 4294973502 rc 0/0 00000100:00100000:0.0:1789760749.509272:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6213 00000100:00100000:0.0:1789760749.509291:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 729002407957 time 729031792414 state 0x2 ref 0 00000001:00080000:0.0:1789760749.509329:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973502 is committed 00000100:00100000:0.0:1789760749.512406:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760749.512488:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512652544 00000100:00100000:2.0:1789760749.512497:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512652544 00000100:00100000:2.0:1789760749.512547:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 729002407957 time 729031792414 state 0x0 ref 1 00000100:00100000:2.0:1789760749.512564:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 729032052830 time 729031792414 state 0x1 ref 1 00000100:00100000:2.0:1789760749.512595:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 729032052830 time 729076894898 state 0x0 ref 1 00000100:00100000:2.0:1789760749.512609:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2745880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512652544:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760749.512654:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25329664, mtime: 0x6aad94ec, atime: 0x0, ctime: 0x6aad94ec, blocks: 0xc807 00010000:00010000:2.0:1789760749.512664:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2ea42 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.512691:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=729s v=(0 0) 00010000:00010000:2.0:1789760749.512720:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760749.512728:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2ea42 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25329664->25333759] (req 25329664->25333759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.512742:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2ea42 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25329664->25333759] (req 25329664->25333759) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760749.546629:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25329664 -> 25333760 00002000:00010000:1.0:1789760749.546638:0:6708:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760748 -> 1789760749 00002000:00010000:1.0:1789760749.546642:0:6708:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760748 -> 1789760749 00010000:00010000:1.0:1789760749.546653:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2ea42 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25329664->25333759] (req 25329664->25333759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.546678:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2ea42 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25329664->25333759] (req 25329664->25333759) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.546689:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2ea42 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25329664->25333759] (req 25329664->25333759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.546733:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760749.546739:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2ea42 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25329664->25333759] (req 25329664->25333759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.546749:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2ea42 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25329664->25333759] (req 25329664->25333759) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.546771:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2ea42 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25329664->25333759] (req 25329664->25333759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.546778:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2ea42 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25329664->25333759] (req 25329664->25333759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760749.547066:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2745880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2360:x1876699512652544:12345-192.168.203.56@tcp:4:dd.0 Request processed in 34454us (34665us total) trans 4294973503 rc 0/0 00000100:00100000:1.0:1789760749.547077:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6214 00000100:00100000:1.0:1789760749.547087:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 729032052830 time 729076894898 state 0x2 ref 0 00000001:00080000:1.0:1789760749.548000:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973503 is committed 00000100:00100000:2.0:1789760749.550521:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760749.550601:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512652672 00000100:00100000:1.0:1789760749.550612:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512652672 00000100:00100000:1.0:1789760749.550679:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 729032052830 time 729076894898 state 0x0 ref 1 00000100:00100000:1.0:1789760749.550722:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 729077155314 time 729076894898 state 0x1 ref 1 00000100:00100000:1.0:1789760749.550745:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 729077155314 time 729115045084 state 0x0 ref 1 00000100:00100000:1.0:1789760749.550758:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febc7b33480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512652672:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760749.550817:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25333760, mtime: 0x6aad94ed, atime: 0x0, ctime: 0x6aad94ed, blocks: 0xc807 00010000:00010000:1.0:1789760749.550826:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2ea49 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.550852:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=729s v=(0 0) 00010000:00010000:1.0:1789760749.550858:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760749.550864:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2ea49 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25333760->25337855] (req 25333760->25337855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.550878:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2ea49 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25333760->25337855] (req 25333760->25337855) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760749.586080:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973504 is committed 00002000:00010000:0.0:1789760749.586161:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25333760 -> 25337856 00010000:00010000:0.0:1789760749.586177:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2ea49 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25333760->25337855] (req 25333760->25337855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.586190:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2ea49 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25333760->25337855] (req 25333760->25337855) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.586200:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2ea49 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25333760->25337855] (req 25333760->25337855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.586209:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760749.586214:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2ea49 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25333760->25337855] (req 25333760->25337855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.586224:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2ea49 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25333760->25337855] (req 25333760->25337855) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.586244:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2ea49 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25333760->25337855] (req 25333760->25337855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.586253:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2ea49 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25333760->25337855] (req 25333760->25337855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760749.586363:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febc7b33480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512652672:12345-192.168.203.56@tcp:4:dd.0 Request processed in 35602us (35845us total) trans 4294973504 rc 0/0 00000100:00100000:0.0:1789760749.586374:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6215 00000100:00100000:0.0:1789760749.586384:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 729077155314 time 729115045084 state 0x2 ref 0 00000100:00100000:3.0:1789760749.589324:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760749.589394:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512652800 00000100:00100000:0.0:1789760749.589405:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512652800 00000100:00100000:0.0:1789760749.589457:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 729077155314 time 729115045084 state 0x0 ref 1 00000100:00100000:0.0:1789760749.589472:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 729115305500 time 729115045084 state 0x1 ref 1 00000100:00100000:0.0:1789760749.589503:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 729115305500 time 729153803110 state 0x0 ref 1 00000100:00100000:0.0:1789760749.589515:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac287a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512652800:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760749.589562:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25337856, mtime: 0x6aad94ed, atime: 0x0, ctime: 0x6aad94ed, blocks: 0xc807 00010000:00010000:0.0:1789760749.589571:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2ea50 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.589587:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=729s v=(0 0) 00010000:00010000:0.0:1789760749.589591:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760749.589597:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2ea50 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25337856->25341951] (req 25337856->25341951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.589616:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2ea50 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25337856->25341951] (req 25337856->25341951) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760749.626781:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973505 is committed 00002000:00010000:1.0:1789760749.627316:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25337856 -> 25341952 00010000:00010000:1.0:1789760749.627332:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2ea50 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25337856->25341951] (req 25337856->25341951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.627345:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2ea50 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25337856->25341951] (req 25337856->25341951) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.627359:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2ea50 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25337856->25341951] (req 25337856->25341951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.627367:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760749.627372:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2ea50 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25337856->25341951] (req 25337856->25341951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.627381:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2ea50 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25337856->25341951] (req 25337856->25341951) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.627401:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2ea50 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25337856->25341951] (req 25337856->25341951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.627408:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2ea50 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25337856->25341951] (req 25337856->25341951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760749.627512:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac287a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512652800:12345-192.168.203.56@tcp:4:dd.0 Request processed in 37994us (38192us total) trans 4294973505 rc 0/0 00000100:00100000:1.0:1789760749.627523:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6216 00000100:00100000:1.0:1789760749.627533:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 729115305500 time 729153803110 state 0x2 ref 0 00000100:00100000:0.0:1789760749.632312:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760749.632430:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512652928 00000100:00100000:2.0:1789760749.632439:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512652928 00000100:00100000:2.0:1789760749.632495:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 729115305500 time 729153803110 state 0x0 ref 1 00000100:00100000:2.0:1789760749.632510:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 729154063526 time 729153803110 state 0x1 ref 1 00000100:00100000:2.0:1789760749.632542:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 729154063526 time 729196841455 state 0x0 ref 1 00000100:00100000:2.0:1789760749.632554:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2745180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512652928:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760749.632616:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25341952, mtime: 0x6aad94ed, atime: 0x0, ctime: 0x6aad94ed, blocks: 0xc807 00010000:00010000:2.0:1789760749.632625:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2ea57 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.632641:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=729s v=(0 0) 00010000:00010000:2.0:1789760749.632646:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760749.632652:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2ea57 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25341952->25346047] (req 25341952->25346047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.632876:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2ea57 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25341952->25346047] (req 25341952->25346047) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760749.696251:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973506 is committed 00002000:00010000:0.0:1789760749.696412:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25341952 -> 25346048 00010000:00010000:0.0:1789760749.696428:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2ea57 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25341952->25346047] (req 25341952->25346047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.696441:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2ea57 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25341952->25346047] (req 25341952->25346047) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.696451:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2ea57 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25341952->25346047] (req 25341952->25346047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.696457:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760749.696462:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2ea57 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25341952->25346047] (req 25341952->25346047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.696471:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2ea57 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25341952->25346047] (req 25341952->25346047) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.696488:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2ea57 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25341952->25346047] (req 25341952->25346047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.696496:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2ea57 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25341952->25346047] (req 25341952->25346047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760749.696587:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2745180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512652928:12345-192.168.203.56@tcp:4:dd.0 Request processed in 64029us (64278us total) trans 4294973506 rc 0/0 00000100:00100000:0.0:1789760749.696598:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6217 00000100:00100000:0.0:1789760749.697196:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 729154063526 time 729196841455 state 0x2 ref 0 00000100:00100000:2.0:1789760749.702234:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760749.702793:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512653056 00000100:00100000:0.0:1789760749.702805:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512653056 00000100:00100000:0.0:1789760749.702874:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 729154063526 time 729196841455 state 0x0 ref 1 00000100:00100000:0.0:1789760749.702890:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 729197101871 time 729196841455 state 0x1 ref 1 00000100:00100000:0.0:1789760749.702922:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 729197101871 time 729267221316 state 0x0 ref 1 00000100:00100000:0.0:1789760749.702934:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febc7b32680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512653056:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760749.702994:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25346048, mtime: 0x6aad94ed, atime: 0x0, ctime: 0x6aad94ed, blocks: 0xc807 00010000:00010000:0.0:1789760749.703003:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2ea5e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.703019:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=729s v=(0 0) 00010000:00010000:0.0:1789760749.703023:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760749.703029:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2ea5e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25346048->25350143] (req 25346048->25350143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.703044:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2ea5e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25346048->25350143] (req 25346048->25350143) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760749.753526:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973507 is committed 00002000:00010000:0.0:1789760749.756146:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25346048 -> 25350144 00010000:00010000:0.0:1789760749.756165:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2ea5e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25346048->25350143] (req 25346048->25350143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.756178:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2ea5e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25346048->25350143] (req 25346048->25350143) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.756187:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2ea5e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25346048->25350143] (req 25346048->25350143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.756194:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760749.756198:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2ea5e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25346048->25350143] (req 25346048->25350143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.756207:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2ea5e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25346048->25350143] (req 25346048->25350143) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.756228:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2ea5e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25346048->25350143] (req 25346048->25350143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.756237:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2ea5e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25346048->25350143] (req 25346048->25350143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760749.756352:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febc7b32680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512653056:12345-192.168.203.56@tcp:4:dd.0 Request processed in 53415us (54122us total) trans 4294973507 rc 0/0 00000100:00100000:0.0:1789760749.756366:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6218 00000100:00100000:0.0:1789760749.756376:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 729197101871 time 729267221316 state 0x2 ref 0 00000100:00100000:2.0:1789760749.759693:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760749.759818:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512653184 00000100:00100000:1.0:1789760749.759826:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512653184 00000100:00100000:1.0:1789760749.759872:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 729197101871 time 729267221316 state 0x0 ref 1 00000100:00100000:1.0:1789760749.759887:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 729267481732 time 729267221316 state 0x1 ref 1 00000100:00100000:1.0:1789760749.759913:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 729267481732 time 729324213810 state 0x0 ref 1 00000100:00100000:1.0:1789760749.759924:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febea899180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512653184:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760749.760058:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25350144, mtime: 0x6aad94ed, atime: 0x0, ctime: 0x6aad94ed, blocks: 0xc807 00010000:00010000:1.0:1789760749.760067:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2ea65 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.760083:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=729s v=(0 0) 00010000:00010000:1.0:1789760749.760087:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760749.760094:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2ea65 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25350144->25354239] (req 25350144->25354239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.760107:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2ea65 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25350144->25354239] (req 25350144->25354239) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760749.801356:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973508 is committed 00002000:00010000:1.0:1789760749.801461:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25350144 -> 25354240 00010000:00010000:1.0:1789760749.801476:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2ea65 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25350144->25354239] (req 25350144->25354239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.801489:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2ea65 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25350144->25354239] (req 25350144->25354239) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.801498:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2ea65 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25350144->25354239] (req 25350144->25354239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.801505:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760749.801509:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2ea65 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25350144->25354239] (req 25350144->25354239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.801517:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2ea65 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25350144->25354239] (req 25350144->25354239) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.801536:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2ea65 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25350144->25354239] (req 25350144->25354239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.801544:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2ea65 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25350144->25354239] (req 25350144->25354239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760749.801639:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febea899180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512653184:12345-192.168.203.56@tcp:4:dd.0 Request processed in 41710us (41949us total) trans 4294973508 rc 0/0 00000100:00100000:1.0:1789760749.801649:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6219 00000100:00100000:1.0:1789760749.801658:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 729267481732 time 729324213810 state 0x2 ref 0 00000100:00100000:0.0:1789760749.805202:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760749.805379:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512653312 00000100:00100000:2.0:1789760749.805389:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512653312 00000100:00100000:2.0:1789760749.805440:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 729267481732 time 729324213810 state 0x0 ref 1 00000100:00100000:2.0:1789760749.805456:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 729324474226 time 729324213810 state 0x1 ref 1 00000100:00100000:2.0:1789760749.805526:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 729324474226 time 729369825582 state 0x0 ref 1 00000100:00100000:2.0:1789760749.805539:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2744a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512653312:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760749.805587:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25354240, mtime: 0x6aad94ed, atime: 0x0, ctime: 0x6aad94ed, blocks: 0xc807 00010000:00010000:2.0:1789760749.805596:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2ea6c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.805612:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=729s v=(0 0) 00010000:00010000:2.0:1789760749.805616:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760749.805623:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2ea6c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25354240->25358335] (req 25354240->25358335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.805640:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2ea6c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25354240->25358335] (req 25354240->25358335) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760749.836949:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760749.836993:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760749.837426:0:8229:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512653568 00000100:00100000:2.0:1789760749.837437:0:8229:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512653568 00000100:00100000:2.0:1789760749.837494:0:8229:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 479 00000100:00100000:2.0:1789760749.837502:0:8229:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2746680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+8:2361:x1876699512653568:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:2.0:1789760749.837568:0:8229:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2746680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+8:2361:x1876699512653568:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 66us (574us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760749.837575:0:8229:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 479 00000100:00100000:3.0:1789760749.838099:0:5842:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512653440 00000100:00100000:3.0:1789760749.838105:0:5842:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512653440 00000100:00100000:3.0:1789760749.838144:0:5842:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 400 00000100:00100000:3.0:1789760749.838152:0:5842:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febe0501c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:e6cf7267-9b19-462a-a263-14115df15dab+13:2361:x1876699512653440:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:3.0:1789760749.838188:0:5842:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febe0501c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:e6cf7267-9b19-462a-a263-14115df15dab+13:2361:x1876699512653440:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 37us (1258us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760749.838195:0:5842:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 400 00000100:00080000:3.0:1789760749.838222:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789760749.841652:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760749.842216:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512653696 00000100:00100000:3.0:1789760749.842225:0:5847:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512653696 00000100:00080000:3.0:1789760749.842774:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789760749.842792:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 480 00000100:00100000:3.0:1789760749.842803:0:5847:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febea89b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:d2c40150-e97b-40da-952b-cb9f18bf7e96+5:2360:x1876699512653696:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:3.0:1789760749.842882:0:5847:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febea89b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:d2c40150-e97b-40da-952b-cb9f18bf7e96+5:2360:x1876699512653696:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 82us (1237us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760749.842890:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 480 00002000:00010000:0.0:1789760749.851121:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25354240 -> 25358336 00010000:00010000:0.0:1789760749.851140:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2ea6c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25354240->25358335] (req 25354240->25358335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.851153:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2ea6c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25354240->25358335] (req 25354240->25358335) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.851162:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2ea6c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25354240->25358335] (req 25354240->25358335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.851169:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760749.851173:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2ea6c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25354240->25358335] (req 25354240->25358335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.851182:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2ea6c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25354240->25358335] (req 25354240->25358335) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.851201:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2ea6c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25354240->25358335] (req 25354240->25358335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.851214:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2ea6c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25354240->25358335] (req 25354240->25358335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760749.851324:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2744a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2358:x1876699512653312:12345-192.168.203.56@tcp:4:dd.0 Request processed in 45782us (46128us total) trans 4294973509 rc 0/0 00000100:00100000:0.0:1789760749.851335:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6220 00000100:00100000:0.0:1789760749.851344:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 729324474226 time 729369825582 state 0x2 ref 0 00000001:00080000:1.0:1789760749.851427:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973509 is committed 00000100:00100000:2.0:1789760749.855192:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760749.855286:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512653824 00000100:00100000:0.0:1789760749.855296:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512653824 00000100:00100000:0.0:1789760749.855351:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 729324474226 time 729369825582 state 0x0 ref 1 00000100:00100000:0.0:1789760749.855367:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 729370085998 time 729369825582 state 0x1 ref 1 00000100:00100000:0.0:1789760749.855399:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 729370085998 time 729419698310 state 0x0 ref 1 00000100:00100000:0.0:1789760749.855411:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febea89b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512653824:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760749.855458:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25358336, mtime: 0x6aad94ed, atime: 0x0, ctime: 0x6aad94ed, blocks: 0xc807 00010000:00010000:0.0:1789760749.855466:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2ea73 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.855482:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=729s v=(0 0) 00010000:00010000:0.0:1789760749.855486:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760749.855493:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2ea73 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25358336->25362431] (req 25358336->25362431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.855510:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2ea73 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25358336->25362431] (req 25358336->25362431) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760749.881491:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973510 is committed 00002000:00010000:1.0:1789760749.881596:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25358336 -> 25362432 00010000:00010000:1.0:1789760749.881612:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2ea73 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25358336->25362431] (req 25358336->25362431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.881625:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2ea73 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25358336->25362431] (req 25358336->25362431) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.881638:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2ea73 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25358336->25362431] (req 25358336->25362431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.881644:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760749.881650:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2ea73 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25358336->25362431] (req 25358336->25362431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.881659:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2ea73 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25358336->25362431] (req 25358336->25362431) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.881679:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2ea73 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25358336->25362431] (req 25358336->25362431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.881688:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2ea73 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25358336->25362431] (req 25358336->25362431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760749.881831:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febea89b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512653824:12345-192.168.203.56@tcp:4:dd.0 Request processed in 26416us (26643us total) trans 4294973510 rc 0/0 00000100:00100000:1.0:1789760749.881841:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6221 00000100:00100000:1.0:1789760749.881861:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 729370085998 time 729419698310 state 0x2 ref 0 00000100:00100000:3.0:1789760749.885880:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760749.886098:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512653952 00000100:00100000:1.0:1789760749.886109:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512653952 00000100:00100000:1.0:1789760749.886174:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 729370085998 time 729419698310 state 0x0 ref 1 00000100:00100000:1.0:1789760749.886191:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 729419958726 time 729419698310 state 0x1 ref 1 00000100:00100000:1.0:1789760749.886220:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 729419958726 time 729450519958 state 0x0 ref 1 00000100:00100000:1.0:1789760749.886233:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2879c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512653952:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760749.886303:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25362432, mtime: 0x6aad94ed, atime: 0x0, ctime: 0x6aad94ed, blocks: 0xc807 00010000:00010000:1.0:1789760749.886312:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2ea7a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.886327:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=729s v=(0 0) 00010000:00010000:1.0:1789760749.886332:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760749.886338:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2ea7a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25362432->25366527] (req 25362432->25366527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.886351:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2ea7a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25362432->25366527] (req 25362432->25366527) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760749.924807:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25362432 -> 25366528 00010000:00010000:0.0:1789760749.924825:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2ea7a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25362432->25366527] (req 25362432->25366527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.924837:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2ea7a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25362432->25366527] (req 25362432->25366527) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.924845:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2ea7a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25362432->25366527] (req 25362432->25366527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.924853:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760749.924857:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2ea7a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25362432->25366527] (req 25362432->25366527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.924866:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2ea7a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25362432->25366527] (req 25362432->25366527) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.924884:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2ea7a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25362432->25366527] (req 25362432->25366527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.924891:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2ea7a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25362432->25366527] (req 25362432->25366527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760749.924998:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2879c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2361:x1876699512653952:12345-192.168.203.56@tcp:4:dd.0 Request processed in 38763us (39125us total) trans 4294973511 rc 0/0 00000100:00100000:0.0:1789760749.925009:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6222 00000100:00100000:0.0:1789760749.925018:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 729419958726 time 729450519958 state 0x2 ref 0 00000100:00100000:3.0:1789760749.928338:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760749.928426:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512654080 00000100:00100000:0.0:1789760749.928435:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512654080 00000100:00100000:0.0:1789760749.928488:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 729419958726 time 729450519958 state 0x0 ref 1 00000100:00100000:0.0:1789760749.928502:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 729450780374 time 729450519958 state 0x1 ref 1 00000100:00100000:0.0:1789760749.928532:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 729450780374 time 729492831750 state 0x0 ref 1 00000100:00100000:0.0:1789760749.928545:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2878e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2361:x1876699512654080:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760749.928592:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25366528, mtime: 0x6aad94ed, atime: 0x0, ctime: 0x6aad94ed, blocks: 0xc807 00010000:00010000:0.0:1789760749.928600:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2ea81 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.928616:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=729s v=(0 0) 00010000:00010000:0.0:1789760749.928620:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760749.928626:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2ea81 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25366528->25370623] (req 25366528->25370623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.928639:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2ea81 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25366528->25370623] (req 25366528->25370623) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760749.929535:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973511 is committed 00010000:00010000:2.0:1789760749.938851:0:7647:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:0.0:1789760749.958236:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25366528 -> 25370624 00010000:00010000:0.0:1789760749.958257:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2ea81 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25366528->25370623] (req 25366528->25370623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.958278:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2ea81 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25366528->25370623] (req 25366528->25370623) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.958287:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2ea81 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25366528->25370623] (req 25366528->25370623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.958295:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760749.958300:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2ea81 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25366528->25370623] (req 25366528->25370623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.958310:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2ea81 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25366528->25370623] (req 25366528->25370623) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.958328:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2ea81 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25366528->25370623] (req 25366528->25370623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.958337:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2ea81 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25366528->25370623] (req 25366528->25370623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760749.958432:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2878e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2361:x1876699512654080:12345-192.168.203.56@tcp:4:dd.0 Request processed in 29884us (30098us total) trans 4294973512 rc 0/0 00000100:00100000:0.0:1789760749.958446:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6223 00000100:00100000:0.0:1789760749.958455:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 729450780374 time 729492831750 state 0x2 ref 0 00000001:00080000:3.0:1789760749.958532:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973512 is committed 00000100:00100000:2.0:1789760749.962102:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760749.962195:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512654208 00000100:00100000:1.0:1789760749.962205:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512654208 00000100:00100000:1.0:1789760749.962256:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 729450780374 time 729492831750 state 0x0 ref 1 00000100:00100000:1.0:1789760749.962283:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 729493092166 time 729492831750 state 0x1 ref 1 00000100:00100000:1.0:1789760749.962352:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 729493092166 time 729526651192 state 0x0 ref 1 00000100:00100000:1.0:1789760749.962366:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febea89aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512654208:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760749.962411:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25370624, mtime: 0x6aad94ed, atime: 0x0, ctime: 0x6aad94ed, blocks: 0xc807 00010000:00010000:1.0:1789760749.962420:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2ea88 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.962435:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=729s v=(0 0) 00010000:00010000:1.0:1789760749.962440:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760749.962446:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2ea88 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25370624->25374719] (req 25370624->25374719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.962459:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2ea88 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25370624->25374719] (req 25370624->25374719) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760749.991533:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973513 is committed 00002000:00010000:0.0:1789760749.991633:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25370624 -> 25374720 00010000:00010000:0.0:1789760749.991649:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2ea88 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25370624->25374719] (req 25370624->25374719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.991662:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2ea88 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25370624->25374719] (req 25370624->25374719) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.991671:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2ea88 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25370624->25374719] (req 25370624->25374719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.991678:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760749.991683:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2ea88 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25370624->25374719] (req 25370624->25374719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.991692:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2ea88 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25370624->25374719] (req 25370624->25374719) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.991732:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2ea88 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25370624->25374719] (req 25370624->25374719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.991740:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2ea88 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25370624->25374719] (req 25370624->25374719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760749.991842:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febea89aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512654208:12345-192.168.203.56@tcp:4:dd.0 Request processed in 29474us (29743us total) trans 4294973513 rc 0/0 00000100:00100000:0.0:1789760749.991852:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6224 00000100:00100000:0.0:1789760749.991861:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 729493092166 time 729526651192 state 0x2 ref 0 00000100:00100000:2.0:1789760749.996158:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760749.997442:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512654336 00000100:00100000:0.0:1789760749.997455:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512654336 00000100:00100000:0.0:1789760749.997510:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 729493092166 time 729526651192 state 0x0 ref 1 00000100:00100000:0.0:1789760749.997525:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 729526911608 time 729526651192 state 0x1 ref 1 00000100:00100000:0.0:1789760749.997595:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 729526911608 time 729561894467 state 0x0 ref 1 00000100:00100000:0.0:1789760749.997609:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febffb9f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512654336:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760749.997656:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25374720, mtime: 0x6aad94ed, atime: 0x0, ctime: 0x6aad94ed, blocks: 0xc807 00010000:00010000:0.0:1789760749.997665:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2ea8f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.997681:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=729s v=(0 0) 00010000:00010000:0.0:1789760749.997685:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760749.997692:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2ea8f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25374720->25378815] (req 25374720->25378815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.997742:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2ea8f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25374720->25378815] (req 25374720->25378815) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760750.036205:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973514 is committed 00002000:00010000:0.0:1789760750.036311:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25374720 -> 25378816 00010000:00010000:0.0:1789760750.036327:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2ea8f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25374720->25378815] (req 25374720->25378815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.036339:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2ea8f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25374720->25378815] (req 25374720->25378815) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.036349:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2ea8f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25374720->25378815] (req 25374720->25378815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.036357:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760750.036362:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2ea8f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25374720->25378815] (req 25374720->25378815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.036371:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2ea8f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25374720->25378815] (req 25374720->25378815) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.036391:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2ea8f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25374720->25378815] (req 25374720->25378815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.036399:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2ea8f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25374720->25378815] (req 25374720->25378815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760750.036518:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febffb9f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512654336:12345-192.168.203.56@tcp:4:dd.0 Request processed in 38907us (40364us total) trans 4294973514 rc 0/0 00000100:00100000:0.0:1789760750.036533:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6225 00000100:00100000:0.0:1789760750.036543:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 729526911608 time 729561894467 state 0x2 ref 0 00000100:00100000:3.0:1789760750.041608:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760750.041774:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512654464 00000100:00100000:0.0:1789760750.041785:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512654464 00000100:00100000:0.0:1789760750.041835:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 729526911608 time 729561894467 state 0x0 ref 1 00000100:00100000:0.0:1789760750.041850:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 729562154883 time 729561894467 state 0x1 ref 1 00000100:00100000:0.0:1789760750.041882:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 729562154883 time 729606181652 state 0x0 ref 1 00000100:00100000:0.0:1789760750.041895:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac287b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512654464:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760750.041954:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25378816, mtime: 0x6aad94ed, atime: 0x0, ctime: 0x6aad94ed, blocks: 0xc807 00010000:00010000:0.0:1789760750.041964:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2ea96 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.041980:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=729s v=(0 0) 00010000:00010000:0.0:1789760750.041985:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760750.041991:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2ea96 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25378816->25382911] (req 25378816->25382911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.042005:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2ea96 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25378816->25382911] (req 25378816->25382911) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760750.070690:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973515 is committed 00002000:00010000:0.0:1789760750.070804:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25378816 -> 25382912 00010000:00010000:0.0:1789760750.070819:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2ea96 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25378816->25382911] (req 25378816->25382911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.070831:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2ea96 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25378816->25382911] (req 25378816->25382911) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.070840:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2ea96 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25378816->25382911] (req 25378816->25382911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.070846:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760750.070851:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2ea96 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25378816->25382911] (req 25378816->25382911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.070859:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2ea96 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25378816->25382911] (req 25378816->25382911) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.070877:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2ea96 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25378816->25382911] (req 25378816->25382911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.070885:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2ea96 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25378816->25382911] (req 25378816->25382911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760750.071077:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac287b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512654464:12345-192.168.203.56@tcp:4:dd.0 Request processed in 29179us (29474us total) trans 4294973515 rc 0/0 00000100:00100000:0.0:1789760750.071087:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6226 00000100:00100000:0.0:1789760750.071096:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 729562154883 time 729606181652 state 0x2 ref 0 00000100:00100000:0.0:1789760750.075398:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760750.075510:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512654592 00000100:00100000:1.0:1789760750.075520:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512654592 00000100:00100000:1.0:1789760750.075569:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 729562154883 time 729606181652 state 0x0 ref 1 00000100:00100000:1.0:1789760750.075584:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 729606442068 time 729606181652 state 0x1 ref 1 00000100:00100000:1.0:1789760750.075651:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 729606442068 time 729639949990 state 0x0 ref 1 00000100:00100000:1.0:1789760750.075674:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2746300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512654592:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760750.075747:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25382912, mtime: 0x6aad94ed, atime: 0x0, ctime: 0x6aad94ed, blocks: 0xc807 00010000:00010000:1.0:1789760750.075756:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2ea9d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760750.075771:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=729s v=(0 0) 00010000:00010000:1.0:1789760750.075776:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760750.075783:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2ea9d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25382912->25387007] (req 25382912->25387007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760750.075795:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2ea9d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25382912->25387007] (req 25382912->25387007) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760750.117538:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25382912 -> 25387008 00010000:00010000:0.0:1789760750.117571:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2ea9d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25382912->25387007] (req 25382912->25387007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.117586:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2ea9d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25382912->25387007] (req 25382912->25387007) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.117595:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2ea9d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25382912->25387007] (req 25382912->25387007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.117606:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760750.117611:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2ea9d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25382912->25387007] (req 25382912->25387007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.117620:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2ea9d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25382912->25387007] (req 25382912->25387007) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.117638:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2ea9d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25382912->25387007] (req 25382912->25387007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.117646:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2ea9d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25382912->25387007] (req 25382912->25387007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760750.117772:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2746300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2360:x1876699512654592:12345-192.168.203.56@tcp:4:dd.0 Request processed in 42106us (42379us total) trans 4294973516 rc 0/0 00000100:00100000:0.0:1789760750.117782:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6227 00000100:00100000:0.0:1789760750.117792:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 729606442068 time 729639949990 state 0x2 ref 0 00000001:00080000:3.0:1789760750.119408:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973516 is committed 00000100:00100000:2.0:1789760750.121532:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760750.121751:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512654720 00000100:00100000:0.0:1789760750.121762:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512654720 00000100:00100000:0.0:1789760750.121816:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 729606442068 time 729639949990 state 0x0 ref 1 00000100:00100000:0.0:1789760750.121831:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 729640210406 time 729639949990 state 0x1 ref 1 00000100:00100000:0.0:1789760750.121860:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 729640210406 time 729686159696 state 0x0 ref 1 00000100:00100000:0.0:1789760750.121873:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febffb9ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512654720:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760750.121922:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25387008, mtime: 0x6aad94ed, atime: 0x0, ctime: 0x6aad94ed, blocks: 0xc807 00010000:00010000:0.0:1789760750.121931:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2eaa4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.121957:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=729s v=(0 0) 00010000:00010000:0.0:1789760750.121962:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760750.121968:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2eaa4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25387008->25391103] (req 25387008->25391103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.121981:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2eaa4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25387008->25391103] (req 25387008->25391103) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760750.159670:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25387008 -> 25391104 00010000:00010000:0.0:1789760750.159691:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2eaa4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25387008->25391103] (req 25387008->25391103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.159728:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2eaa4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25387008->25391103] (req 25387008->25391103) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.159738:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2eaa4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25387008->25391103] (req 25387008->25391103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.159745:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760750.159750:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2eaa4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25387008->25391103] (req 25387008->25391103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.159759:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2eaa4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25387008->25391103] (req 25387008->25391103) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.159778:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2eaa4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25387008->25391103] (req 25387008->25391103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.159786:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2eaa4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25387008->25391103] (req 25387008->25391103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760750.159875:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febffb9ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2360:x1876699512654720:12345-192.168.203.56@tcp:4:dd.0 Request processed in 37999us (38346us total) trans 4294973517 rc 0/0 00000100:00100000:0.0:1789760750.159885:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6228 00000100:00100000:0.0:1789760750.159895:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 729640210406 time 729686159696 state 0x2 ref 0 00000001:00080000:3.0:1789760750.159967:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973517 is committed 00000100:00100000:2.0:1789760750.163593:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760750.163809:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512654848 00000100:00100000:0.0:1789760750.163819:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512654848 00000100:00100000:0.0:1789760750.163873:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 729640210406 time 729686159696 state 0x0 ref 1 00000100:00100000:0.0:1789760750.163893:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 729686420112 time 729686159696 state 0x1 ref 1 00000100:00100000:0.0:1789760750.163921:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 729686420112 time 729728221049 state 0x0 ref 1 00000100:00100000:0.0:1789760750.163934:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febffb9c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512654848:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760750.163981:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25391104, mtime: 0x6aad94ed, atime: 0x0, ctime: 0x6aad94ed, blocks: 0xc807 00010000:00010000:0.0:1789760750.164106:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2eaab lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.164123:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=729s v=(0 0) 00010000:00010000:0.0:1789760750.164127:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760750.164135:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2eaab lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25391104->25395199] (req 25391104->25395199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.164149:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2eaab lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25391104->25395199] (req 25391104->25395199) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760750.198907:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973518 is committed 00002000:00010000:0.0:1789760750.199088:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25391104 -> 25395200 00010000:00010000:0.0:1789760750.199103:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2eaab lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25391104->25395199] (req 25391104->25395199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.199116:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2eaab lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25391104->25395199] (req 25391104->25395199) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.199126:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2eaab lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25391104->25395199] (req 25391104->25395199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.199134:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760750.199140:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2eaab lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25391104->25395199] (req 25391104->25395199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.199149:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2eaab lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25391104->25395199] (req 25391104->25395199) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.199169:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2eaab lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25391104->25395199] (req 25391104->25395199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.199177:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2eaab lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25391104->25395199] (req 25391104->25395199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760750.199318:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febffb9c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512654848:12345-192.168.203.56@tcp:4:dd.0 Request processed in 35381us (35730us total) trans 4294973518 rc 0/0 00000100:00100000:0.0:1789760750.199329:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6229 00000100:00100000:0.0:1789760750.199338:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 729686420112 time 729728221049 state 0x2 ref 0 00000100:00100000:2.0:1789760750.203395:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760750.203483:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512654976 00000100:00100000:0.0:1789760750.203676:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512654976 00000100:00100000:0.0:1789760750.203955:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 729686420112 time 729728221049 state 0x0 ref 1 00000100:00100000:0.0:1789760750.203971:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 729728481465 time 729728221049 state 0x1 ref 1 00000100:00100000:0.0:1789760750.204038:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 729728481465 time 729768336983 state 0x0 ref 1 00000100:00100000:0.0:1789760750.204052:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febffb9c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512654976:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760750.204106:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25395200, mtime: 0x6aad94ed, atime: 0x0, ctime: 0x6aad94ed, blocks: 0xc807 00010000:00010000:0.0:1789760750.204115:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2eab2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.204131:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=729s v=(0 0) 00010000:00010000:0.0:1789760750.204135:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760750.204142:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2eab2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25395200->25399295] (req 25395200->25399295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.204155:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2eab2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25395200->25399295] (req 25395200->25399295) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760750.238738:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973519 is committed 00002000:00010000:3.0:1789760750.239081:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25395200 -> 25399296 00010000:00010000:3.0:1789760750.239098:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2eab2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25395200->25399295] (req 25395200->25399295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.239111:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2eab2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25395200->25399295] (req 25395200->25399295) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.239126:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2eab2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25395200->25399295] (req 25395200->25399295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.239133:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760750.239139:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2eab2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25395200->25399295] (req 25395200->25399295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.239148:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2eab2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25395200->25399295] (req 25395200->25399295) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.239170:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2eab2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25395200->25399295] (req 25395200->25399295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.239178:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2eab2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25395200->25399295] (req 25395200->25399295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760750.239266:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febffb9c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512654976:12345-192.168.203.56@tcp:4:dd.0 Request processed in 35212us (35876us total) trans 4294973519 rc 0/0 00000100:00100000:3.0:1789760750.239284:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6230 00000100:00100000:3.0:1789760750.239293:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 729728481465 time 729768336983 state 0x2 ref 0 00000100:00100000:2.0:1789760750.251309:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760750.252924:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512655104 00000100:00100000:0.0:1789760750.252948:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512655104 00000100:00100000:0.0:1789760750.253010:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 729728481465 time 729768336983 state 0x0 ref 1 00000100:00100000:0.0:1789760750.253027:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 729768597399 time 729768336983 state 0x1 ref 1 00000100:00100000:0.0:1789760750.253062:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 729768597399 time 729817362196 state 0x0 ref 1 00000100:00100000:0.0:1789760750.253077:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febffb9f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512655104:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760750.253127:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25399296, mtime: 0x6aad94ed, atime: 0x0, ctime: 0x6aad94ed, blocks: 0xc807 00010000:00010000:0.0:1789760750.253137:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2eab9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.253153:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=729s v=(0 0) 00010000:00010000:0.0:1789760750.253158:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760750.253166:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2eab9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25399296->25403391] (req 25399296->25403391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.253180:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2eab9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25399296->25403391] (req 25399296->25403391) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760750.302635:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973520 is committed 00002000:00010000:0.0:1789760750.302872:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25399296 -> 25403392 00010000:00010000:0.0:1789760750.302888:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2eab9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25399296->25403391] (req 25399296->25403391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.302901:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2eab9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25399296->25403391] (req 25399296->25403391) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.302910:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2eab9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25399296->25403391] (req 25399296->25403391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.302918:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760750.302923:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2eab9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25399296->25403391] (req 25399296->25403391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.302932:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2eab9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25399296->25403391] (req 25399296->25403391) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.302962:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2eab9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25399296->25403391] (req 25399296->25403391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.302970:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2eab9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25399296->25403391] (req 25399296->25403391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760750.303064:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febffb9f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512655104:12345-192.168.203.56@tcp:4:dd.0 Request processed in 49985us (51759us total) trans 4294973520 rc 0/0 00000100:00100000:0.0:1789760750.303074:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6231 00000100:00100000:0.0:1789760750.303084:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 729768597399 time 729817362196 state 0x2 ref 0 00000100:00100000:2.0:1789760750.307693:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760750.308057:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512655232 00000100:00100000:1.0:1789760750.308068:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512655232 00000100:00100000:1.0:1789760750.308122:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 729768597399 time 729817362196 state 0x0 ref 1 00000100:00100000:1.0:1789760750.308138:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 729817622612 time 729817362196 state 0x1 ref 1 00000100:00100000:1.0:1789760750.308168:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 729817622612 time 729872468214 state 0x0 ref 1 00000100:00100000:1.0:1789760750.308182:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febea651500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512655232:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760750.308231:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25403392, mtime: 0x6aad94ed, atime: 0x0, ctime: 0x6aad94ed, blocks: 0xc807 00010000:00010000:1.0:1789760750.308241:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2eac0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760750.308266:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=729s v=(0 0) 00010000:00010000:1.0:1789760750.308279:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760750.308286:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2eac0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25403392->25407487] (req 25403392->25407487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760750.308300:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2eac0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25403392->25407487] (req 25403392->25407487) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760750.350776:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25403392 -> 25407488 00010000:00010000:0.0:1789760750.350797:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2eac0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25403392->25407487] (req 25403392->25407487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.350810:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2eac0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25403392->25407487] (req 25403392->25407487) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.350819:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2eac0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25403392->25407487] (req 25403392->25407487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.350827:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760750.350831:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2eac0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25403392->25407487] (req 25403392->25407487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.350841:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2eac0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25403392->25407487] (req 25403392->25407487) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.350858:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2eac0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25403392->25407487] (req 25403392->25407487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.350870:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2eac0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25403392->25407487] (req 25403392->25407487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760750.350963:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febea651500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2360:x1876699512655232:12345-192.168.203.56@tcp:4:dd.0 Request processed in 42780us (43275us total) trans 4294973521 rc 0/0 00000100:00100000:0.0:1789760750.350974:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6232 00000100:00100000:0.0:1789760750.350984:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 729817622612 time 729872468214 state 0x2 ref 0 00000001:00080000:3.0:1789760750.351069:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973521 is committed 00000100:00100000:3.0:1789760750.358075:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760750.358353:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512655360 00000100:00100000:2.0:1789760750.358374:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512655360 00000100:00100000:2.0:1789760750.358431:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 729817622612 time 729872468214 state 0x0 ref 1 00000100:00100000:2.0:1789760750.358450:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 729872728630 time 729872468214 state 0x1 ref 1 00000100:00100000:2.0:1789760750.358480:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 729872728630 time 729922780747 state 0x0 ref 1 00000100:00100000:2.0:1789760750.358494:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2878380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512655360:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760750.358550:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25407488, mtime: 0x6aad94ed, atime: 0x0, ctime: 0x6aad94ed, blocks: 0xc807 00010000:00010000:2.0:1789760750.358559:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2eac7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.358589:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=729s v=(0 0) 00010000:00010000:2.0:1789760750.358593:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760750.358600:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2eac7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25407488->25411583] (req 25407488->25411583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.358614:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2eac7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25407488->25411583] (req 25407488->25411583) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760750.407395:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25407488 -> 25411584 00010000:00010000:2.0:1789760750.407415:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2eac7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25407488->25411583] (req 25407488->25411583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.407430:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2eac7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25407488->25411583] (req 25407488->25411583) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.407440:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2eac7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25407488->25411583] (req 25407488->25411583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.407453:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760750.407458:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2eac7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25407488->25411583] (req 25407488->25411583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.407467:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2eac7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25407488->25411583] (req 25407488->25411583) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.407488:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2eac7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25407488->25411583] (req 25407488->25411583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.407497:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2eac7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25407488->25411583] (req 25407488->25411583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760750.407689:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2878380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2361:x1876699512655360:12345-192.168.203.56@tcp:4:dd.0 Request processed in 49192us (49618us total) trans 4294973522 rc 0/0 00000100:00100000:2.0:1789760750.407739:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6233 00000100:00100000:2.0:1789760750.407750:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 729872728630 time 729922780747 state 0x2 ref 0 00000001:00080000:3.0:1789760750.407877:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973522 is committed 00000100:00100000:1.0:1789760750.417135:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760750.418334:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512655488 00000100:00100000:0.0:1789760750.418347:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512655488 00000100:00100000:2.0:1789760750.418503:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 729872728630 time 729922780747 state 0x0 ref 1 00000100:00100000:2.0:1789760750.418519:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 729923041163 time 729922780747 state 0x1 ref 1 00000100:00100000:2.0:1789760750.418606:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 729923041163 time 729982904862 state 0x0 ref 1 00000100:00100000:2.0:1789760750.418621:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4058a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512655488:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760750.418672:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25411584, mtime: 0x6aad94ed, atime: 0x0, ctime: 0x6aad94ed, blocks: 0xc807 00010000:00010000:2.0:1789760750.418681:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2eace lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.418733:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=729s v=(0 0) 00010000:00010000:2.0:1789760750.418739:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760750.418746:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2eace lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25411584->25415679] (req 25411584->25415679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.418760:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2eace lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25411584->25415679] (req 25411584->25415679) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760750.472906:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973523 is committed 00002000:00010000:2.0:1789760750.474301:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25411584 -> 25415680 00010000:00010000:2.0:1789760750.474318:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2eace lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25411584->25415679] (req 25411584->25415679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.474331:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2eace lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25411584->25415679] (req 25411584->25415679) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.474340:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2eace lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25411584->25415679] (req 25411584->25415679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.474347:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760750.474351:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2eace lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25411584->25415679] (req 25411584->25415679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.474430:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2eace lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25411584->25415679] (req 25411584->25415679) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.474452:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2eace lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25411584->25415679] (req 25411584->25415679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.474460:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2eace lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25411584->25415679] (req 25411584->25415679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760750.474589:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4058a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512655488:12345-192.168.203.56@tcp:4:dd.0 Request processed in 55968us (57459us total) trans 4294973523 rc 0/0 00000100:00100000:2.0:1789760750.474598:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6234 00000100:00100000:2.0:1789760750.474607:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 729923041163 time 729982904862 state 0x2 ref 0 00000100:00100000:1.0:1789760750.487109:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760750.487327:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512655616 00000100:00100000:0.0:1789760750.487337:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512655616 00000100:00100000:0.0:1789760750.487405:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 729923041163 time 729982904862 state 0x0 ref 1 00000100:00100000:0.0:1789760750.487419:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 729983165278 time 729982904862 state 0x1 ref 1 00000100:00100000:0.0:1789760750.487451:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 729983165278 time 730051750338 state 0x0 ref 1 00000100:00100000:0.0:1789760750.487464:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac405ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512655616:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760750.487511:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25415680, mtime: 0x6aad94ed, atime: 0x0, ctime: 0x6aad94ed, blocks: 0xc807 00010000:00010000:0.0:1789760750.487521:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2ead5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.487536:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=730s v=(0 0) 00010000:00010000:0.0:1789760750.487541:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760750.487549:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2ead5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25415680->25419775] (req 25415680->25419775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.487563:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2ead5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25415680->25419775] (req 25415680->25419775) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760750.543406:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25415680 -> 25419776 00010000:00010000:2.0:1789760750.543426:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2ead5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25415680->25419775] (req 25415680->25419775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.543438:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2ead5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25415680->25419775] (req 25415680->25419775) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.543446:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2ead5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25415680->25419775] (req 25415680->25419775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.543453:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760750.543458:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2ead5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25415680->25419775] (req 25415680->25419775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.543472:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2ead5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25415680->25419775] (req 25415680->25419775) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.543491:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2ead5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25415680->25419775] (req 25415680->25419775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.543498:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2ead5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25415680->25419775] (req 25415680->25419775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760750.543588:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac405ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2360:x1876699512655616:12345-192.168.203.56@tcp:4:dd.0 Request processed in 56121us (56483us total) trans 4294973524 rc 0/0 00000100:00100000:2.0:1789760750.543598:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6235 00000100:00100000:2.0:1789760750.543608:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 729983165278 time 730051750338 state 0x2 ref 0 00000001:00080000:0.0:1789760750.543859:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973524 is committed 00000100:00100000:2.0:1789760750.549536:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760750.549644:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512655744 00000100:00100000:0.0:1789760750.549654:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512655744 00000100:00100000:0.0:1789760750.549753:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 729983165278 time 730051750338 state 0x0 ref 1 00000100:00100000:0.0:1789760750.549769:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 730052010754 time 730051750338 state 0x1 ref 1 00000100:00100000:0.0:1789760750.549796:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 730052010754 time 730114096802 state 0x0 ref 1 00000100:00100000:0.0:1789760750.549808:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febea650380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512655744:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760750.549855:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25419776, mtime: 0x6aad94ed, atime: 0x0, ctime: 0x6aad94ed, blocks: 0xc807 00010000:00010000:0.0:1789760750.549863:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2eadc lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.549879:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=730s v=(0 0) 00010000:00010000:0.0:1789760750.549884:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760750.549891:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2eadc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25419776->25423871] (req 25419776->25423871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.549905:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2eadc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25419776->25423871] (req 25419776->25423871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760750.595014:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973525 is committed 00002000:00010000:3.0:1789760750.595155:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25419776 -> 25423872 00002000:00010000:3.0:1789760750.595160:0:6710:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760749 -> 1789760750 00002000:00010000:3.0:1789760750.595162:0:6710:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760749 -> 1789760750 00010000:00010000:3.0:1789760750.595176:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2eadc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25419776->25423871] (req 25419776->25423871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.595187:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2eadc lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25419776->25423871] (req 25419776->25423871) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.595196:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2eadc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25419776->25423871] (req 25419776->25423871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.595203:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760750.595207:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2eadc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25419776->25423871] (req 25419776->25423871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.595216:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2eadc lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25419776->25423871] (req 25419776->25423871) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.595234:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2eadc lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25419776->25423871] (req 25419776->25423871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.595241:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2eadc lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25419776->25423871] (req 25419776->25423871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760750.595339:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febea650380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512655744:12345-192.168.203.56@tcp:4:dd.0 Request processed in 45528us (45808us total) trans 4294973525 rc 0/0 00000100:00100000:3.0:1789760750.595352:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6236 00000100:00100000:3.0:1789760750.595362:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 730052010754 time 730114096802 state 0x2 ref 0 00000100:00100000:3.0:1789760750.598156:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760750.598483:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512655872 00000100:00100000:2.0:1789760750.598493:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512655872 00000100:00100000:2.0:1789760750.598548:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 730052010754 time 730114096802 state 0x0 ref 1 00000100:00100000:2.0:1789760750.598565:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 730114357218 time 730114096802 state 0x1 ref 1 00000100:00100000:2.0:1789760750.598595:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 730114357218 time 730162894855 state 0x0 ref 1 00000100:00100000:2.0:1789760750.598607:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2878000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512655872:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760750.598653:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25423872, mtime: 0x6aad94ee, atime: 0x0, ctime: 0x6aad94ee, blocks: 0xc807 00010000:00010000:2.0:1789760750.598663:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2eae3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.598677:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=730s v=(0 0) 00010000:00010000:2.0:1789760750.598682:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760750.598689:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2eae3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25423872->25427967] (req 25423872->25427967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.598745:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2eae3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25423872->25427967] (req 25423872->25427967) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760750.637871:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973526 is committed 00002000:00010000:3.0:1789760750.638995:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25423872 -> 25427968 00010000:00010000:3.0:1789760750.639013:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2eae3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25423872->25427967] (req 25423872->25427967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.639025:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2eae3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25423872->25427967] (req 25423872->25427967) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.639034:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2eae3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25423872->25427967] (req 25423872->25427967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.639041:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760750.639045:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2eae3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25423872->25427967] (req 25423872->25427967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.639054:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2eae3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25423872->25427967] (req 25423872->25427967) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.639072:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2eae3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25423872->25427967] (req 25423872->25427967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.639079:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2eae3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25423872->25427967] (req 25423872->25427967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760750.639288:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2878000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512655872:12345-192.168.203.56@tcp:4:dd.0 Request processed in 40678us (41136us total) trans 4294973526 rc 0/0 00000100:00100000:3.0:1789760750.639297:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6237 00000100:00100000:3.0:1789760750.639306:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 730114357218 time 730162894855 state 0x2 ref 0 00000100:00100000:1.0:1789760750.642979:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760750.643139:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512656000 00000100:00100000:2.0:1789760750.643152:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512656000 00000100:00100000:2.0:1789760750.643219:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 730114357218 time 730162894855 state 0x0 ref 1 00000100:00100000:2.0:1789760750.643235:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 730163155271 time 730162894855 state 0x1 ref 1 00000100:00100000:2.0:1789760750.643264:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 730163155271 time 730207563389 state 0x0 ref 1 00000100:00100000:2.0:1789760750.643287:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4058e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512656000:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760750.643345:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25427968, mtime: 0x6aad94ee, atime: 0x0, ctime: 0x6aad94ee, blocks: 0xc807 00010000:00010000:2.0:1789760750.643354:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2eaea lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.643374:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=730s v=(0 0) 00010000:00010000:2.0:1789760750.643378:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760750.643386:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2eaea lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25427968->25432063] (req 25427968->25432063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.643403:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2eaea lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25427968->25432063] (req 25427968->25432063) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760750.676608:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973527 is committed 00002000:00010000:2.0:1789760750.676779:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25427968 -> 25432064 00010000:00010000:2.0:1789760750.676795:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2eaea lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25427968->25432063] (req 25427968->25432063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.676808:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2eaea lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25427968->25432063] (req 25427968->25432063) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.676816:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2eaea lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25427968->25432063] (req 25427968->25432063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.676823:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760750.676828:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2eaea lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25427968->25432063] (req 25427968->25432063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.676836:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2eaea lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25427968->25432063] (req 25427968->25432063) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.676855:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2eaea lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25427968->25432063] (req 25427968->25432063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.676863:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2eaea lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25427968->25432063] (req 25427968->25432063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760750.677189:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4058e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512656000:12345-192.168.203.56@tcp:4:dd.0 Request processed in 33899us (34215us total) trans 4294973527 rc 0/0 00000100:00100000:2.0:1789760750.677201:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6238 00000100:00100000:2.0:1789760750.677211:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 730163155271 time 730207563389 state 0x2 ref 0 00000100:00100000:2.0:1789760750.681368:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760750.681480:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512656128 00000100:00100000:3.0:1789760750.681490:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512656128 00000100:00100000:3.0:1789760750.681546:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 730163155271 time 730207563389 state 0x0 ref 1 00000100:00100000:3.0:1789760750.681563:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 730207823805 time 730207563389 state 0x1 ref 1 00000100:00100000:3.0:1789760750.681624:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 730207823805 time 730245922617 state 0x0 ref 1 00000100:00100000:3.0:1789760750.681638:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febea4b5050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512656128:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760750.681714:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25432064, mtime: 0x6aad94ee, atime: 0x0, ctime: 0x6aad94ee, blocks: 0xc807 00010000:00010000:3.0:1789760750.681723:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2eaf1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.681739:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=730s v=(0 0) 00010000:00010000:3.0:1789760750.681743:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760750.681750:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2eaf1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25432064->25436159] (req 25432064->25436159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.681763:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2eaf1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25432064->25436159] (req 25432064->25436159) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760750.711713:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25432064 -> 25436160 00010000:00010000:3.0:1789760750.711738:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2eaf1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25432064->25436159] (req 25432064->25436159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.711752:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2eaf1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25432064->25436159] (req 25432064->25436159) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.711760:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2eaf1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25432064->25436159] (req 25432064->25436159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.711767:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760750.711772:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2eaf1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25432064->25436159] (req 25432064->25436159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.711781:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2eaf1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25432064->25436159] (req 25432064->25436159) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.711800:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2eaf1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25432064->25436159] (req 25432064->25436159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.711808:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2eaf1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25432064->25436159] (req 25432064->25436159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760750.711905:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febea4b5050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2360:x1876699512656128:12345-192.168.203.56@tcp:4:dd.0 Request processed in 30265us (30541us total) trans 4294973528 rc 0/0 00000100:00100000:3.0:1789760750.711919:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6239 00000100:00100000:3.0:1789760750.711929:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 730207823805 time 730245922617 state 0x2 ref 0 00000001:00080000:3.0:1789760750.712239:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973528 is committed 00000100:00100000:0.0:1789760750.715224:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760750.715518:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512656256 00000100:00100000:3.0:1789760750.715528:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512656256 00000100:00100000:3.0:1789760750.715581:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 730207823805 time 730245922617 state 0x0 ref 1 00000100:00100000:3.0:1789760750.715596:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 730246183033 time 730245922617 state 0x1 ref 1 00000100:00100000:3.0:1789760750.715626:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 730246183033 time 730279925789 state 0x0 ref 1 00000100:00100000:3.0:1789760750.715637:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2747b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512656256:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760750.715690:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25436160, mtime: 0x6aad94ee, atime: 0x0, ctime: 0x6aad94ee, blocks: 0xc807 00010000:00010000:3.0:1789760750.715723:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2eaf8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.715739:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=730s v=(0 0) 00010000:00010000:3.0:1789760750.715743:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760750.715749:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2eaf8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25436160->25440255] (req 25436160->25440255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.715763:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2eaf8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25436160->25440255] (req 25436160->25440255) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760750.774963:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25436160 -> 25440256 00010000:00010000:3.0:1789760750.774984:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2eaf8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25436160->25440255] (req 25436160->25440255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.774997:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2eaf8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25436160->25440255] (req 25436160->25440255) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.775006:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2eaf8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25436160->25440255] (req 25436160->25440255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.775013:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760750.775017:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2eaf8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25436160->25440255] (req 25436160->25440255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.775026:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2eaf8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25436160->25440255] (req 25436160->25440255) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.775044:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2eaf8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25436160->25440255] (req 25436160->25440255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.775051:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2eaf8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25436160->25440255] (req 25436160->25440255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760750.775148:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2747b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2360:x1876699512656256:12345-192.168.203.56@tcp:4:dd.0 Request processed in 59508us (59929us total) trans 4294973529 rc 0/0 00000100:00100000:3.0:1789760750.775159:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6240 00000100:00100000:3.0:1789760750.775169:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 730246183033 time 730279925789 state 0x2 ref 0 00000001:00080000:0.0:1789760750.775779:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973529 is committed 00000100:00100000:0.0:1789760750.779222:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760750.779435:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512656384 00000100:00100000:2.0:1789760750.779445:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512656384 00000100:00100000:2.0:1789760750.779500:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 730246183033 time 730279925789 state 0x0 ref 1 00000100:00100000:2.0:1789760750.779517:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 730280186205 time 730279925789 state 0x1 ref 1 00000100:00100000:2.0:1789760750.779549:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 730280186205 time 730343849359 state 0x0 ref 1 00000100:00100000:2.0:1789760750.779561:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2747800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512656384:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760750.779609:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25440256, mtime: 0x6aad94ee, atime: 0x0, ctime: 0x6aad94ee, blocks: 0xc807 00010000:00010000:2.0:1789760750.779618:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2eaff lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.779634:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=730s v=(0 0) 00010000:00010000:2.0:1789760750.779639:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760750.779646:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2eaff lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25440256->25444351] (req 25440256->25444351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.779660:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2eaff lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25440256->25444351] (req 25440256->25444351) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760750.839626:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25440256 -> 25444352 00010000:00010000:0.0:1789760750.839647:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2eaff lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25440256->25444351] (req 25440256->25444351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.839662:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2eaff lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25440256->25444351] (req 25440256->25444351) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.839672:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2eaff lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25440256->25444351] (req 25440256->25444351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.839680:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760750.839686:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2eaff lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25440256->25444351] (req 25440256->25444351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.839714:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2eaff lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25440256->25444351] (req 25440256->25444351) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.839735:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2eaff lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25440256->25444351] (req 25440256->25444351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.839743:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2eaff lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25440256->25444351] (req 25440256->25444351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760750.839854:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2747800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2360:x1876699512656384:12345-192.168.203.56@tcp:4:dd.0 Request processed in 60291us (60637us total) trans 4294973530 rc 0/0 00000100:00100000:0.0:1789760750.839865:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6241 00000100:00100000:0.0:1789760750.839874:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 730280186205 time 730343849359 state 0x2 ref 0 00000001:00080000:1.0:1789760750.840561:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973530 is committed 00000100:00100000:1.0:1789760750.846150:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760750.846330:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512656512 00000100:00100000:3.0:1789760750.846340:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512656512 00000100:00100000:3.0:1789760750.846405:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 730280186205 time 730343849359 state 0x0 ref 1 00000100:00100000:3.0:1789760750.846420:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 730344109775 time 730343849359 state 0x1 ref 1 00000100:00100000:3.0:1789760750.846539:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 730344109775 time 730410836891 state 0x0 ref 1 00000100:00100000:3.0:1789760750.846553:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4059180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512656512:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760750.846614:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25444352, mtime: 0x6aad94ee, atime: 0x0, ctime: 0x6aad94ee, blocks: 0xc807 00010000:00010000:3.0:1789760750.846624:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2eb06 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.846641:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=730s v=(0 0) 00010000:00010000:3.0:1789760750.846646:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760750.846653:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2eb06 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25444352->25448447] (req 25444352->25448447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.846667:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2eb06 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25444352->25448447] (req 25444352->25448447) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760750.884387:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25444352 -> 25448448 00010000:00010000:1.0:1789760750.884407:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2eb06 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25444352->25448447] (req 25444352->25448447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760750.884421:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2eb06 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25444352->25448447] (req 25444352->25448447) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760750.884432:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2eb06 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25444352->25448447] (req 25444352->25448447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760750.884440:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760750.884445:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2eb06 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25444352->25448447] (req 25444352->25448447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760750.884454:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2eb06 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25444352->25448447] (req 25444352->25448447) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760750.884476:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2eb06 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25444352->25448447] (req 25444352->25448447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760750.884489:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2eb06 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25444352->25448447] (req 25444352->25448447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760750.884592:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4059180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2361:x1876699512656512:12345-192.168.203.56@tcp:4:dd.0 Request processed in 38033us (38442us total) trans 4294973531 rc 0/0 00000100:00100000:1.0:1789760750.884603:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6242 00000100:00100000:1.0:1789760750.884614:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 730344109775 time 730410836891 state 0x2 ref 0 00000001:00080000:1.0:1789760750.886888:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973531 is committed 00000100:00100000:2.0:1789760750.896565:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760750.896645:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512656640 00000100:00100000:3.0:1789760750.896666:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512656640 00000100:00100000:3.0:1789760750.896755:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 730344109775 time 730410836891 state 0x0 ref 1 00000100:00100000:3.0:1789760750.896770:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 730411097307 time 730410836891 state 0x1 ref 1 00000100:00100000:3.0:1789760750.896804:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 730411097307 time 730461104637 state 0x0 ref 1 00000100:00100000:3.0:1789760750.896818:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febea651500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512656640:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760750.896867:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25448448, mtime: 0x6aad94ee, atime: 0x0, ctime: 0x6aad94ee, blocks: 0xc807 00010000:00010000:3.0:1789760750.896877:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2eb0d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.896893:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=730s v=(0 0) 00010000:00010000:3.0:1789760750.896897:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760750.896904:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2eb0d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25448448->25452543] (req 25448448->25452543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.896918:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2eb0d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25448448->25452543] (req 25448448->25452543) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760750.963560:0:5836:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:1.0:1789760750.974471:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25448448 -> 25452544 00010000:00010000:1.0:1789760750.974492:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2eb0d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25448448->25452543] (req 25448448->25452543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760750.974505:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2eb0d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25448448->25452543] (req 25448448->25452543) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760750.974515:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2eb0d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25448448->25452543] (req 25448448->25452543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760750.974522:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760750.974526:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2eb0d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25448448->25452543] (req 25448448->25452543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760750.974536:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2eb0d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25448448->25452543] (req 25448448->25452543) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760750.974556:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2eb0d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25448448->25452543] (req 25448448->25452543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760750.974565:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2eb0d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25448448->25452543] (req 25448448->25452543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760750.974663:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febea651500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2360:x1876699512656640:12345-192.168.203.56@tcp:4:dd.0 Request processed in 77843us (78103us total) trans 4294973532 rc 0/0 00000100:00100000:1.0:1789760750.974672:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6243 00000100:00100000:1.0:1789760750.974681:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 730411097307 time 730461104637 state 0x2 ref 0 00000001:00080000:1.0:1789760750.974785:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973532 is committed 00000100:00100000:0.0:1789760750.981177:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760750.983510:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512656768 00000100:00100000:2.0:1789760750.983525:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512656768 00000100:00100000:2.0:1789760750.983585:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 730411097307 time 730461104637 state 0x0 ref 1 00000100:00100000:2.0:1789760750.983600:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 730461365053 time 730461104637 state 0x1 ref 1 00000100:00100000:2.0:1789760750.983671:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 730461365053 time 730547970223 state 0x0 ref 1 00000100:00100000:2.0:1789760750.983683:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2745500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512656768:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760750.983745:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25452544, mtime: 0x6aad94ee, atime: 0x0, ctime: 0x6aad94ee, blocks: 0xc807 00010000:00010000:2.0:1789760750.983754:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2eb14 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.983771:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=730s v=(0 0) 00010000:00010000:2.0:1789760750.983776:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760750.983782:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2eb14 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25452544->25456639] (req 25452544->25456639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.983797:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2eb14 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25452544->25456639] (req 25452544->25456639) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760751.019797:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973533 is committed 00002000:00010000:1.0:1789760751.019962:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25452544 -> 25456640 00010000:00010000:1.0:1789760751.019977:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2eb14 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25452544->25456639] (req 25452544->25456639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.019989:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2eb14 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25452544->25456639] (req 25452544->25456639) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.019997:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2eb14 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25452544->25456639] (req 25452544->25456639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.020005:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760751.020009:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2eb14 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25452544->25456639] (req 25452544->25456639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.020025:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2eb14 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25452544->25456639] (req 25452544->25456639) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.020043:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2eb14 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25452544->25456639] (req 25452544->25456639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.020051:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2eb14 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25452544->25456639] (req 25452544->25456639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760751.020176:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2745500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512656768:12345-192.168.203.56@tcp:4:dd.0 Request processed in 36490us (39004us total) trans 4294973533 rc 0/0 00000100:00100000:1.0:1789760751.020186:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6244 00000100:00100000:1.0:1789760751.020196:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 730461365053 time 730547970223 state 0x2 ref 0 00000100:00100000:2.0:1789760751.031974:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760751.032107:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512656896 00000100:00100000:1.0:1789760751.032116:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512656896 00000100:00100000:1.0:1789760751.032183:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 730461365053 time 730547970223 state 0x0 ref 1 00000100:00100000:1.0:1789760751.032197:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 730548230639 time 730547970223 state 0x1 ref 1 00000100:00100000:1.0:1789760751.032226:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 730548230639 time 730596525778 state 0x0 ref 1 00000100:00100000:1.0:1789760751.032238:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febea650380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512656896:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760751.032298:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25456640, mtime: 0x6aad94ee, atime: 0x0, ctime: 0x6aad94ee, blocks: 0xc807 00010000:00010000:1.0:1789760751.032306:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2eb1b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.032323:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=730s v=(0 0) 00010000:00010000:1.0:1789760751.032328:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760751.032334:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2eb1b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25456640->25460735] (req 25456640->25460735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.032347:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2eb1b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25456640->25460735] (req 25456640->25460735) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1789760751.032847:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:1.0:1789760751.101010:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973534 is committed 00002000:00010000:3.0:1789760751.101259:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25456640 -> 25460736 00010000:00010000:3.0:1789760751.101287:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2eb1b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25456640->25460735] (req 25456640->25460735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.101300:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2eb1b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25456640->25460735] (req 25456640->25460735) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.101309:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2eb1b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25456640->25460735] (req 25456640->25460735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.101323:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760751.101327:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2eb1b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25456640->25460735] (req 25456640->25460735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.101337:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2eb1b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25456640->25460735] (req 25456640->25460735) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.101354:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2eb1b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25456640->25460735] (req 25456640->25460735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.101362:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2eb1b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25456640->25460735] (req 25456640->25460735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760751.101451:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febea650380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512656896:12345-192.168.203.56@tcp:4:dd.0 Request processed in 69211us (69482us total) trans 4294973534 rc 0/0 00000100:00100000:3.0:1789760751.101461:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6245 00000100:00100000:3.0:1789760751.101471:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 730548230639 time 730596525778 state 0x2 ref 0 00000100:00100000:0.0:1789760751.104485:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760751.104587:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512657024 00000100:00100000:1.0:1789760751.104597:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512657024 00000100:00100000:1.0:1789760751.104650:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 730548230639 time 730596525778 state 0x0 ref 1 00000100:00100000:1.0:1789760751.104676:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 730596786194 time 730596525778 state 0x1 ref 1 00000100:00100000:1.0:1789760751.104737:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 730596786194 time 730669036037 state 0x0 ref 1 00000100:00100000:1.0:1789760751.104751:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2746a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512657024:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760751.104807:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25460736, mtime: 0x6aad94ee, atime: 0x0, ctime: 0x6aad94ee, blocks: 0xc807 00010000:00010000:1.0:1789760751.104816:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2eb22 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.104832:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=730s v=(0 0) 00010000:00010000:1.0:1789760751.104837:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760751.104843:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2eb22 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25460736->25464831] (req 25460736->25464831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.104857:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2eb22 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25460736->25464831] (req 25460736->25464831) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760751.131748:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25460736 -> 25464832 00010000:00010000:3.0:1789760751.131771:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2eb22 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25460736->25464831] (req 25460736->25464831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.131784:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2eb22 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25460736->25464831] (req 25460736->25464831) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.131798:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2eb22 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25460736->25464831] (req 25460736->25464831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.131805:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760751.131810:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2eb22 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25460736->25464831] (req 25460736->25464831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.131818:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2eb22 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25460736->25464831] (req 25460736->25464831) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.131837:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2eb22 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25460736->25464831] (req 25460736->25464831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.131844:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2eb22 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25460736->25464831] (req 25460736->25464831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760751.131941:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2746a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2358:x1876699512657024:12345-192.168.203.56@tcp:4:dd.0 Request processed in 27180us (27453us total) trans 4294973535 rc 0/0 00000100:00100000:3.0:1789760751.131952:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6246 00000100:00100000:3.0:1789760751.131962:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 730596786194 time 730669036037 state 0x2 ref 0 00000001:00080000:1.0:1789760751.132439:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973535 is committed 00000100:00100000:2.0:1789760751.142145:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760751.142282:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512657152 00000100:00100000:1.0:1789760751.142291:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512657152 00000100:00100000:1.0:1789760751.142343:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 730596786194 time 730669036037 state 0x0 ref 1 00000100:00100000:1.0:1789760751.142358:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 730669296453 time 730669036037 state 0x1 ref 1 00000100:00100000:1.0:1789760751.142415:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 730669296453 time 730706714316 state 0x0 ref 1 00000100:00100000:1.0:1789760751.142428:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5305f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512657152:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760751.142551:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25464832, mtime: 0x6aad94ee, atime: 0x0, ctime: 0x6aad94ee, blocks: 0xc807 00010000:00010000:1.0:1789760751.142560:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2eb29 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.142575:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=730s v=(0 0) 00010000:00010000:1.0:1789760751.142580:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760751.142586:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2eb29 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25464832->25468927] (req 25464832->25468927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.142599:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2eb29 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25464832->25468927] (req 25464832->25468927) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760751.190191:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973536 is committed 00002000:00010000:3.0:1789760751.190370:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25464832 -> 25468928 00010000:00010000:3.0:1789760751.190384:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2eb29 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25464832->25468927] (req 25464832->25468927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.190402:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2eb29 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25464832->25468927] (req 25464832->25468927) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.190411:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2eb29 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25464832->25468927] (req 25464832->25468927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.190420:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760751.190424:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2eb29 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25464832->25468927] (req 25464832->25468927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.190433:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2eb29 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25464832->25468927] (req 25464832->25468927) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.190454:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2eb29 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25464832->25468927] (req 25464832->25468927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.190461:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2eb29 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25464832->25468927] (req 25464832->25468927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760751.190554:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5305f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512657152:12345-192.168.203.56@tcp:4:dd.0 Request processed in 48124us (48414us total) trans 4294973536 rc 0/0 00000100:00100000:3.0:1789760751.190564:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6247 00000100:00100000:3.0:1789760751.190574:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 730669296453 time 730706714316 state 0x2 ref 0 00000100:00100000:0.0:1789760751.193342:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760751.193497:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512657280 00000100:00100000:3.0:1789760751.193507:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512657280 00000100:00100000:3.0:1789760751.193563:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 730669296453 time 730706714316 state 0x0 ref 1 00000100:00100000:3.0:1789760751.193579:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 730706974732 time 730706714316 state 0x1 ref 1 00000100:00100000:3.0:1789760751.193614:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 730706974732 time 730757913592 state 0x0 ref 1 00000100:00100000:3.0:1789760751.193627:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2745880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512657280:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760751.193675:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25468928, mtime: 0x6aad94ee, atime: 0x0, ctime: 0x6aad94ee, blocks: 0xc807 00010000:00010000:3.0:1789760751.193684:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2eb30 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.193733:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=730s v=(0 0) 00010000:00010000:3.0:1789760751.193739:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760751.193746:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2eb30 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25468928->25473023] (req 25468928->25473023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.193760:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2eb30 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25468928->25473023] (req 25468928->25473023) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760751.259902:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973537 is committed 00002000:00010000:3.0:1789760751.260184:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25468928 -> 25473024 00010000:00010000:3.0:1789760751.260201:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2eb30 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25468928->25473023] (req 25468928->25473023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.260212:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2eb30 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25468928->25473023] (req 25468928->25473023) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.260224:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2eb30 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25468928->25473023] (req 25468928->25473023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.260231:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760751.260236:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2eb30 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25468928->25473023] (req 25468928->25473023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.260245:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2eb30 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25468928->25473023] (req 25468928->25473023) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.260273:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2eb30 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25468928->25473023] (req 25468928->25473023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.260281:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2eb30 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25468928->25473023] (req 25468928->25473023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760751.260376:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2745880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512657280:12345-192.168.203.56@tcp:4:dd.0 Request processed in 66746us (67037us total) trans 4294973537 rc 0/0 00000100:00100000:3.0:1789760751.260386:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6248 00000100:00100000:3.0:1789760751.260397:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 730706974732 time 730757913592 state 0x2 ref 0 00000100:00100000:0.0:1789760751.264750:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760751.264874:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512657408 00000100:00100000:3.0:1789760751.264884:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512657408 00000100:00100000:3.0:1789760751.264953:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 730706974732 time 730757913592 state 0x0 ref 1 00000100:00100000:3.0:1789760751.264969:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 730758174008 time 730757913592 state 0x1 ref 1 00000100:00100000:3.0:1789760751.264999:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 730758174008 time 730829299238 state 0x0 ref 1 00000100:00100000:3.0:1789760751.265011:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2745180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512657408:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760751.265061:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25473024, mtime: 0x6aad94ee, atime: 0x0, ctime: 0x6aad94ee, blocks: 0xc807 00010000:00010000:3.0:1789760751.265071:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2eb37 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.265085:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=730s v=(0 0) 00010000:00010000:3.0:1789760751.265090:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760751.265096:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2eb37 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25473024->25477119] (req 25473024->25477119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.265111:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2eb37 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25473024->25477119] (req 25473024->25477119) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760751.346053:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973538 is committed 00002000:00010000:3.0:1789760751.346167:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25473024 -> 25477120 00010000:00010000:3.0:1789760751.346183:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2eb37 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25473024->25477119] (req 25473024->25477119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.346197:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2eb37 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25473024->25477119] (req 25473024->25477119) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.346205:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2eb37 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25473024->25477119] (req 25473024->25477119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.346212:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760751.346218:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2eb37 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25473024->25477119] (req 25473024->25477119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.346227:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2eb37 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25473024->25477119] (req 25473024->25477119) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.346246:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2eb37 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25473024->25477119] (req 25473024->25477119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.346253:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2eb37 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25473024->25477119] (req 25473024->25477119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760751.346389:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2745180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512657408:12345-192.168.203.56@tcp:4:dd.0 Request processed in 81375us (81643us total) trans 4294973538 rc 0/0 00000100:00100000:3.0:1789760751.346400:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6249 00000100:00100000:3.0:1789760751.346410:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 730758174008 time 730829299238 state 0x2 ref 0 00000100:00100000:0.0:1789760751.351661:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760751.351803:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512657536 00000100:00100000:3.0:1789760751.351813:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512657536 00000100:00100000:3.0:1789760751.351864:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 730758174008 time 730829299238 state 0x0 ref 1 00000100:00100000:3.0:1789760751.351878:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 730829559654 time 730829299238 state 0x1 ref 1 00000100:00100000:3.0:1789760751.351911:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 730829559654 time 730916210421 state 0x0 ref 1 00000100:00100000:3.0:1789760751.351924:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2744a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512657536:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760751.351969:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25477120, mtime: 0x6aad94ee, atime: 0x0, ctime: 0x6aad94ee, blocks: 0xc807 00010000:00010000:3.0:1789760751.351978:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2eb3e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.351993:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=730s v=(0 0) 00010000:00010000:3.0:1789760751.351997:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760751.352003:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2eb3e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25477120->25481215] (req 25477120->25481215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.352016:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2eb3e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25477120->25481215] (req 25477120->25481215) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760751.438556:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25477120 -> 25481216 00010000:00010000:3.0:1789760751.438577:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2eb3e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25477120->25481215] (req 25477120->25481215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.438589:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2eb3e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25477120->25481215] (req 25477120->25481215) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.438598:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2eb3e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25477120->25481215] (req 25477120->25481215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.438606:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760751.438610:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2eb3e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25477120->25481215] (req 25477120->25481215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.438620:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2eb3e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25477120->25481215] (req 25477120->25481215) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760751.438796:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973539 is committed 00010000:00010000:3.0:1789760751.438901:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2eb3e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25477120->25481215] (req 25477120->25481215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.438912:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2eb3e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25477120->25481215] (req 25477120->25481215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760751.439098:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2744a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512657536:12345-192.168.203.56@tcp:4:dd.0 Request processed in 87172us (87440us total) trans 4294973539 rc 0/0 00000100:00100000:3.0:1789760751.439109:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6250 00000100:00100000:3.0:1789760751.439118:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 730829559654 time 730916210421 state 0x2 ref 0 00000100:00100000:0.0:1789760751.443607:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760751.445755:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512657664 00000100:00100000:3.0:1789760751.445771:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512657664 00000100:00100000:3.0:1789760751.445833:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 730829559654 time 730916210421 state 0x0 ref 1 00000100:00100000:3.0:1789760751.445851:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 730916470837 time 730916210421 state 0x1 ref 1 00000100:00100000:3.0:1789760751.445946:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 730916470837 time 731010244987 state 0x0 ref 1 00000100:00100000:3.0:1789760751.445961:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2746300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512657664:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760751.446010:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25481216, mtime: 0x6aad94ee, atime: 0x0, ctime: 0x6aad94ee, blocks: 0xc807 00010000:00010000:3.0:1789760751.446020:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2eb45 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.446035:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=731s v=(0 0) 00010000:00010000:3.0:1789760751.446040:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760751.446046:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2eb45 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25481216->25485311] (req 25481216->25485311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.446059:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2eb45 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25481216->25485311] (req 25481216->25485311) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760751.477015:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973540 is committed 00002000:00010000:3.0:1789760751.477174:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25481216 -> 25485312 00010000:00010000:3.0:1789760751.477188:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2eb45 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25481216->25485311] (req 25481216->25485311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.477199:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2eb45 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25481216->25485311] (req 25481216->25485311) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.477207:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2eb45 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25481216->25485311] (req 25481216->25485311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.477214:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760751.477218:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2eb45 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25481216->25485311] (req 25481216->25485311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.477226:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2eb45 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25481216->25485311] (req 25481216->25485311) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.477244:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2eb45 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25481216->25485311] (req 25481216->25485311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.477251:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2eb45 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25481216->25485311] (req 25481216->25485311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760751.477345:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2746300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512657664:12345-192.168.203.56@tcp:4:dd.0 Request processed in 31383us (33743us total) trans 4294973540 rc 0/0 00000100:00100000:3.0:1789760751.477354:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6251 00000100:00100000:3.0:1789760751.477363:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 730916470837 time 731010244987 state 0x2 ref 0 00000100:00100000:0.0:1789760751.480806:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760751.481235:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512657792 00000100:00100000:3.0:1789760751.481245:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512657792 00000100:00100000:3.0:1789760751.481309:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 730916470837 time 731010244987 state 0x0 ref 1 00000100:00100000:3.0:1789760751.481324:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 731010505403 time 731010244987 state 0x1 ref 1 00000100:00100000:3.0:1789760751.481355:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 731010505403 time 731045654585 state 0x0 ref 1 00000100:00100000:3.0:1789760751.481368:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febd02e0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512657792:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760751.481417:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25485312, mtime: 0x6aad94ee, atime: 0x0, ctime: 0x6aad94ee, blocks: 0xc807 00010000:00010000:3.0:1789760751.481426:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2eb4c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.481442:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=731s v=(0 0) 00010000:00010000:3.0:1789760751.481447:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760751.481453:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2eb4c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25485312->25489407] (req 25485312->25489407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.481467:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2eb4c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25485312->25489407] (req 25485312->25489407) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760751.534023:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973541 is committed 00002000:00010000:0.0:1789760751.534187:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25485312 -> 25489408 00010000:00010000:0.0:1789760751.534204:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2eb4c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25485312->25489407] (req 25485312->25489407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.534216:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2eb4c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25485312->25489407] (req 25485312->25489407) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.534226:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2eb4c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25485312->25489407] (req 25485312->25489407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.534238:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760751.534243:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2eb4c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25485312->25489407] (req 25485312->25489407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.534253:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2eb4c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25485312->25489407] (req 25485312->25489407) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.534284:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2eb4c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25485312->25489407] (req 25485312->25489407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.534300:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2eb4c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25485312->25489407] (req 25485312->25489407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760751.534897:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febd02e0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512657792:12345-192.168.203.56@tcp:4:dd.0 Request processed in 53525us (54094us total) trans 4294973541 rc 0/0 00000100:00100000:0.0:1789760751.534907:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6252 00000100:00100000:0.0:1789760751.534917:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 731010505403 time 731045654585 state 0x2 ref 0 00000100:00100000:0.0:1789760751.538514:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760751.538596:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512657920 00000100:00100000:3.0:1789760751.538606:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512657920 00000100:00100000:3.0:1789760751.538658:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 731010505403 time 731045654585 state 0x0 ref 1 00000100:00100000:3.0:1789760751.538673:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 731045915001 time 731045654585 state 0x1 ref 1 00000100:00100000:3.0:1789760751.538738:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 731045915001 time 731103037414 state 0x0 ref 1 00000100:00100000:3.0:1789760751.538751:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac3302680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512657920:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760751.538797:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25489408, mtime: 0x6aad94ee, atime: 0x0, ctime: 0x6aad94ee, blocks: 0xc807 00010000:00010000:3.0:1789760751.538806:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2eb53 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.538821:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=731s v=(0 0) 00000100:00100000:1.0:1789760751.538822:0:3310:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00010000:00010000:3.0:1789760751.538826:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760751.538831:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2eb53 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25489408->25493503] (req 25489408->25493503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760751.538837:0:3310:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8feac4059500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3310:1876699540225536:0@lo:13:osp-pre-0-0.0 00010000:00010000:3.0:1789760751.538845:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2eb53 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25489408->25493503] (req 25489408->25493503) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760751.538882:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760751.538934:0:6707:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540225536 00000100:00100000:1.0:1789760751.538938:0:6707:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540225536 00000100:00100000:1.0:1789760751.538974:0:6707:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 200 00000100:00100000:1.0:1789760751.538980:0:6707:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4058a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3310:x1876699540225536:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1789760751.538993:0:6707:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 527761408 pending 1703936 free 3832545280 avail 3811573760 00000100:00100000:1.0:1789760751.539033:0:6707:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4058a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3310:x1876699540225536:12345-0@lo:13:osp-pre-0-0.0 Request processed in 53us (150us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760751.539040:0:6707:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 200 00000100:00080000:1.0:1789760751.539065:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1789760751.539104:0:3310:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8feac4059500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3310:1876699540225536:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1789760751.546971:0:3310:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8feac3300e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:6df98d54-eb52-4c5c-8fc9-ee9d74f90f98:3310:1876699540225664:0@lo:400:kworker.0 00000100:00100000:0.0:1789760751.547055:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760751.547114:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540225664 00000100:00100000:0.0:1789760751.547121:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540225664 00000100:00100000:0.0:1789760751.547161:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 401 00000100:00100000:0.0:1789760751.547170:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febe0502c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:6df98d54-eb52-4c5c-8fc9-ee9d74f90f98+19:3310:x1876699540225664:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760751.547223:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febe0502c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:6df98d54-eb52-4c5c-8fc9-ee9d74f90f98+19:3310:x1876699540225664:12345-0@lo:400:kworker.0 Request processed in 53us (168us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760751.547230:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 401 00000100:00100000:0.0:1789760751.547482:0:3310:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8feac3300e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:6df98d54-eb52-4c5c-8fc9-ee9d74f90f98:3310:1876699540225664:0@lo:400:kworker.0 00000100:00100000:0.0:1789760751.547527:0:3311:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8feac3300e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3311:1876699540225792:0@lo:400:kworker.0 00000100:00100000:0.0:1789760751.547557:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760751.547591:0:8229:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540225792 00000100:00100000:0.0:1789760751.547595:0:8229:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540225792 00000100:00100000:0.0:1789760751.547622:0:8229:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 481 00000100:00100000:0.0:1789760751.547628:0:8229:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac3301c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3311:x1876699540225792:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760751.547663:0:8229:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac3301c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3311:x1876699540225792:12345-0@lo:400:kworker.0 Request processed in 35us (105us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760751.547669:0:8229:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 481 00000100:00100000:0.0:1789760751.547718:0:3311:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8feac3300e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3311:1876699540225792:0@lo:400:kworker.0 00000100:00100000:0.0:1789760751.547753:0:3312:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8feac3300e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3312:1876699540225920:0@lo:400:kworker.0 00000100:00100000:0.0:1789760751.547777:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760751.547805:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540225920 00000100:00100000:0.0:1789760751.547809:0:5847:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540225920 00000100:00100000:0.0:1789760751.547838:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 482 00000100:00100000:0.0:1789760751.547844:0:5847:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac38b3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3312:x1876699540225920:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760751.547875:0:5847:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac38b3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3312:x1876699540225920:12345-0@lo:400:kworker.0 Request processed in 32us (98us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760751.547881:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 482 00000100:00100000:0.0:1789760751.547909:0:3312:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8feac3300e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3312:1876699540225920:0@lo:400:kworker.0 00000100:00080000:0.0:1789760751.547925:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789760751.548114:0:3309:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8feac38b2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3309:1876699540226048:0@lo:400:kworker.0 00000100:00100000:0.0:1789760751.548141:0:3309:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760751.548168:0:8229:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540226048 00000100:00100000:0.0:1789760751.548171:0:8229:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540226048 00000100:00100000:0.0:1789760751.548196:0:8229:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 483 00000100:00100000:0.0:1789760751.548202:0:8229:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac38b1880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3309:x1876699540226048:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760751.548236:0:8229:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac38b1880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3309:x1876699540226048:12345-0@lo:400:kworker.0 Request processed in 34us (94us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760751.548242:0:8229:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 483 00000100:00100000:0.0:1789760751.548462:0:3309:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8feac38b2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3309:1876699540226048:0@lo:400:kworker.0 00000100:00080000:0.0:1789760751.548480:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789760751.548516:0:3309:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1789760751.548524:0:3309:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8feac38b2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3309:1876699540226176:0@lo:400:kworker.0 00000100:00100000:0.0:1789760751.548551:0:3309:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760751.548578:0:13064:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540226176 00000100:00100000:0.0:1789760751.548582:0:13064:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540226176 00000100:00100000:0.0:1789760751.548613:0:13064:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 184 00000100:00100000:0.0:1789760751.548619:0:13064:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac38b0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3309:x1876699540226176:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760751.548654:0:13064:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac38b0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3309:x1876699540226176:12345-0@lo:400:kworker.0 Request processed in 35us (102us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760751.548660:0:13064:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 184 00000100:00100000:0.0:1789760751.548688:0:3309:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8feac38b2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3309:1876699540226176:0@lo:400:kworker.0 00000100:00080000:0.0:1789760751.548718:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000001:00080000:1.0:1789760751.604215:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973542 is committed 00002000:00010000:0.0:1789760751.604361:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25489408 -> 25493504 00002000:00010000:0.0:1789760751.604366:0:6708:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760750 -> 1789760751 00002000:00010000:0.0:1789760751.604368:0:6708:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760750 -> 1789760751 00010000:00010000:0.0:1789760751.604380:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2eb53 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25489408->25493503] (req 25489408->25493503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.604392:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2eb53 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25489408->25493503] (req 25489408->25493503) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.604400:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2eb53 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25489408->25493503] (req 25489408->25493503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.604406:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760751.604411:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2eb53 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25489408->25493503] (req 25489408->25493503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.604420:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2eb53 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25489408->25493503] (req 25489408->25493503) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.604438:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2eb53 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25489408->25493503] (req 25489408->25493503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.604445:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2eb53 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25489408->25493503] (req 25489408->25493503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760751.604530:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac3302680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512657920:12345-192.168.203.56@tcp:4:dd.0 Request processed in 65777us (66022us total) trans 4294973542 rc 0/0 00000100:00100000:0.0:1789760751.604540:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6253 00000100:00100000:0.0:1789760751.604549:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 731045915001 time 731103037414 state 0x2 ref 0 00000100:00100000:1.0:1789760751.608092:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760751.608270:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512658048 00000100:00100000:0.0:1789760751.608279:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512658048 00000100:00100000:0.0:1789760751.608335:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 731045915001 time 731103037414 state 0x0 ref 1 00000100:00100000:0.0:1789760751.608348:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 731103297830 time 731103037414 state 0x1 ref 1 00000100:00100000:0.0:1789760751.608367:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 731103297830 time 731172667446 state 0x0 ref 1 00000100:00100000:0.0:1789760751.608378:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4059500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512658048:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760751.608421:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25493504, mtime: 0x6aad94ef, atime: 0x0, ctime: 0x6aad94ef, blocks: 0xc807 00010000:00010000:0.0:1789760751.608430:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2eb5a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.608446:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=731s v=(0 0) 00010000:00010000:0.0:1789760751.608450:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760751.608457:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2eb5a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25493504->25497599] (req 25493504->25497599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.608470:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2eb5a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25493504->25497599] (req 25493504->25497599) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760751.658308:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973543 is committed 00002000:00010000:2.0:1789760751.658576:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25493504 -> 25497600 00010000:00010000:2.0:1789760751.658591:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2eb5a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25493504->25497599] (req 25493504->25497599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.658605:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2eb5a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25493504->25497599] (req 25493504->25497599) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.658614:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2eb5a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25493504->25497599] (req 25493504->25497599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.658622:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760751.658627:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2eb5a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25493504->25497599] (req 25493504->25497599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.658636:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2eb5a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25493504->25497599] (req 25493504->25497599) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.658654:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2eb5a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25493504->25497599] (req 25493504->25497599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.658661:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2eb5a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25493504->25497599] (req 25493504->25497599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760751.658960:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4059500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512658048:12345-192.168.203.56@tcp:4:dd.0 Request processed in 50577us (50871us total) trans 4294973543 rc 0/0 00000100:00100000:3.0:1789760751.658970:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6254 00000100:00100000:3.0:1789760751.658979:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 731103297830 time 731172667446 state 0x2 ref 0 00000100:00100000:2.0:1789760751.661795:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760751.661867:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512658176 00000100:00100000:0.0:1789760751.661878:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512658176 00000100:00100000:0.0:1789760751.661930:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 731103297830 time 731172667446 state 0x0 ref 1 00000100:00100000:0.0:1789760751.661947:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 731172927862 time 731172667446 state 0x1 ref 1 00000100:00100000:0.0:1789760751.661980:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 731172927862 time 731226280080 state 0x0 ref 1 00000100:00100000:0.0:1789760751.661992:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5307480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512658176:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760751.662039:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25497600, mtime: 0x6aad94ef, atime: 0x0, ctime: 0x6aad94ef, blocks: 0xc807 00010000:00010000:0.0:1789760751.662049:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2eb61 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.662065:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=731s v=(0 0) 00010000:00010000:0.0:1789760751.662070:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760751.662075:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2eb61 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25497600->25501695] (req 25497600->25501695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.662088:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2eb61 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25497600->25501695] (req 25497600->25501695) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760751.700385:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973544 is committed 00002000:00010000:0.0:1789760751.700504:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25497600 -> 25501696 00010000:00010000:0.0:1789760751.700520:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2eb61 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25497600->25501695] (req 25497600->25501695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.700534:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2eb61 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25497600->25501695] (req 25497600->25501695) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.700542:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2eb61 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25497600->25501695] (req 25497600->25501695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.700550:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760751.700555:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2eb61 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25497600->25501695] (req 25497600->25501695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.700565:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2eb61 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25497600->25501695] (req 25497600->25501695) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.700583:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2eb61 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25497600->25501695] (req 25497600->25501695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.700591:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2eb61 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25497600->25501695] (req 25497600->25501695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760751.700680:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5307480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512658176:12345-192.168.203.56@tcp:4:dd.0 Request processed in 38685us (38889us total) trans 4294973544 rc 0/0 00000100:00100000:0.0:1789760751.700690:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6255 00000100:00100000:0.0:1789760751.700729:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 731172927862 time 731226280080 state 0x2 ref 0 00000100:00100000:0.0:1789760751.705346:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760751.705461:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512658304 00000100:00100000:3.0:1789760751.705471:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512658304 00000100:00100000:3.0:1789760751.705523:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 731172927862 time 731226280080 state 0x0 ref 1 00000100:00100000:3.0:1789760751.705538:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 731226540496 time 731226280080 state 0x1 ref 1 00000100:00100000:3.0:1789760751.705567:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 731226540496 time 731269867518 state 0x0 ref 1 00000100:00100000:3.0:1789760751.705580:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac38b2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512658304:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760751.705628:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25501696, mtime: 0x6aad94ef, atime: 0x0, ctime: 0x6aad94ef, blocks: 0xc807 00010000:00010000:3.0:1789760751.705638:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2eb68 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.705653:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=731s v=(0 0) 00010000:00010000:3.0:1789760751.705658:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760751.705664:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2eb68 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25501696->25505791] (req 25501696->25505791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.705678:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2eb68 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25501696->25505791] (req 25501696->25505791) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760751.743878:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973545 is committed 00002000:00010000:3.0:1789760751.743949:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25501696 -> 25505792 00010000:00010000:3.0:1789760751.743965:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2eb68 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25501696->25505791] (req 25501696->25505791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.743985:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2eb68 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25501696->25505791] (req 25501696->25505791) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.743994:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2eb68 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25501696->25505791] (req 25501696->25505791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.744002:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760751.744007:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2eb68 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25501696->25505791] (req 25501696->25505791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.744018:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2eb68 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25501696->25505791] (req 25501696->25505791) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.744037:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2eb68 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25501696->25505791] (req 25501696->25505791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.744045:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2eb68 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25501696->25505791] (req 25501696->25505791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760751.744175:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac38b2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512658304:12345-192.168.203.56@tcp:4:dd.0 Request processed in 38593us (38835us total) trans 4294973545 rc 0/0 00000100:00100000:3.0:1789760751.744186:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6256 00000100:00100000:3.0:1789760751.744195:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 731226540496 time 731269867518 state 0x2 ref 0 00000100:00100000:3.0:1789760751.747126:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760751.747297:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512658432 00000100:00100000:0.0:1789760751.747307:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512658432 00000100:00100000:0.0:1789760751.747358:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 731226540496 time 731269867518 state 0x0 ref 1 00000100:00100000:0.0:1789760751.747373:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 731270127934 time 731269867518 state 0x1 ref 1 00000100:00100000:0.0:1789760751.747444:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 731270127934 time 731311743535 state 0x0 ref 1 00000100:00100000:0.0:1789760751.747456:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac287a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512658432:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760751.747508:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25505792, mtime: 0x6aad94ef, atime: 0x0, ctime: 0x6aad94ef, blocks: 0xc807 00010000:00010000:0.0:1789760751.747516:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2eb6f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.747532:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=731s v=(0 0) 00010000:00010000:0.0:1789760751.747537:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760751.747544:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2eb6f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25505792->25509887] (req 25505792->25509887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.747557:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2eb6f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25505792->25509887] (req 25505792->25509887) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760751.781070:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973546 is committed 00002000:00010000:0.0:1789760751.781223:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25505792 -> 25509888 00010000:00010000:0.0:1789760751.781239:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2eb6f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25505792->25509887] (req 25505792->25509887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.781252:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2eb6f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25505792->25509887] (req 25505792->25509887) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.781272:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2eb6f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25505792->25509887] (req 25505792->25509887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.781279:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760751.781284:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2eb6f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25505792->25509887] (req 25505792->25509887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.781293:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2eb6f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25505792->25509887] (req 25505792->25509887) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.781313:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2eb6f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25505792->25509887] (req 25505792->25509887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.781321:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2eb6f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25505792->25509887] (req 25505792->25509887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760751.781412:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac287a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512658432:12345-192.168.203.56@tcp:4:dd.0 Request processed in 33952us (34292us total) trans 4294973546 rc 0/0 00000100:00100000:0.0:1789760751.781422:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6257 00000100:00100000:0.0:1789760751.781432:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 731270127934 time 731311743535 state 0x2 ref 0 00000100:00100000:2.0:1789760751.787159:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760751.787243:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512658560 00000100:00100000:0.0:1789760751.787254:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512658560 00000100:00100000:0.0:1789760751.787317:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 731270127934 time 731311743535 state 0x0 ref 1 00000100:00100000:0.0:1789760751.787332:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 731312003951 time 731311743535 state 0x1 ref 1 00000100:00100000:0.0:1789760751.787364:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 731312003951 time 731351663693 state 0x0 ref 1 00000100:00100000:0.0:1789760751.787376:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5304700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512658560:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760751.787424:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25509888, mtime: 0x6aad94ef, atime: 0x0, ctime: 0x6aad94ef, blocks: 0xc807 00010000:00010000:0.0:1789760751.787432:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2eb76 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.787448:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=731s v=(0 0) 00010000:00010000:0.0:1789760751.787454:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760751.787461:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2eb76 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25509888->25513983] (req 25509888->25513983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.787474:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2eb76 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25509888->25513983] (req 25509888->25513983) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760751.844335:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973547 is committed 00002000:00010000:0.0:1789760751.844470:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25509888 -> 25513984 00010000:00010000:0.0:1789760751.844486:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2eb76 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25509888->25513983] (req 25509888->25513983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.844498:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2eb76 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25509888->25513983] (req 25509888->25513983) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.844507:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2eb76 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25509888->25513983] (req 25509888->25513983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.844514:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760751.844520:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2eb76 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25509888->25513983] (req 25509888->25513983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.844528:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2eb76 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25509888->25513983] (req 25509888->25513983) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.844549:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2eb76 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25509888->25513983] (req 25509888->25513983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.844567:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2eb76 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25509888->25513983] (req 25509888->25513983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760751.844663:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5304700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512658560:12345-192.168.203.56@tcp:4:dd.0 Request processed in 57284us (57509us total) trans 4294973547 rc 0/0 00000100:00100000:0.0:1789760751.844673:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6258 00000100:00100000:0.0:1789760751.844683:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 731312003951 time 731351663693 state 0x2 ref 0 00000100:00100000:3.0:1789760751.847636:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760751.847746:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512658688 00000100:00100000:0.0:1789760751.847756:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512658688 00000100:00100000:0.0:1789760751.847810:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 731312003951 time 731351663693 state 0x0 ref 1 00000100:00100000:0.0:1789760751.847826:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 731351924109 time 731351663693 state 0x1 ref 1 00000100:00100000:0.0:1789760751.847853:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 731351924109 time 731412153300 state 0x0 ref 1 00000100:00100000:0.0:1789760751.847866:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2879c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512658688:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760751.847912:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25513984, mtime: 0x6aad94ef, atime: 0x0, ctime: 0x6aad94ef, blocks: 0xc807 00010000:00010000:0.0:1789760751.847920:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2eb7d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.847950:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=731s v=(0 0) 00010000:00010000:0.0:1789760751.847956:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760751.847962:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2eb7d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25513984->25518079] (req 25513984->25518079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.847983:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2eb7d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25513984->25518079] (req 25513984->25518079) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760751.887566:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973548 is committed 00002000:00010000:3.0:1789760751.888826:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25513984 -> 25518080 00010000:00010000:3.0:1789760751.888844:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2eb7d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25513984->25518079] (req 25513984->25518079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.888858:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2eb7d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25513984->25518079] (req 25513984->25518079) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.888868:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2eb7d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25513984->25518079] (req 25513984->25518079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.888876:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760751.888881:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2eb7d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25513984->25518079] (req 25513984->25518079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.888890:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2eb7d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25513984->25518079] (req 25513984->25518079) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.888910:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2eb7d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25513984->25518079] (req 25513984->25518079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.888919:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2eb7d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25513984->25518079] (req 25513984->25518079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760751.889122:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2879c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512658688:12345-192.168.203.56@tcp:4:dd.0 Request processed in 41253us (41490us total) trans 4294973548 rc 0/0 00000100:00100000:3.0:1789760751.889136:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6259 00000100:00100000:3.0:1789760751.889145:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 731351924109 time 731412153300 state 0x2 ref 0 00000100:00100000:1.0:1789760751.894281:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760751.894356:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512658816 00000100:00100000:0.0:1789760751.894366:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512658816 00000100:00100000:0.0:1789760751.894418:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 731351924109 time 731412153300 state 0x0 ref 1 00000100:00100000:0.0:1789760751.894434:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 731412413716 time 731412153300 state 0x1 ref 1 00000100:00100000:0.0:1789760751.894463:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 731412413716 time 731458763285 state 0x0 ref 1 00000100:00100000:0.0:1789760751.894476:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac405ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512658816:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760751.894523:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25518080, mtime: 0x6aad94ef, atime: 0x0, ctime: 0x6aad94ef, blocks: 0xc807 00010000:00010000:0.0:1789760751.894532:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2eb84 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.894548:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=731s v=(0 0) 00010000:00010000:0.0:1789760751.894553:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760751.894559:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2eb84 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25518080->25522175] (req 25518080->25522175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.894573:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2eb84 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25518080->25522175] (req 25518080->25522175) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760751.929246:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973549 is committed 00002000:00010000:0.0:1789760751.929342:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25518080 -> 25522176 00010000:00010000:0.0:1789760751.929357:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2eb84 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25518080->25522175] (req 25518080->25522175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.929370:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2eb84 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25518080->25522175] (req 25518080->25522175) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.929378:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2eb84 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25518080->25522175] (req 25518080->25522175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.929384:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760751.929389:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2eb84 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25518080->25522175] (req 25518080->25522175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.929396:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2eb84 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25518080->25522175] (req 25518080->25522175) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.929414:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2eb84 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25518080->25522175] (req 25518080->25522175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.929420:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2eb84 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25518080->25522175] (req 25518080->25522175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760751.929509:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac405ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512658816:12345-192.168.203.56@tcp:4:dd.0 Request processed in 35031us (35234us total) trans 4294973549 rc 0/0 00000100:00100000:0.0:1789760751.929519:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6260 00000100:00100000:0.0:1789760751.929528:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 731412413716 time 731458763285 state 0x2 ref 0 00000100:00100000:3.0:1789760751.932126:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760751.932566:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512658944 00000100:00100000:0.0:1789760751.932577:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512658944 00000100:00100000:0.0:1789760751.932638:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 731412413716 time 731458763285 state 0x0 ref 1 00000100:00100000:0.0:1789760751.932741:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 731459023701 time 731458763285 state 0x1 ref 1 00000100:00100000:0.0:1789760751.932824:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 731459023701 time 731497122816 state 0x0 ref 1 00000100:00100000:0.0:1789760751.932837:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2878e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512658944:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760751.932885:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25522176, mtime: 0x6aad94ef, atime: 0x0, ctime: 0x6aad94ef, blocks: 0xc807 00010000:00010000:0.0:1789760751.932894:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2eb8b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.932909:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=731s v=(0 0) 00010000:00010000:0.0:1789760751.932913:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760751.932919:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2eb8b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25522176->25526271] (req 25522176->25526271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.932994:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2eb8b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25522176->25526271] (req 25522176->25526271) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760751.979330:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973550 is committed 00002000:00010000:0.0:1789760751.979785:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25522176 -> 25526272 00010000:00010000:0.0:1789760751.979800:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2eb8b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25522176->25526271] (req 25522176->25526271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.979814:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2eb8b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25522176->25526271] (req 25522176->25526271) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.979824:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2eb8b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25522176->25526271] (req 25522176->25526271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.979831:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760751.979836:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2eb8b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25522176->25526271] (req 25522176->25526271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.979846:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2eb8b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25522176->25526271] (req 25522176->25526271) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.979866:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2eb8b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25522176->25526271] (req 25522176->25526271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.979874:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2eb8b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25522176->25526271] (req 25522176->25526271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760751.980070:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2878e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512658944:12345-192.168.203.56@tcp:4:dd.0 Request processed in 47230us (47949us total) trans 4294973550 rc 0/0 00000100:00100000:0.0:1789760751.980080:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6261 00000100:00100000:0.0:1789760751.980090:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 731459023701 time 731497122816 state 0x2 ref 0 00000100:00100000:3.0:1789760751.984375:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760751.984452:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512659072 00000100:00100000:0.0:1789760751.984462:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512659072 00000100:00100000:0.0:1789760751.984516:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 731459023701 time 731497122816 state 0x0 ref 1 00000100:00100000:0.0:1789760751.984532:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 731497383232 time 731497122816 state 0x1 ref 1 00000100:00100000:0.0:1789760751.984562:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 731497383232 time 731548861751 state 0x0 ref 1 00000100:00100000:0.0:1789760751.984573:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac287b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512659072:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760751.984620:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25526272, mtime: 0x6aad94ef, atime: 0x0, ctime: 0x6aad94ef, blocks: 0xc807 00010000:00010000:0.0:1789760751.984628:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2eb92 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.984643:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=731s v=(0 0) 00010000:00010000:0.0:1789760751.984647:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760751.984664:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2eb92 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25526272->25530367] (req 25526272->25530367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.984679:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2eb92 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25526272->25530367] (req 25526272->25530367) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.987961:0:5835:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:2.0:1789760752.023826:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973551 is committed 00002000:00010000:1.0:1789760752.023940:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25526272 -> 25530368 00010000:00010000:1.0:1789760752.023957:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2eb92 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25526272->25530367] (req 25526272->25530367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.023972:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2eb92 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25526272->25530367] (req 25526272->25530367) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.023981:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2eb92 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25526272->25530367] (req 25526272->25530367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.023989:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760752.023994:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2eb92 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25526272->25530367] (req 25526272->25530367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.024003:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2eb92 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25526272->25530367] (req 25526272->25530367) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.024022:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2eb92 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25526272->25530367] (req 25526272->25530367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.024031:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2eb92 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25526272->25530367] (req 25526272->25530367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760752.024130:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac287b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512659072:12345-192.168.203.56@tcp:4:dd.0 Request processed in 39553us (39759us total) trans 4294973551 rc 0/0 00000100:00100000:1.0:1789760752.024141:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6262 00000100:00100000:1.0:1789760752.024151:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 731497383232 time 731548861751 state 0x2 ref 0 00000100:00100000:1.0:1789760752.026833:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760752.026901:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512659200 00000100:00100000:0.0:1789760752.026909:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512659200 00000100:00100000:0.0:1789760752.026964:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 731497383232 time 731548861751 state 0x0 ref 1 00000100:00100000:0.0:1789760752.026980:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 731549122167 time 731548861751 state 0x1 ref 1 00000100:00100000:0.0:1789760752.027010:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 731549122167 time 731591309944 state 0x0 ref 1 00000100:00100000:0.0:1789760752.027023:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4058e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512659200:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760752.027079:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25530368, mtime: 0x6aad94ef, atime: 0x0, ctime: 0x6aad94ef, blocks: 0xc807 00010000:00010000:0.0:1789760752.027088:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2eb99 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.027105:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=731s v=(0 0) 00010000:00010000:0.0:1789760752.027109:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760752.027114:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2eb99 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25530368->25534463] (req 25530368->25534463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.027127:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2eb99 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25530368->25534463] (req 25530368->25534463) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760752.051487:0:3312:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1789760752.051509:0:3312:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8feac5306a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3312:1876699540226304:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1789760752.051574:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760752.051795:0:6707:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540226304 00000100:00100000:2.0:1789760752.051803:0:6707:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540226304 00000100:00100000:2.0:1789760752.051852:0:6707:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 201 00000100:00100000:2.0:1789760752.051861:0:6707:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2878380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3312:x1876699540226304:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1789760752.051894:0:6707:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3859808256 avail 3857711104 00000100:00100000:2.0:1789760752.051937:0:6707:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2878380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3312:x1876699540226304:12345-0@lo:13:osp-pre-1-0.0 Request processed in 77us (362us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760752.051944:0:6707:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 201 00000100:00100000:3.0:1789760752.052070:0:3312:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8feac5306a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3312:1876699540226304:0@lo:13:osp-pre-1-0.0 00000001:00080000:1.0:1789760752.060128:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973552 is committed 00002000:00010000:2.0:1789760752.060245:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25530368 -> 25534464 00010000:00010000:2.0:1789760752.060271:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2eb99 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25530368->25534463] (req 25530368->25534463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.060284:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2eb99 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25530368->25534463] (req 25530368->25534463) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.060293:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2eb99 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25530368->25534463] (req 25530368->25534463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.060300:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760752.060305:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2eb99 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25530368->25534463] (req 25530368->25534463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.060314:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2eb99 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25530368->25534463] (req 25530368->25534463) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.060331:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2eb99 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25530368->25534463] (req 25530368->25534463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.060344:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2eb99 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25530368->25534463] (req 25530368->25534463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760752.060439:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4058e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512659200:12345-192.168.203.56@tcp:4:dd.0 Request processed in 33414us (33611us total) trans 4294973552 rc 0/0 00000100:00100000:2.0:1789760752.060450:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6263 00000100:00100000:2.0:1789760752.060460:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 731549122167 time 731591309944 state 0x2 ref 0 00000100:00100000:3.0:1789760752.065198:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760752.065392:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512659328 00000100:00100000:0.0:1789760752.065402:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512659328 00000100:00100000:0.0:1789760752.065455:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 731549122167 time 731591309944 state 0x0 ref 1 00000100:00100000:0.0:1789760752.065472:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 731591570360 time 731591309944 state 0x1 ref 1 00000100:00100000:0.0:1789760752.065504:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 731591570360 time 731629803161 state 0x0 ref 1 00000100:00100000:0.0:1789760752.065517:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2878000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512659328:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760752.065569:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25534464, mtime: 0x6aad94ef, atime: 0x0, ctime: 0x6aad94ef, blocks: 0xc807 00010000:00010000:0.0:1789760752.065578:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2eba0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.065593:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=731s v=(0 0) 00010000:00010000:0.0:1789760752.065608:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760752.065615:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2eba0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25534464->25538559] (req 25534464->25538559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.065631:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2eba0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25534464->25538559] (req 25534464->25538559) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760752.101670:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973553 is committed 00002000:00010000:1.0:1789760752.102298:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25534464 -> 25538560 00010000:00010000:1.0:1789760752.102315:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2eba0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25534464->25538559] (req 25534464->25538559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.102329:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2eba0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25534464->25538559] (req 25534464->25538559) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.102339:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2eba0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25534464->25538559] (req 25534464->25538559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.102347:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760752.102352:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2eba0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25534464->25538559] (req 25534464->25538559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.102359:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2eba0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25534464->25538559] (req 25534464->25538559) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.102374:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2eba0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25534464->25538559] (req 25534464->25538559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.102378:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2eba0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25534464->25538559] (req 25534464->25538559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760752.102493:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2878000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512659328:12345-192.168.203.56@tcp:4:dd.0 Request processed in 36976us (37302us total) trans 4294973553 rc 0/0 00000100:00100000:1.0:1789760752.102501:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6264 00000100:00100000:1.0:1789760752.102508:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 731591570360 time 731629803161 state 0x2 ref 0 00000100:00100000:1.0:1789760752.105968:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760752.106048:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512659456 00000100:00100000:0.0:1789760752.106058:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512659456 00000100:00100000:0.0:1789760752.106113:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 731591570360 time 731629803161 state 0x0 ref 1 00000100:00100000:0.0:1789760752.106127:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 731630063577 time 731629803161 state 0x1 ref 1 00000100:00100000:0.0:1789760752.106157:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 731630063577 time 731670457032 state 0x0 ref 1 00000100:00100000:0.0:1789760752.106169:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febfb89ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512659456:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760752.106221:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25538560, mtime: 0x6aad94ef, atime: 0x0, ctime: 0x6aad94ef, blocks: 0xc807 00010000:00010000:0.0:1789760752.106230:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2eba7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.106246:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=731s v=(0 0) 00010000:00010000:0.0:1789760752.106250:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760752.106266:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2eba7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25538560->25542655] (req 25538560->25542655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.106281:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2eba7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25538560->25542655] (req 25538560->25542655) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760752.163943:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25538560 -> 25542656 00010000:00010000:1.0:1789760752.163963:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2eba7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25538560->25542655] (req 25538560->25542655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.163978:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2eba7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25538560->25542655] (req 25538560->25542655) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.163987:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2eba7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25538560->25542655] (req 25538560->25542655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.163994:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760752.163999:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2eba7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25538560->25542655] (req 25538560->25542655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.164008:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2eba7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25538560->25542655] (req 25538560->25542655) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.164027:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2eba7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25538560->25542655] (req 25538560->25542655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.164035:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2eba7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25538560->25542655] (req 25538560->25542655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760752.164128:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febfb89ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2358:x1876699512659456:12345-192.168.203.56@tcp:4:dd.0 Request processed in 57955us (58165us total) trans 4294973554 rc 0/0 00000100:00100000:1.0:1789760752.164139:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6265 00000100:00100000:1.0:1789760752.164149:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 731630063577 time 731670457032 state 0x2 ref 0 00000001:00080000:2.0:1789760752.169924:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973554 is committed 00000100:00100000:0.0:1789760752.173342:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760752.173452:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512659584 00000100:00100000:2.0:1789760752.173462:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512659584 00000100:00100000:2.0:1789760752.173584:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 731630063577 time 731670457032 state 0x0 ref 1 00000100:00100000:2.0:1789760752.173599:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 731670717448 time 731670457032 state 0x1 ref 1 00000100:00100000:2.0:1789760752.174061:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 731670717448 time 731738358234 state 0x0 ref 1 00000100:00100000:2.0:1789760752.174078:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac3300e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512659584:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760752.174129:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25542656, mtime: 0x6aad94ef, atime: 0x0, ctime: 0x6aad94ef, blocks: 0xc807 00010000:00010000:2.0:1789760752.174138:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2ebae lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.174155:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=731s v=(0 0) 00010000:00010000:2.0:1789760752.174159:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760752.174167:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2ebae lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25542656->25546751] (req 25542656->25546751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.174181:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2ebae lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25542656->25546751] (req 25542656->25546751) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760752.246459:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973555 is committed 00002000:00010000:1.0:1789760752.246541:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25542656 -> 25546752 00010000:00010000:1.0:1789760752.246557:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2ebae lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25542656->25546751] (req 25542656->25546751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.246571:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2ebae lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25542656->25546751] (req 25542656->25546751) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.246580:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2ebae lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25542656->25546751] (req 25542656->25546751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.246587:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760752.246593:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2ebae lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25542656->25546751] (req 25542656->25546751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.246603:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2ebae lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25542656->25546751] (req 25542656->25546751) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.246621:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2ebae lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25542656->25546751] (req 25542656->25546751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.246630:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2ebae lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25542656->25546751] (req 25542656->25546751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760752.246863:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac3300e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512659584:12345-192.168.203.56@tcp:4:dd.0 Request processed in 72784us (73524us total) trans 4294973555 rc 0/0 00000100:00100000:1.0:1789760752.246874:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6266 00000100:00100000:1.0:1789760752.246884:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 731670717448 time 731738358234 state 0x2 ref 0 00000100:00100000:2.0:1789760752.249892:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760752.251177:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512659712 00000100:00100000:1.0:1789760752.251191:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512659712 00000100:00100000:1.0:1789760752.251263:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 731670717448 time 731738358234 state 0x0 ref 1 00000100:00100000:1.0:1789760752.251279:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 731738618650 time 731738358234 state 0x1 ref 1 00000100:00100000:1.0:1789760752.251309:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 731738618650 time 731815609166 state 0x0 ref 1 00000100:00100000:1.0:1789760752.251322:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5304380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512659712:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760752.251374:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25546752, mtime: 0x6aad94ef, atime: 0x0, ctime: 0x6aad94ef, blocks: 0xc807 00010000:00010000:1.0:1789760752.252832:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2ebb5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.252860:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=731s v=(0 0) 00010000:00010000:1.0:1789760752.252864:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760752.252873:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2ebb5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25546752->25550847] (req 25546752->25550847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.252892:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2ebb5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25546752->25550847] (req 25546752->25550847) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760752.296544:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25546752 -> 25550848 00010000:00010000:1.0:1789760752.296564:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2ebb5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25546752->25550847] (req 25546752->25550847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.296583:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2ebb5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25546752->25550847] (req 25546752->25550847) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.296592:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2ebb5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25546752->25550847] (req 25546752->25550847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.296598:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760752.296603:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2ebb5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25546752->25550847] (req 25546752->25550847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.296611:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2ebb5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25546752->25550847] (req 25546752->25550847) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.296632:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2ebb5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25546752->25550847] (req 25546752->25550847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.296639:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2ebb5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25546752->25550847] (req 25546752->25550847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760752.296767:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5304380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2358:x1876699512659712:12345-192.168.203.56@tcp:4:dd.0 Request processed in 45442us (46878us total) trans 4294973556 rc 0/0 00000100:00100000:1.0:1789760752.296778:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6267 00000100:00100000:1.0:1789760752.296786:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 731738618650 time 731815609166 state 0x2 ref 0 00000001:00080000:2.0:1789760752.296962:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973556 is committed 00000100:00100000:3.0:1789760752.314658:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760752.314833:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512659840 00000100:00100000:1.0:1789760752.314844:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512659840 00000100:00100000:1.0:1789760752.314896:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 731738618650 time 731815609166 state 0x0 ref 1 00000100:00100000:1.0:1789760752.314912:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 731815869582 time 731815609166 state 0x1 ref 1 00000100:00100000:1.0:1789760752.314945:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 731815869582 time 731879244367 state 0x0 ref 1 00000100:00100000:1.0:1789760752.314957:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac287b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512659840:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760752.315008:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25550848, mtime: 0x6aad94ef, atime: 0x0, ctime: 0x6aad94ef, blocks: 0xc807 00010000:00010000:1.0:1789760752.315018:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2ebbc lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.315033:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=731s v=(0 0) 00010000:00010000:1.0:1789760752.315038:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760752.315044:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2ebbc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25550848->25554943] (req 25550848->25554943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.315059:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2ebbc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25550848->25554943] (req 25550848->25554943) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760752.391476:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973557 is committed 00002000:00010000:2.0:1789760752.391762:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25550848 -> 25554944 00010000:00010000:2.0:1789760752.391778:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2ebbc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25550848->25554943] (req 25550848->25554943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.391790:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2ebbc lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25550848->25554943] (req 25550848->25554943) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.391799:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2ebbc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25550848->25554943] (req 25550848->25554943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.391807:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760752.391812:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2ebbc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25550848->25554943] (req 25550848->25554943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.391821:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2ebbc lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25550848->25554943] (req 25550848->25554943) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.391841:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2ebbc lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25550848->25554943] (req 25550848->25554943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.391848:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2ebbc lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25550848->25554943] (req 25550848->25554943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760752.391940:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac287b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512659840:12345-192.168.203.56@tcp:4:dd.0 Request processed in 76970us (77279us total) trans 4294973557 rc 0/0 00000100:00100000:2.0:1789760752.391950:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6268 00000100:00100000:2.0:1789760752.391960:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 731815869582 time 731879244367 state 0x2 ref 0 00000100:00100000:0.0:1789760752.395911:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760752.396012:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512659968 00000100:00100000:1.0:1789760752.396022:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512659968 00000100:00100000:1.0:1789760752.396080:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 731815869582 time 731879244367 state 0x0 ref 1 00000100:00100000:1.0:1789760752.396095:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 731879504783 time 731879244367 state 0x1 ref 1 00000100:00100000:1.0:1789760752.396129:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 731879504783 time 731960429121 state 0x0 ref 1 00000100:00100000:1.0:1789760752.396144:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febfb374380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512659968:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760752.396194:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25554944, mtime: 0x6aad94ef, atime: 0x0, ctime: 0x6aad94ef, blocks: 0xc807 00010000:00010000:1.0:1789760752.396204:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2ebc3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.396222:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=731s v=(0 0) 00010000:00010000:1.0:1789760752.396227:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760752.396235:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2ebc3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25554944->25559039] (req 25554944->25559039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.396250:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2ebc3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25554944->25559039] (req 25554944->25559039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760752.455895:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25554944 -> 25559040 00010000:00010000:1.0:1789760752.455927:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2ebc3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25554944->25559039] (req 25554944->25559039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.455941:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2ebc3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25554944->25559039] (req 25554944->25559039) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.455950:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2ebc3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25554944->25559039] (req 25554944->25559039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.455958:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760752.455963:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2ebc3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25554944->25559039] (req 25554944->25559039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760752.455967:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973558 is committed 00010000:00010000:1.0:1789760752.455978:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2ebc3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25554944->25559039] (req 25554944->25559039) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.456001:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2ebc3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25554944->25559039] (req 25554944->25559039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.456009:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2ebc3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25554944->25559039] (req 25554944->25559039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760752.456118:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febfb374380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512659968:12345-192.168.203.56@tcp:4:dd.0 Request processed in 59971us (60213us total) trans 4294973558 rc 0/0 00000100:00100000:1.0:1789760752.456129:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6269 00000100:00100000:1.0:1789760752.456138:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 731879504783 time 731960429121 state 0x2 ref 0 00000100:00100000:3.0:1789760752.462526:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760752.462757:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512660096 00000100:00100000:1.0:1789760752.462767:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512660096 00000100:00100000:1.0:1789760752.462824:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 731879504783 time 731960429121 state 0x0 ref 1 00000100:00100000:1.0:1789760752.462840:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 731960689537 time 731960429121 state 0x1 ref 1 00000100:00100000:1.0:1789760752.462869:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 731960689537 time 732027168616 state 0x0 ref 1 00000100:00100000:1.0:1789760752.462881:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2878700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512660096:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760752.462927:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25559040, mtime: 0x6aad94ef, atime: 0x0, ctime: 0x6aad94ef, blocks: 0xc807 00010000:00010000:1.0:1789760752.462936:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2ebca lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.462953:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=732s v=(0 0) 00010000:00010000:1.0:1789760752.462957:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760752.462963:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2ebca lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25559040->25563135] (req 25559040->25563135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.462977:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2ebca lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25559040->25563135] (req 25559040->25563135) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760752.536339:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973559 is committed 00002000:00010000:1.0:1789760752.536440:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25559040 -> 25563136 00010000:00010000:1.0:1789760752.536456:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2ebca lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25559040->25563135] (req 25559040->25563135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.536470:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2ebca lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25559040->25563135] (req 25559040->25563135) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.536479:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2ebca lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25559040->25563135] (req 25559040->25563135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.536488:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760752.536494:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2ebca lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25559040->25563135] (req 25559040->25563135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.536505:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2ebca lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25559040->25563135] (req 25559040->25563135) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.536526:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2ebca lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25559040->25563135] (req 25559040->25563135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.536535:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2ebca lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25559040->25563135] (req 25559040->25563135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760752.536632:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2878700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512660096:12345-192.168.203.56@tcp:4:dd.0 Request processed in 73748us (74112us total) trans 4294973559 rc 0/0 00000100:00100000:1.0:1789760752.536642:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6270 00000100:00100000:1.0:1789760752.536653:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 731960689537 time 732027168616 state 0x2 ref 0 00000100:00100000:3.0:1789760752.543381:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760752.543957:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512660224 00000100:00100000:2.0:1789760752.543967:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512660224 00000100:00100000:2.0:1789760752.544021:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 731960689537 time 732027168616 state 0x0 ref 1 00000100:00100000:2.0:1789760752.544047:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 732027429032 time 732027168616 state 0x1 ref 1 00000100:00100000:2.0:1789760752.544160:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 732027429032 time 732108458073 state 0x0 ref 1 00000100:00100000:2.0:1789760752.544173:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac287aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512660224:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760752.544224:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25563136, mtime: 0x6aad94ef, atime: 0x0, ctime: 0x6aad94ef, blocks: 0xc807 00010000:00010000:2.0:1789760752.544233:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2ebd1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.544249:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=732s v=(0 0) 00010000:00010000:2.0:1789760752.544263:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760752.544269:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2ebd1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25563136->25567231] (req 25563136->25567231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.544346:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2ebd1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25563136->25567231] (req 25563136->25567231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760752.591756:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973560 is committed 00002000:00010000:0.0:1789760752.591913:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25563136 -> 25567232 00002000:00010000:0.0:1789760752.591917:0:6708:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760751 -> 1789760752 00002000:00010000:0.0:1789760752.591977:0:6708:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760751 -> 1789760752 00010000:00010000:0.0:1789760752.592001:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2ebd1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25563136->25567231] (req 25563136->25567231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.592013:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2ebd1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25563136->25567231] (req 25563136->25567231) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.592023:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2ebd1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25563136->25567231] (req 25563136->25567231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.592030:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760752.592036:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2ebd1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25563136->25567231] (req 25563136->25567231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.592218:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2ebd1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25563136->25567231] (req 25563136->25567231) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.592240:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2ebd1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25563136->25567231] (req 25563136->25567231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.592248:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2ebd1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25563136->25567231] (req 25563136->25567231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760752.592350:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac287aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512660224:12345-192.168.203.56@tcp:4:dd.0 Request processed in 48174us (48976us total) trans 4294973560 rc 0/0 00000100:00100000:0.0:1789760752.592361:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6271 00000100:00100000:0.0:1789760752.592371:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 732027429032 time 732108458073 state 0x2 ref 0 00000100:00100000:0.0:1789760752.595532:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760752.595734:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512660352 00000100:00100000:1.0:1789760752.595744:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512660352 00000100:00100000:1.0:1789760752.595793:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 732027429032 time 732108458073 state 0x0 ref 1 00000100:00100000:1.0:1789760752.595807:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 732108718489 time 732108458073 state 0x1 ref 1 00000100:00100000:1.0:1789760752.595837:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 732108718489 time 732160136456 state 0x0 ref 1 00000100:00100000:1.0:1789760752.595848:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac531ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512660352:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760752.595898:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25567232, mtime: 0x6aad94f0, atime: 0x0, ctime: 0x6aad94f0, blocks: 0xc807 00010000:00010000:1.0:1789760752.595907:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2ebd8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.596026:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=732s v=(0 0) 00010000:00010000:1.0:1789760752.596032:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760752.596039:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2ebd8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25567232->25571327] (req 25567232->25571327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.596059:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2ebd8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25567232->25571327] (req 25567232->25571327) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760752.634947:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973561 is committed 00002000:00010000:0.0:1789760752.635077:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25567232 -> 25571328 00010000:00010000:0.0:1789760752.635093:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2ebd8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25567232->25571327] (req 25567232->25571327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.635106:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2ebd8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25567232->25571327] (req 25567232->25571327) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.635115:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2ebd8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25567232->25571327] (req 25567232->25571327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.635122:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760752.635128:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2ebd8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25567232->25571327] (req 25567232->25571327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.635136:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2ebd8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25567232->25571327] (req 25567232->25571327) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.635155:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2ebd8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25567232->25571327] (req 25567232->25571327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.635162:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2ebd8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25567232->25571327] (req 25567232->25571327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760752.635268:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac531ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512660352:12345-192.168.203.56@tcp:4:dd.0 Request processed in 39407us (39729us total) trans 4294973561 rc 0/0 00000100:00100000:0.0:1789760752.635277:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6272 00000100:00100000:0.0:1789760752.635287:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 732108718489 time 732160136456 state 0x2 ref 0 00000100:00100000:3.0:1789760752.643233:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760752.643401:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512660480 00000100:00100000:1.0:1789760752.643411:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512660480 00000100:00100000:1.0:1789760752.643467:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 732108718489 time 732160136456 state 0x0 ref 1 00000100:00100000:1.0:1789760752.643484:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 732160396872 time 732160136456 state 0x1 ref 1 00000100:00100000:1.0:1789760752.643521:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 732160396872 time 732207820188 state 0x0 ref 1 00000100:00100000:1.0:1789760752.643533:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2879880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512660480:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760752.643584:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25571328, mtime: 0x6aad94f0, atime: 0x0, ctime: 0x6aad94f0, blocks: 0xc807 00010000:00010000:1.0:1789760752.643593:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2ebdf lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.643608:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=732s v=(0 0) 00010000:00010000:1.0:1789760752.643613:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760752.643618:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2ebdf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25571328->25575423] (req 25571328->25575423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.643634:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2ebdf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25571328->25575423] (req 25571328->25575423) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760752.685082:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973562 is committed 00002000:00010000:1.0:1789760752.685192:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25571328 -> 25575424 00010000:00010000:1.0:1789760752.685209:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2ebdf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25571328->25575423] (req 25571328->25575423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.685222:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2ebdf lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25571328->25575423] (req 25571328->25575423) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.685231:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2ebdf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25571328->25575423] (req 25571328->25575423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.685239:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760752.685244:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2ebdf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25571328->25575423] (req 25571328->25575423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.685264:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2ebdf lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25571328->25575423] (req 25571328->25575423) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.685284:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2ebdf lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25571328->25575423] (req 25571328->25575423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.685292:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2ebdf lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25571328->25575423] (req 25571328->25575423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760752.685388:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2879880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512660480:12345-192.168.203.56@tcp:4:dd.0 Request processed in 41853us (42159us total) trans 4294973562 rc 0/0 00000100:00100000:1.0:1789760752.685398:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6273 00000100:00100000:1.0:1789760752.685408:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 732160396872 time 732207820188 state 0x2 ref 0 00000100:00100000:3.0:1789760752.688625:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760752.688828:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512660608 00000100:00100000:0.0:1789760752.688839:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512660608 00000100:00100000:0.0:1789760752.688894:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 732160396872 time 732207820188 state 0x0 ref 1 00000100:00100000:0.0:1789760752.688909:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 732208080604 time 732207820188 state 0x1 ref 1 00000100:00100000:0.0:1789760752.688946:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 732208080604 time 732253245703 state 0x0 ref 1 00000100:00100000:0.0:1789760752.688959:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2879180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512660608:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760752.689006:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25575424, mtime: 0x6aad94f0, atime: 0x0, ctime: 0x6aad94f0, blocks: 0xc807 00010000:00010000:0.0:1789760752.689014:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2ebe6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.689030:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=732s v=(0 0) 00010000:00010000:0.0:1789760752.689035:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760752.689041:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2ebe6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25575424->25579519] (req 25575424->25579519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.689056:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2ebe6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25575424->25579519] (req 25575424->25579519) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760752.719231:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25575424 -> 25579520 00010000:00010000:0.0:1789760752.719261:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2ebe6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25575424->25579519] (req 25575424->25579519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.719275:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2ebe6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25575424->25579519] (req 25575424->25579519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.719284:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2ebe6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25575424->25579519] (req 25575424->25579519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.719291:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760752.719297:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2ebe6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25575424->25579519] (req 25575424->25579519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.719306:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2ebe6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25575424->25579519] (req 25575424->25579519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.719326:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2ebe6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25575424->25579519] (req 25575424->25579519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.719346:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2ebe6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25575424->25579519] (req 25575424->25579519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760752.719441:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2879180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2359:x1876699512660608:12345-192.168.203.56@tcp:4:dd.0 Request processed in 30480us (30820us total) trans 4294973563 rc 0/0 00000100:00100000:0.0:1789760752.719456:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6274 00000100:00100000:0.0:1789760752.719466:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 732208080604 time 732253245703 state 0x2 ref 0 00000001:00080000:3.0:1789760752.721191:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973563 is committed 00000100:00100000:0.0:1789760752.725082:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760752.725233:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512660736 00000100:00100000:1.0:1789760752.725243:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512660736 00000100:00100000:1.0:1789760752.725307:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 732208080604 time 732253245703 state 0x0 ref 1 00000100:00100000:1.0:1789760752.725323:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 732253506119 time 732253245703 state 0x1 ref 1 00000100:00100000:1.0:1789760752.725351:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 732253506119 time 732289651465 state 0x0 ref 1 00000100:00100000:1.0:1789760752.725364:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac531e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512660736:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760752.725413:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25579520, mtime: 0x6aad94f0, atime: 0x0, ctime: 0x6aad94f0, blocks: 0xc807 00010000:00010000:1.0:1789760752.725422:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2ebed lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.725438:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=732s v=(0 0) 00010000:00010000:1.0:1789760752.725443:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760752.725449:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2ebed lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25579520->25583615] (req 25579520->25583615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.725463:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2ebed lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25579520->25583615] (req 25579520->25583615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760752.761174:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25579520 -> 25583616 00010000:00010000:0.0:1789760752.761194:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2ebed lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25579520->25583615] (req 25579520->25583615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.761207:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2ebed lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25579520->25583615] (req 25579520->25583615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.761216:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2ebed lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25579520->25583615] (req 25579520->25583615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.761224:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760752.761229:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2ebed lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25579520->25583615] (req 25579520->25583615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.761237:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2ebed lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25579520->25583615] (req 25579520->25583615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.761279:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2ebed lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25579520->25583615] (req 25579520->25583615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.761287:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2ebed lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25579520->25583615] (req 25579520->25583615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760752.761419:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac531e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2359:x1876699512660736:12345-192.168.203.56@tcp:4:dd.0 Request processed in 36052us (36341us total) trans 4294973564 rc 0/0 00000100:00100000:0.0:1789760752.761430:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6275 00000100:00100000:0.0:1789760752.761439:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 732253506119 time 732289651465 state 0x2 ref 0 00000001:00080000:3.0:1789760752.762070:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973564 is committed 00000100:00100000:1.0:1789760752.766282:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760752.766429:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512660864 00000100:00100000:0.0:1789760752.766439:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512660864 00000100:00100000:0.0:1789760752.766491:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 732253506119 time 732289651465 state 0x0 ref 1 00000100:00100000:0.0:1789760752.766507:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 732289911881 time 732289651465 state 0x1 ref 1 00000100:00100000:0.0:1789760752.766533:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 732289911881 time 732330833220 state 0x0 ref 1 00000100:00100000:0.0:1789760752.766545:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febfb89fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512660864:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760752.766589:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25583616, mtime: 0x6aad94f0, atime: 0x0, ctime: 0x6aad94f0, blocks: 0xc807 00010000:00010000:0.0:1789760752.766597:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2ebf4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.766614:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=732s v=(0 0) 00010000:00010000:0.0:1789760752.766618:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760752.766625:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2ebf4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25583616->25587711] (req 25583616->25587711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.766638:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2ebf4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25583616->25587711] (req 25583616->25587711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760752.792640:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25583616 -> 25587712 00010000:00010000:1.0:1789760752.792660:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2ebf4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25583616->25587711] (req 25583616->25587711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.792678:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2ebf4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25583616->25587711] (req 25583616->25587711) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.792687:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2ebf4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25583616->25587711] (req 25583616->25587711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.792693:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760752.792728:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2ebf4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25583616->25587711] (req 25583616->25587711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.792738:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2ebf4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25583616->25587711] (req 25583616->25587711) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.792758:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2ebf4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25583616->25587711] (req 25583616->25587711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.792766:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2ebf4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25583616->25587711] (req 25583616->25587711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760752.792853:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febfb89fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2359:x1876699512660864:12345-192.168.203.56@tcp:4:dd.0 Request processed in 26305us (26574us total) trans 4294973565 rc 0/0 00000100:00100000:1.0:1789760752.792864:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6276 00000100:00100000:1.0:1789760752.792873:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 732289911881 time 732330833220 state 0x2 ref 0 00000001:00080000:0.0:1789760752.793227:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973565 is committed 00000100:00100000:1.0:1789760752.795910:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760752.796027:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512660992 00000100:00100000:0.0:1789760752.796037:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512660992 00000100:00100000:0.0:1789760752.796093:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 732289911881 time 732330833220 state 0x0 ref 1 00000100:00100000:0.0:1789760752.796110:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 732331093636 time 732330833220 state 0x1 ref 1 00000100:00100000:0.0:1789760752.796189:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 732331093636 time 732360487382 state 0x0 ref 1 00000100:00100000:0.0:1789760752.796202:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4132d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512660992:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760752.796264:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25587712, mtime: 0x6aad94f0, atime: 0x0, ctime: 0x6aad94f0, blocks: 0xc807 00010000:00010000:0.0:1789760752.796273:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2ebfb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.796288:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=732s v=(0 0) 00010000:00010000:0.0:1789760752.796292:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760752.796299:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2ebfb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25587712->25591807] (req 25587712->25591807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.796313:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2ebfb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25587712->25591807] (req 25587712->25591807) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760752.849323:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25587712 -> 25591808 00010000:00010000:2.0:1789760752.849345:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2ebfb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25587712->25591807] (req 25587712->25591807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.849358:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2ebfb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25587712->25591807] (req 25587712->25591807) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.849367:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2ebfb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25587712->25591807] (req 25587712->25591807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.849374:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760752.849378:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2ebfb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25587712->25591807] (req 25587712->25591807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.849387:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2ebfb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25587712->25591807] (req 25587712->25591807) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.849409:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2ebfb lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25587712->25591807] (req 25587712->25591807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.849416:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2ebfb lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25587712->25591807] (req 25587712->25591807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760752.849507:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4132d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2359:x1876699512660992:12345-192.168.203.56@tcp:4:dd.0 Request processed in 53301us (53600us total) trans 4294973566 rc 0/0 00000100:00100000:2.0:1789760752.849522:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6277 00000100:00100000:2.0:1789760752.849531:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 732331093636 time 732360487382 state 0x2 ref 0 00000001:00080000:3.0:1789760752.850284:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973566 is committed 00000100:00100000:2.0:1789760752.858154:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760752.858494:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512661120 00000100:00100000:0.0:1789760752.858504:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512661120 00000100:00100000:0.0:1789760752.858557:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 732331093636 time 732360487382 state 0x0 ref 1 00000100:00100000:0.0:1789760752.858577:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 732360747798 time 732360487382 state 0x1 ref 1 00000100:00100000:0.0:1789760752.858607:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 732360747798 time 732422906848 state 0x0 ref 1 00000100:00100000:0.0:1789760752.858621:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5306680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512661120:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760752.858667:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25591808, mtime: 0x6aad94f0, atime: 0x0, ctime: 0x6aad94f0, blocks: 0xc807 00010000:00010000:0.0:1789760752.858676:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2ec02 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.858691:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=732s v=(0 0) 00010000:00010000:0.0:1789760752.858732:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760752.858740:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2ec02 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25591808->25595903] (req 25591808->25595903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.858753:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2ec02 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25591808->25595903] (req 25591808->25595903) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760752.895587:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25591808 -> 25595904 00010000:00010000:0.0:1789760752.895609:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2ec02 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25591808->25595903] (req 25591808->25595903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.895622:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2ec02 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25591808->25595903] (req 25591808->25595903) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.895630:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2ec02 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25591808->25595903] (req 25591808->25595903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.895654:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760752.895658:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2ec02 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25591808->25595903] (req 25591808->25595903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.895667:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2ec02 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25591808->25595903] (req 25591808->25595903) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.895686:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2ec02 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25591808->25595903] (req 25591808->25595903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.895692:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2ec02 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25591808->25595903] (req 25591808->25595903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760752.895807:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5306680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2359:x1876699512661120:12345-192.168.203.56@tcp:4:dd.0 Request processed in 37184us (37656us total) trans 4294973567 rc 0/0 00000100:00100000:0.0:1789760752.895818:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6278 00000100:00100000:0.0:1789760752.895828:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 732360747798 time 732422906848 state 0x2 ref 0 00000001:00080000:3.0:1789760752.896433:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973567 is committed 00000100:00100000:1.0:1789760752.900013:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760752.900179:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512661248 00000100:00100000:3.0:1789760752.900189:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512661248 00000100:00100000:3.0:1789760752.900246:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 732360747798 time 732422906848 state 0x0 ref 1 00000100:00100000:3.0:1789760752.900274:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 732423167264 time 732422906848 state 0x1 ref 1 00000100:00100000:3.0:1789760752.900303:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 732423167264 time 732464602483 state 0x0 ref 1 00000100:00100000:3.0:1789760752.900315:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4132300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512661248:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760752.900372:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25595904, mtime: 0x6aad94f0, atime: 0x0, ctime: 0x6aad94f0, blocks: 0xc807 00010000:00010000:3.0:1789760752.900381:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2ec09 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.900396:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=732s v=(0 0) 00010000:00010000:3.0:1789760752.900401:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760752.900408:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2ec09 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25595904->25599999] (req 25595904->25599999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.900421:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2ec09 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25595904->25599999] (req 25595904->25599999) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760752.923267:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25595904 -> 25600000 00010000:00010000:3.0:1789760752.923290:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2ec09 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25595904->25599999] (req 25595904->25599999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.923304:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2ec09 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25595904->25599999] (req 25595904->25599999) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.923317:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2ec09 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25595904->25599999] (req 25595904->25599999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.923329:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760752.923334:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2ec09 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25595904->25599999] (req 25595904->25599999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.923343:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2ec09 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25595904->25599999] (req 25595904->25599999) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.923362:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2ec09 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25595904->25599999] (req 25595904->25599999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.923370:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2ec09 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25595904->25599999] (req 25595904->25599999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760752.923596:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4132300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2359:x1876699512661248:12345-192.168.203.56@tcp:4:dd.0 Request processed in 23274us (23585us total) trans 4294973568 rc 0/0 00000100:00100000:3.0:1789760752.923608:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6279 00000100:00100000:3.0:1789760752.923619:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 732423167264 time 732464602483 state 0x2 ref 0 00000001:00080000:3.0:1789760752.923802:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973568 is committed 00000100:00100000:1.0:1789760752.927115:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760752.927414:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512661376 00000100:00100000:2.0:1789760752.927424:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512661376 00000100:00100000:2.0:1789760752.927477:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 732423167264 time 732464602483 state 0x0 ref 1 00000100:00100000:2.0:1789760752.927492:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 732464862899 time 732464602483 state 0x1 ref 1 00000100:00100000:2.0:1789760752.927522:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 732464862899 time 732491821942 state 0x0 ref 1 00000100:00100000:2.0:1789760752.927535:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4131f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512661376:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760752.927581:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25600000, mtime: 0x6aad94f0, atime: 0x0, ctime: 0x6aad94f0, blocks: 0xc807 00010000:00010000:2.0:1789760752.927591:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2ec10 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.927606:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=732s v=(0 0) 00010000:00010000:2.0:1789760752.927611:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760752.927617:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2ec10 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25600000->25604095] (req 25600000->25604095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.927632:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2ec10 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25600000->25604095] (req 25600000->25604095) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760752.958158:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973569 is committed 00002000:00010000:2.0:1789760752.958333:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25600000 -> 25604096 00010000:00010000:2.0:1789760752.958349:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2ec10 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25600000->25604095] (req 25600000->25604095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.958362:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2ec10 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25600000->25604095] (req 25600000->25604095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.958370:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2ec10 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25600000->25604095] (req 25600000->25604095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.958377:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760752.958381:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2ec10 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25600000->25604095] (req 25600000->25604095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.958390:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2ec10 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25600000->25604095] (req 25600000->25604095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.958407:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2ec10 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25600000->25604095] (req 25600000->25604095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.958415:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2ec10 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25600000->25604095] (req 25600000->25604095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760752.958507:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4131f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512661376:12345-192.168.203.56@tcp:4:dd.0 Request processed in 30969us (31395us total) trans 4294973569 rc 0/0 00000100:00100000:2.0:1789760752.958517:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6280 00000100:00100000:2.0:1789760752.958527:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 732464862899 time 732491821942 state 0x2 ref 0 00000100:00100000:2.0:1789760752.962685:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760752.963170:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512661504 00000100:00100000:0.0:1789760752.963180:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512661504 00000100:00100000:0.0:1789760752.963234:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 732464862899 time 732491821942 state 0x0 ref 1 00000100:00100000:0.0:1789760752.963261:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 732492082358 time 732491821942 state 0x1 ref 1 00000100:00100000:0.0:1789760752.963322:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 732492082358 time 732527620705 state 0x0 ref 1 00000100:00100000:0.0:1789760752.963339:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5307b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512661504:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760752.963394:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25604096, mtime: 0x6aad94f0, atime: 0x0, ctime: 0x6aad94f0, blocks: 0xc807 00010000:00010000:0.0:1789760752.963403:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2ec17 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.963419:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=732s v=(0 0) 00010000:00010000:0.0:1789760752.963424:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760752.963430:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2ec17 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25604096->25608191] (req 25604096->25608191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.963445:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2ec17 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25604096->25608191] (req 25604096->25608191) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760752.993475:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973570 is committed 00002000:00010000:3.0:1789760752.993582:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25604096 -> 25608192 00010000:00010000:3.0:1789760752.993600:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2ec17 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25604096->25608191] (req 25604096->25608191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.993612:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2ec17 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25604096->25608191] (req 25604096->25608191) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.993621:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2ec17 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25604096->25608191] (req 25604096->25608191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.993628:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760752.993633:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2ec17 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25604096->25608191] (req 25604096->25608191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.993642:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2ec17 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25604096->25608191] (req 25604096->25608191) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.993662:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2ec17 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25604096->25608191] (req 25604096->25608191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.993670:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2ec17 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25604096->25608191] (req 25604096->25608191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760752.993793:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5307b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512661504:12345-192.168.203.56@tcp:4:dd.0 Request processed in 30452us (31113us total) trans 4294973570 rc 0/0 00000100:00100000:3.0:1789760752.993804:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6281 00000100:00100000:3.0:1789760752.993813:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 732492082358 time 732527620705 state 0x2 ref 0 00000100:00100000:3.0:1789760752.997599:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760752.999560:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512661632 00000100:00100000:1.0:1789760752.999575:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512661632 00000100:00100000:1.0:1789760752.999637:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 732492082358 time 732527620705 state 0x0 ref 1 00000100:00100000:1.0:1789760752.999652:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 732527881121 time 732527620705 state 0x1 ref 1 00000100:00100000:1.0:1789760752.999675:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 732527881121 time 732563975744 state 0x0 ref 1 00000100:00100000:1.0:1789760752.999688:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2879500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512661632:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760753.000153:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25608192, mtime: 0x6aad94f0, atime: 0x0, ctime: 0x6aad94f0, blocks: 0xc807 00010000:00010000:1.0:1789760753.000164:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2ec1e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.000179:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=732s v=(0 0) 00010000:00010000:1.0:1789760753.000184:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760753.000190:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2ec1e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25608192->25612287] (req 25608192->25612287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.000204:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2ec1e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25608192->25612287] (req 25608192->25612287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.010828:0:7647:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:2.0:1789760753.039403:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973571 is committed 00002000:00010000:1.0:1789760753.039520:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25608192 -> 25612288 00010000:00010000:1.0:1789760753.039536:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2ec1e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25608192->25612287] (req 25608192->25612287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.039554:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2ec1e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25608192->25612287] (req 25608192->25612287) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.039563:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2ec1e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25608192->25612287] (req 25608192->25612287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.039570:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760753.039575:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2ec1e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25608192->25612287] (req 25608192->25612287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.039584:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2ec1e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25608192->25612287] (req 25608192->25612287) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.039603:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2ec1e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25608192->25612287] (req 25608192->25612287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.039611:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2ec1e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25608192->25612287] (req 25608192->25612287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760753.039933:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2879500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512661632:12345-192.168.203.56@tcp:4:dd.0 Request processed in 40233us (42329us total) trans 4294973571 rc 0/0 00000100:00100000:1.0:1789760753.039945:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6282 00000100:00100000:1.0:1789760753.039955:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 732527881121 time 732563975744 state 0x2 ref 0 00000100:00100000:1.0:1789760753.042869:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760753.043039:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512661760 00000100:00100000:2.0:1789760753.043049:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512661760 00000100:00100000:2.0:1789760753.043105:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 732527881121 time 732563975744 state 0x0 ref 1 00000100:00100000:2.0:1789760753.043120:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 732564236160 time 732563975744 state 0x1 ref 1 00000100:00100000:2.0:1789760753.043153:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 732564236160 time 732607452953 state 0x0 ref 1 00000100:00100000:2.0:1789760753.043165:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4130380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512661760:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760753.043207:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25612288, mtime: 0x6aad94f0, atime: 0x0, ctime: 0x6aad94f0, blocks: 0xc807 00010000:00010000:2.0:1789760753.043216:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2ec25 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.043231:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=732s v=(0 0) 00010000:00010000:2.0:1789760753.043236:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760753.043242:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2ec25 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25612288->25616383] (req 25612288->25616383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.043264:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2ec25 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25612288->25616383] (req 25612288->25616383) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1789760753.046002:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:1.0:1789760753.082776:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973572 is committed 00002000:00010000:3.0:1789760753.083499:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25612288 -> 25616384 00010000:00010000:3.0:1789760753.083517:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2ec25 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25612288->25616383] (req 25612288->25616383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.083532:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2ec25 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25612288->25616383] (req 25612288->25616383) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.083542:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2ec25 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25612288->25616383] (req 25612288->25616383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.083549:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760753.083554:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2ec25 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25612288->25616383] (req 25612288->25616383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.083563:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2ec25 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25612288->25616383] (req 25612288->25616383) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.083585:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2ec25 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25612288->25616383] (req 25612288->25616383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.083592:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2ec25 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25612288->25616383] (req 25612288->25616383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760753.083842:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4130380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512661760:12345-192.168.203.56@tcp:4:dd.0 Request processed in 40673us (40975us total) trans 4294973572 rc 0/0 00000100:00100000:3.0:1789760753.083854:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6283 00000100:00100000:3.0:1789760753.083864:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 732564236160 time 732607452953 state 0x2 ref 0 00000100:00100000:2.0:1789760753.086315:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760753.086398:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512661888 00000100:00100000:3.0:1789760753.086408:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512661888 00000100:00100000:3.0:1789760753.086464:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 732564236160 time 732607452953 state 0x0 ref 1 00000100:00100000:3.0:1789760753.086479:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 732607713369 time 732607452953 state 0x1 ref 1 00000100:00100000:3.0:1789760753.086515:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 732607713369 time 732650814923 state 0x0 ref 1 00000100:00100000:3.0:1789760753.086528:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5307100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512661888:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760753.086581:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25616384, mtime: 0x6aad94f0, atime: 0x0, ctime: 0x6aad94f0, blocks: 0xc807 00010000:00010000:3.0:1789760753.086589:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2ec2c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.086605:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=732s v=(0 0) 00010000:00010000:3.0:1789760753.086610:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760753.086616:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2ec2c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25616384->25620479] (req 25616384->25620479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.086631:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2ec2c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25616384->25620479] (req 25616384->25620479) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760753.126401:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973573 is committed 00002000:00010000:2.0:1789760753.126631:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25616384 -> 25620480 00010000:00010000:2.0:1789760753.126719:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2ec2c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25616384->25620479] (req 25616384->25620479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.126734:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2ec2c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25616384->25620479] (req 25616384->25620479) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.126742:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2ec2c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25616384->25620479] (req 25616384->25620479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.126749:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760753.126754:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2ec2c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25616384->25620479] (req 25616384->25620479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.126763:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2ec2c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25616384->25620479] (req 25616384->25620479) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.126782:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2ec2c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25616384->25620479] (req 25616384->25620479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.126789:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2ec2c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25616384->25620479] (req 25616384->25620479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760753.130486:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5307100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512661888:12345-192.168.203.56@tcp:4:dd.0 Request processed in 43948us (44169us total) trans 4294973573 rc 0/0 00000100:00100000:2.0:1789760753.130500:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6284 00000100:00100000:2.0:1789760753.130510:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 732607713369 time 732650814923 state 0x2 ref 0 00000100:00100000:0.0:1789760753.133802:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760753.133967:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512662016 00000100:00100000:1.0:1789760753.133977:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512662016 00000100:00100000:1.0:1789760753.134026:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 732607713369 time 732650814923 state 0x0 ref 1 00000100:00100000:1.0:1789760753.134041:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 732651075339 time 732650814923 state 0x1 ref 1 00000100:00100000:1.0:1789760753.134072:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 732651075339 time 732698371813 state 0x0 ref 1 00000100:00100000:1.0:1789760753.134084:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac531dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512662016:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760753.134129:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25620480, mtime: 0x6aad94f0, atime: 0x0, ctime: 0x6aad94f0, blocks: 0xc807 00010000:00010000:1.0:1789760753.134138:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2ec33 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.134152:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=732s v=(0 0) 00010000:00010000:1.0:1789760753.134157:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760753.134163:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2ec33 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25620480->25624575] (req 25620480->25624575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.134175:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2ec33 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25620480->25624575] (req 25620480->25624575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760753.177823:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973574 is committed 00002000:00010000:2.0:1789760753.178170:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25620480 -> 25624576 00010000:00010000:2.0:1789760753.178186:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2ec33 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25620480->25624575] (req 25620480->25624575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.178201:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2ec33 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25620480->25624575] (req 25620480->25624575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.178210:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2ec33 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25620480->25624575] (req 25620480->25624575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.178217:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760753.178222:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2ec33 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25620480->25624575] (req 25620480->25624575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.178230:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2ec33 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25620480->25624575] (req 25620480->25624575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.178258:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2ec33 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25620480->25624575] (req 25620480->25624575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.178269:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2ec33 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25620480->25624575] (req 25620480->25624575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760753.178389:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac531dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512662016:12345-192.168.203.56@tcp:4:dd.0 Request processed in 44302us (44591us total) trans 4294973574 rc 0/0 00000100:00100000:2.0:1789760753.178399:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6285 00000100:00100000:2.0:1789760753.178410:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 732651075339 time 732698371813 state 0x2 ref 0 00000100:00100000:0.0:1789760753.181281:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760753.181388:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512662144 00000100:00100000:3.0:1789760753.181398:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512662144 00000100:00100000:3.0:1789760753.181444:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 732651075339 time 732698371813 state 0x0 ref 1 00000100:00100000:3.0:1789760753.181460:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 732698632229 time 732698371813 state 0x1 ref 1 00000100:00100000:3.0:1789760753.181509:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 732698632229 time 732745809206 state 0x0 ref 1 00000100:00100000:3.0:1789760753.181518:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac531ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512662144:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760753.181561:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25624576, mtime: 0x6aad94f0, atime: 0x0, ctime: 0x6aad94f0, blocks: 0xc807 00010000:00010000:3.0:1789760753.181567:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2ec3a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.181580:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=732s v=(0 0) 00010000:00010000:3.0:1789760753.181584:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760753.181588:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2ec3a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25624576->25628671] (req 25624576->25628671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.181598:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2ec3a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25624576->25628671] (req 25624576->25628671) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760753.218580:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973575 is committed 00002000:00010000:3.0:1789760753.218821:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25624576 -> 25628672 00010000:00010000:3.0:1789760753.218835:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2ec3a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25624576->25628671] (req 25624576->25628671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.218848:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2ec3a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25624576->25628671] (req 25624576->25628671) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.218857:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2ec3a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25624576->25628671] (req 25624576->25628671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.218870:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760753.218875:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2ec3a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25624576->25628671] (req 25624576->25628671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.218884:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2ec3a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25624576->25628671] (req 25624576->25628671) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.218901:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2ec3a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25624576->25628671] (req 25624576->25628671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.218909:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2ec3a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25624576->25628671] (req 25624576->25628671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760753.219081:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac531ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512662144:12345-192.168.203.56@tcp:4:dd.0 Request processed in 37557us (37808us total) trans 4294973575 rc 0/0 00000100:00100000:3.0:1789760753.219092:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6286 00000100:00100000:3.0:1789760753.219103:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 732698632229 time 732745809206 state 0x2 ref 0 00000100:00100000:0.0:1789760753.232293:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760753.232394:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512662272 00000100:00100000:2.0:1789760753.232405:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512662272 00000100:00100000:2.0:1789760753.232462:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 732698632229 time 732745809206 state 0x0 ref 1 00000100:00100000:2.0:1789760753.232478:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 732746069622 time 732745809206 state 0x1 ref 1 00000100:00100000:2.0:1789760753.232511:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 732746069622 time 732796810047 state 0x0 ref 1 00000100:00100000:2.0:1789760753.232523:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac531f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512662272:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760753.232576:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25628672, mtime: 0x6aad94f0, atime: 0x0, ctime: 0x6aad94f0, blocks: 0xc807 00010000:00010000:2.0:1789760753.232585:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2ec41 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.232601:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=732s v=(0 0) 00010000:00010000:2.0:1789760753.232606:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760753.232613:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2ec41 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25628672->25632767] (req 25628672->25632767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.232626:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2ec41 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25628672->25632767] (req 25628672->25632767) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760753.263790:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25628672 -> 25632768 00010000:00010000:2.0:1789760753.263812:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2ec41 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25628672->25632767] (req 25628672->25632767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.263830:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2ec41 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25628672->25632767] (req 25628672->25632767) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.263839:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2ec41 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25628672->25632767] (req 25628672->25632767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.263846:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760753.263851:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2ec41 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25628672->25632767] (req 25628672->25632767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.263859:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2ec41 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25628672->25632767] (req 25628672->25632767) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.263879:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2ec41 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25628672->25632767] (req 25628672->25632767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.263886:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2ec41 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25628672->25632767] (req 25628672->25632767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760753.263989:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac531f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2360:x1876699512662272:12345-192.168.203.56@tcp:4:dd.0 Request processed in 31462us (31700us total) trans 4294973576 rc 0/0 00000100:00100000:2.0:1789760753.263999:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6287 00000100:00100000:2.0:1789760753.264008:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 732746069622 time 732796810047 state 0x2 ref 0 00000001:00080000:3.0:1789760753.265467:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973576 is committed 00000100:00100000:0.0:1789760753.266443:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760753.266534:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512662400 00000100:00100000:2.0:1789760753.266543:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512662400 00000100:00100000:2.0:1789760753.266593:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 732746069622 time 732796810047 state 0x0 ref 1 00000100:00100000:2.0:1789760753.266606:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 732797070463 time 732796810047 state 0x1 ref 1 00000100:00100000:2.0:1789760753.266645:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 732797070463 time 732830937374 state 0x0 ref 1 00000100:00100000:2.0:1789760753.266657:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac531df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512662400:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760753.266733:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25632768, mtime: 0x6aad94f0, atime: 0x0, ctime: 0x6aad94f0, blocks: 0xc807 00010000:00010000:2.0:1789760753.266741:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2ec48 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.266755:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=732s v=(0 0) 00010000:00010000:2.0:1789760753.266759:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760753.266764:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2ec48 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25632768->25636863] (req 25632768->25636863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.266777:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2ec48 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25632768->25636863] (req 25632768->25636863) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760753.307545:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25632768 -> 25636864 00010000:00010000:3.0:1789760753.307582:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2ec48 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25632768->25636863] (req 25632768->25636863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.307596:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2ec48 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25632768->25636863] (req 25632768->25636863) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.307612:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2ec48 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25632768->25636863] (req 25632768->25636863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.307620:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760753.307625:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2ec48 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25632768->25636863] (req 25632768->25636863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.307635:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2ec48 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25632768->25636863] (req 25632768->25636863) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.307850:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2ec48 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25632768->25636863] (req 25632768->25636863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.307859:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2ec48 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25632768->25636863] (req 25632768->25636863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760753.307952:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac531df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2361:x1876699512662400:12345-192.168.203.56@tcp:4:dd.0 Request processed in 41291us (41509us total) trans 4294973577 rc 0/0 00000100:00100000:3.0:1789760753.307962:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6288 00000100:00100000:3.0:1789760753.307972:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 732797070463 time 732830937374 state 0x2 ref 0 00000001:00080000:3.0:1789760753.308014:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973577 is committed 00000100:00100000:2.0:1789760753.313179:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760753.313389:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512662528 00000100:00100000:3.0:1789760753.313399:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512662528 00000100:00100000:3.0:1789760753.313449:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 732797070463 time 732830937374 state 0x0 ref 1 00000100:00100000:3.0:1789760753.313468:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 732831197790 time 732830937374 state 0x1 ref 1 00000100:00100000:3.0:1789760753.313499:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 732831197790 time 732877798400 state 0x0 ref 1 00000100:00100000:3.0:1789760753.313510:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5304a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512662528:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760753.313556:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25636864, mtime: 0x6aad94f0, atime: 0x0, ctime: 0x6aad94f0, blocks: 0xc807 00010000:00010000:3.0:1789760753.313566:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2ec4f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.313581:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=732s v=(0 0) 00010000:00010000:3.0:1789760753.313586:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760753.313592:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2ec4f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25636864->25640959] (req 25636864->25640959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.313606:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2ec4f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25636864->25640959] (req 25636864->25640959) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760753.362509:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25636864 -> 25640960 00010000:00010000:2.0:1789760753.362530:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2ec4f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25636864->25640959] (req 25636864->25640959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.362542:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2ec4f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25636864->25640959] (req 25636864->25640959) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.362556:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2ec4f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25636864->25640959] (req 25636864->25640959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.362563:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760753.362568:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2ec4f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25636864->25640959] (req 25636864->25640959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.362576:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2ec4f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25636864->25640959] (req 25636864->25640959) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.362595:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2ec4f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25636864->25640959] (req 25636864->25640959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.362601:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2ec4f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25636864->25640959] (req 25636864->25640959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760753.362902:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973578 is committed 00000100:00100000:2.0:1789760753.363075:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5304a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512662528:12345-192.168.203.56@tcp:4:dd.0 Request processed in 49561us (49901us total) trans 4294973578 rc 0/0 00000100:00100000:2.0:1789760753.363086:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6289 00000100:00100000:2.0:1789760753.363095:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 732831197790 time 732877798400 state 0x2 ref 0 00000100:00100000:0.0:1789760753.368746:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760753.368853:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512662656 00000100:00100000:2.0:1789760753.368863:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512662656 00000100:00100000:2.0:1789760753.368914:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 732831197790 time 732877798400 state 0x0 ref 1 00000100:00100000:2.0:1789760753.368943:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 732878058816 time 732877798400 state 0x1 ref 1 00000100:00100000:2.0:1789760753.368966:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 732878058816 time 732933265860 state 0x0 ref 1 00000100:00100000:2.0:1789760753.368979:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac531c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512662656:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760753.369028:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25640960, mtime: 0x6aad94f0, atime: 0x0, ctime: 0x6aad94f0, blocks: 0xc807 00010000:00010000:2.0:1789760753.369037:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2ec56 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.369054:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=732s v=(0 0) 00010000:00010000:2.0:1789760753.369059:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760753.369065:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2ec56 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25640960->25645055] (req 25640960->25645055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.369080:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2ec56 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25640960->25645055] (req 25640960->25645055) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760753.398287:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25640960 -> 25645056 00010000:00010000:2.0:1789760753.398308:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2ec56 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25640960->25645055] (req 25640960->25645055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.398321:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2ec56 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25640960->25645055] (req 25640960->25645055) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.398330:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2ec56 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25640960->25645055] (req 25640960->25645055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.398337:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760753.398342:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2ec56 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25640960->25645055] (req 25640960->25645055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.398350:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2ec56 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25640960->25645055] (req 25640960->25645055) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.398369:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2ec56 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25640960->25645055] (req 25640960->25645055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.398376:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2ec56 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25640960->25645055] (req 25640960->25645055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760753.398533:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973579 is committed 00000100:00100000:2.0:1789760753.398589:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac531c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512662656:12345-192.168.203.56@tcp:4:dd.0 Request processed in 29607us (29889us total) trans 4294973579 rc 0/0 00000100:00100000:2.0:1789760753.398599:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6290 00000100:00100000:2.0:1789760753.398609:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 732878058816 time 732933265860 state 0x2 ref 0 00000100:00100000:0.0:1789760753.403585:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760753.403789:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512662784 00000100:00100000:3.0:1789760753.403798:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512662784 00000100:00100000:3.0:1789760753.404073:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 732878058816 time 732933265860 state 0x0 ref 1 00000100:00100000:3.0:1789760753.404089:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 732933526276 time 732933265860 state 0x1 ref 1 00000100:00100000:3.0:1789760753.404160:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 732933526276 time 732968458103 state 0x0 ref 1 00000100:00100000:3.0:1789760753.404177:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac531fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512662784:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760753.404308:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25645056, mtime: 0x6aad94f0, atime: 0x0, ctime: 0x6aad94f0, blocks: 0xc807 00010000:00010000:3.0:1789760753.404319:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2ec5d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.404334:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=732s v=(0 0) 00010000:00010000:3.0:1789760753.404338:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760753.404344:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2ec5d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25645056->25649151] (req 25645056->25649151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.404358:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2ec5d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25645056->25649151] (req 25645056->25649151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760753.441957:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973580 is committed 00002000:00010000:2.0:1789760753.442109:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25645056 -> 25649152 00010000:00010000:2.0:1789760753.442126:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2ec5d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25645056->25649151] (req 25645056->25649151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.442140:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2ec5d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25645056->25649151] (req 25645056->25649151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.442151:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2ec5d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25645056->25649151] (req 25645056->25649151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.442165:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760753.442171:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2ec5d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25645056->25649151] (req 25645056->25649151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.442181:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2ec5d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25645056->25649151] (req 25645056->25649151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.442203:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2ec5d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25645056->25649151] (req 25645056->25649151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.442211:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2ec5d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25645056->25649151] (req 25645056->25649151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760753.442499:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac531fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512662784:12345-192.168.203.56@tcp:4:dd.0 Request processed in 38320us (38917us total) trans 4294973580 rc 0/0 00000100:00100000:2.0:1789760753.442510:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6291 00000100:00100000:2.0:1789760753.442520:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 732933526276 time 732968458103 state 0x2 ref 0 00000100:00100000:1.0:1789760753.451527:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760753.453428:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512662912 00000100:00100000:2.0:1789760753.453441:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512662912 00000100:00100000:2.0:1789760753.453497:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 732933526276 time 732968458103 state 0x0 ref 1 00000100:00100000:2.0:1789760753.453512:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 732968718519 time 732968458103 state 0x1 ref 1 00000100:00100000:2.0:1789760753.453592:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 732968718519 time 733017890503 state 0x0 ref 1 00000100:00100000:2.0:1789760753.453609:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4133b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512662912:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760753.453668:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25649152, mtime: 0x6aad94f0, atime: 0x0, ctime: 0x6aad94f0, blocks: 0xc807 00010000:00010000:2.0:1789760753.453677:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2ec64 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.453693:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=733s v=(0 0) 00010000:00010000:2.0:1789760753.453713:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760753.453719:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2ec64 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25649152->25653247] (req 25649152->25653247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.453732:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2ec64 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25649152->25653247] (req 25649152->25653247) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760753.504865:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25649152 -> 25653248 00010000:00010000:3.0:1789760753.504885:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2ec64 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25649152->25653247] (req 25649152->25653247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.504898:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2ec64 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25649152->25653247] (req 25649152->25653247) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.504906:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2ec64 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25649152->25653247] (req 25649152->25653247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.504913:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760753.504928:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2ec64 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25649152->25653247] (req 25649152->25653247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.504937:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2ec64 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25649152->25653247] (req 25649152->25653247) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.504957:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2ec64 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25649152->25653247] (req 25649152->25653247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.504964:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2ec64 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25649152->25653247] (req 25649152->25653247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760753.505058:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4133b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2359:x1876699512662912:12345-192.168.203.56@tcp:4:dd.0 Request processed in 51446us (53536us total) trans 4294973581 rc 0/0 00000100:00100000:3.0:1789760753.505068:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6292 00000100:00100000:3.0:1789760753.505077:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 732968718519 time 733017890503 state 0x2 ref 0 00000001:00080000:0.0:1789760753.506090:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973581 is committed 00000100:00100000:1.0:1789760753.510180:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760753.510497:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512663040 00000100:00100000:2.0:1789760753.510507:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512663040 00000100:00100000:2.0:1789760753.510562:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 732968718519 time 733017890503 state 0x0 ref 1 00000100:00100000:2.0:1789760753.510578:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 733018150919 time 733017890503 state 0x1 ref 1 00000100:00100000:2.0:1789760753.511435:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 733018150919 time 733075729057 state 0x0 ref 1 00000100:00100000:2.0:1789760753.511454:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4133800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512663040:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760753.511504:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25653248, mtime: 0x6aad94f0, atime: 0x0, ctime: 0x6aad94f0, blocks: 0xc807 00010000:00010000:2.0:1789760753.511513:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2ec6b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.511535:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=733s v=(0 0) 00010000:00010000:2.0:1789760753.511539:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760753.511545:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2ec6b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25653248->25657343] (req 25653248->25657343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.511561:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2ec6b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25653248->25657343] (req 25653248->25657343) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760753.542169:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973582 is committed 00002000:00010000:2.0:1789760753.542286:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25653248 -> 25657344 00002000:00010000:2.0:1789760753.542291:0:6710:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760752 -> 1789760753 00002000:00010000:2.0:1789760753.542294:0:6710:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760752 -> 1789760753 00010000:00010000:2.0:1789760753.542308:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2ec6b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25653248->25657343] (req 25653248->25657343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.542320:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2ec6b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25653248->25657343] (req 25653248->25657343) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.542329:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2ec6b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25653248->25657343] (req 25653248->25657343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.542336:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760753.542342:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2ec6b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25653248->25657343] (req 25653248->25657343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.542351:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2ec6b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25653248->25657343] (req 25653248->25657343) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.542371:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2ec6b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25653248->25657343] (req 25653248->25657343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.542378:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2ec6b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25653248->25657343] (req 25653248->25657343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760753.542554:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4133800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512663040:12345-192.168.203.56@tcp:4:dd.0 Request processed in 31023us (32301us total) trans 4294973582 rc 0/0 00000100:00100000:2.0:1789760753.542566:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6293 00000100:00100000:2.0:1789760753.542576:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 733018150919 time 733075729057 state 0x2 ref 0 00000100:00100000:2.0:1789760753.548877:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760753.549021:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512663168 00000100:00100000:3.0:1789760753.549031:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512663168 00000100:00100000:3.0:1789760753.549088:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 733018150919 time 733075729057 state 0x0 ref 1 00000100:00100000:3.0:1789760753.549104:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 733075989473 time 733075729057 state 0x1 ref 1 00000100:00100000:3.0:1789760753.549183:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 733075989473 time 733113481914 state 0x0 ref 1 00000100:00100000:3.0:1789760753.549197:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5307800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512663168:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760753.549264:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25657344, mtime: 0x6aad94f1, atime: 0x0, ctime: 0x6aad94f1, blocks: 0xc807 00010000:00010000:3.0:1789760753.549273:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2ec72 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.549290:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=733s v=(0 0) 00010000:00010000:3.0:1789760753.549295:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760753.549302:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2ec72 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25657344->25661439] (req 25657344->25661439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.549325:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2ec72 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25657344->25661439] (req 25657344->25661439) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760753.581993:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973583 is committed 00002000:00010000:3.0:1789760753.582106:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25657344 -> 25661440 00010000:00010000:3.0:1789760753.582122:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2ec72 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25657344->25661439] (req 25657344->25661439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.582136:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2ec72 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25657344->25661439] (req 25657344->25661439) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.582145:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2ec72 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25657344->25661439] (req 25657344->25661439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.582153:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760753.582158:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2ec72 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25657344->25661439] (req 25657344->25661439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.582167:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2ec72 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25657344->25661439] (req 25657344->25661439) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.582186:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2ec72 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25657344->25661439] (req 25657344->25661439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.582194:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2ec72 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25657344->25661439] (req 25657344->25661439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760753.582283:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5307800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512663168:12345-192.168.203.56@tcp:4:dd.0 Request processed in 33084us (33414us total) trans 4294973583 rc 0/0 00000100:00100000:3.0:1789760753.582293:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6294 00000100:00100000:3.0:1789760753.582302:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 733075989473 time 733113481914 state 0x2 ref 0 00000100:00100000:3.0:1789760753.584994:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760753.585075:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512663296 00000100:00100000:2.0:1789760753.585083:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512663296 00000100:00100000:2.0:1789760753.585130:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 733075989473 time 733113481914 state 0x0 ref 1 00000100:00100000:2.0:1789760753.585145:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 733113742330 time 733113481914 state 0x1 ref 1 00000100:00100000:2.0:1789760753.585171:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 733113742330 time 733149471369 state 0x0 ref 1 00000100:00100000:2.0:1789760753.585182:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac3353800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512663296:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760753.585224:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25661440, mtime: 0x6aad94f1, atime: 0x0, ctime: 0x6aad94f1, blocks: 0xc807 00010000:00010000:2.0:1789760753.585233:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2ec79 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.585264:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=733s v=(0 0) 00010000:00010000:2.0:1789760753.585268:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760753.585274:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2ec79 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25661440->25665535] (req 25661440->25665535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.585287:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2ec79 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25661440->25665535] (req 25661440->25665535) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760753.609726:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25661440 -> 25665536 00010000:00010000:2.0:1789760753.609745:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2ec79 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25661440->25665535] (req 25661440->25665535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.609759:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2ec79 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25661440->25665535] (req 25661440->25665535) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.609768:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2ec79 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25661440->25665535] (req 25661440->25665535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.609775:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760753.609781:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2ec79 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25661440->25665535] (req 25661440->25665535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.609790:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2ec79 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25661440->25665535] (req 25661440->25665535) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.609809:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2ec79 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25661440->25665535] (req 25661440->25665535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.609818:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2ec79 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25661440->25665535] (req 25661440->25665535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760753.609912:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac3353800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2359:x1876699512663296:12345-192.168.203.56@tcp:4:dd.0 Request processed in 24726us (24922us total) trans 4294973584 rc 0/0 00000100:00100000:2.0:1789760753.611035:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6295 00000100:00100000:2.0:1789760753.611045:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 733113742330 time 733149471369 state 0x2 ref 0 00000001:00080000:0.0:1789760753.612155:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973584 is committed 00000100:00100000:2.0:1789760753.612884:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760753.613063:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512663424 00000100:00100000:3.0:1789760753.613072:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512663424 00000100:00100000:3.0:1789760753.613120:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 733113742330 time 733149471369 state 0x0 ref 1 00000100:00100000:3.0:1789760753.613134:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 733149731785 time 733149471369 state 0x1 ref 1 00000100:00100000:3.0:1789760753.613152:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 733149731785 time 733177452545 state 0x0 ref 1 00000100:00100000:3.0:1789760753.613164:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febea4b4050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512663424:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760753.613207:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25665536, mtime: 0x6aad94f1, atime: 0x0, ctime: 0x6aad94f1, blocks: 0xc807 00010000:00010000:3.0:1789760753.613215:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2ec80 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.613230:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=733s v=(0 0) 00010000:00010000:3.0:1789760753.613234:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760753.613240:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2ec80 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25665536->25669631] (req 25665536->25669631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.613264:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2ec80 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25665536->25669631] (req 25665536->25669631) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760753.643370:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25665536 -> 25669632 00010000:00010000:2.0:1789760753.643388:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2ec80 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25665536->25669631] (req 25665536->25669631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.643401:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2ec80 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25665536->25669631] (req 25665536->25669631) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.643411:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2ec80 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25665536->25669631] (req 25665536->25669631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.643418:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760753.643423:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2ec80 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25665536->25669631] (req 25665536->25669631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.643433:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2ec80 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25665536->25669631] (req 25665536->25669631) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.643458:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2ec80 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25665536->25669631] (req 25665536->25669631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.643466:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2ec80 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25665536->25669631] (req 25665536->25669631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760753.643553:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febea4b4050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2358:x1876699512663424:12345-192.168.203.56@tcp:4:dd.0 Request processed in 30387us (30669us total) trans 4294973585 rc 0/0 00000100:00100000:2.0:1789760753.643562:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6296 00000100:00100000:2.0:1789760753.643571:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 733149731785 time 733177452545 state 0x2 ref 0 00000001:00080000:0.0:1789760753.645180:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973585 is committed 00000100:00100000:1.0:1789760753.646349:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760753.646433:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512663552 00000100:00100000:3.0:1789760753.646442:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512663552 00000100:00100000:3.0:1789760753.646489:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 733149731785 time 733177452545 state 0x0 ref 1 00000100:00100000:3.0:1789760753.646503:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 733177712961 time 733177452545 state 0x1 ref 1 00000100:00100000:3.0:1789760753.646566:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 733177712961 time 733210865116 state 0x0 ref 1 00000100:00100000:3.0:1789760753.646578:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4130000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512663552:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760753.646622:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25669632, mtime: 0x6aad94f1, atime: 0x0, ctime: 0x6aad94f1, blocks: 0xc807 00010000:00010000:3.0:1789760753.646631:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2ec87 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.646655:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=733s v=(0 0) 00010000:00010000:3.0:1789760753.646660:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760753.646666:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2ec87 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25669632->25673727] (req 25669632->25673727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.646680:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2ec87 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25669632->25673727] (req 25669632->25673727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760753.680108:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25669632 -> 25673728 00010000:00010000:2.0:1789760753.680126:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2ec87 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25669632->25673727] (req 25669632->25673727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.680141:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2ec87 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25669632->25673727] (req 25669632->25673727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.680150:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2ec87 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25669632->25673727] (req 25669632->25673727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.680157:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760753.680162:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2ec87 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25669632->25673727] (req 25669632->25673727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.680178:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2ec87 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25669632->25673727] (req 25669632->25673727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.680195:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2ec87 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25669632->25673727] (req 25669632->25673727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.680202:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2ec87 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25669632->25673727] (req 25669632->25673727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760753.680297:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4130000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2358:x1876699512663552:12345-192.168.203.56@tcp:4:dd.0 Request processed in 33717us (33951us total) trans 4294973586 rc 0/0 00000100:00100000:2.0:1789760753.680307:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6297 00000100:00100000:2.0:1789760753.680316:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 733177712961 time 733210865116 state 0x2 ref 0 00000001:00080000:0.0:1789760753.680409:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973586 is committed 00000100:00100000:1.0:1789760753.686616:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760753.686692:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512663680 00000100:00100000:2.0:1789760753.686738:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512663680 00000100:00100000:2.0:1789760753.686784:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 733177712961 time 733210865116 state 0x0 ref 1 00000100:00100000:2.0:1789760753.686799:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 733211125532 time 733210865116 state 0x1 ref 1 00000100:00100000:2.0:1789760753.686818:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 733211125532 time 733251117841 state 0x0 ref 1 00000100:00100000:2.0:1789760753.686829:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4133480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512663680:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760753.686872:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25673728, mtime: 0x6aad94f1, atime: 0x0, ctime: 0x6aad94f1, blocks: 0xc807 00010000:00010000:2.0:1789760753.686880:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2ec8e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.686897:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=733s v=(0 0) 00010000:00010000:2.0:1789760753.686901:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760753.686907:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2ec8e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25673728->25677823] (req 25673728->25677823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.686920:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2ec8e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25673728->25677823] (req 25673728->25677823) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760753.722195:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973587 is committed 00002000:00010000:2.0:1789760753.722363:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25673728 -> 25677824 00010000:00010000:2.0:1789760753.722380:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2ec8e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25673728->25677823] (req 25673728->25677823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.722393:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2ec8e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25673728->25677823] (req 25673728->25677823) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.722402:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2ec8e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25673728->25677823] (req 25673728->25677823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.722409:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760753.722414:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2ec8e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25673728->25677823] (req 25673728->25677823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.722423:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2ec8e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25673728->25677823] (req 25673728->25677823) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.722441:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2ec8e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25673728->25677823] (req 25673728->25677823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.722448:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2ec8e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25673728->25677823] (req 25673728->25677823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760753.722588:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4133480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512663680:12345-192.168.203.56@tcp:4:dd.0 Request processed in 35754us (35974us total) trans 4294973587 rc 0/0 00000100:00100000:2.0:1789760753.722599:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6298 00000100:00100000:2.0:1789760753.722609:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 733211125532 time 733251117841 state 0x2 ref 0 00000100:00100000:1.0:1789760753.728652:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760753.729571:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512663808 00000100:00100000:2.0:1789760753.729582:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512663808 00000100:00100000:2.0:1789760753.729634:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 733211125532 time 733251117841 state 0x0 ref 1 00000100:00100000:2.0:1789760753.729649:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 733251378257 time 733251117841 state 0x1 ref 1 00000100:00100000:2.0:1789760753.729679:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 733251378257 time 733293978522 state 0x0 ref 1 00000100:00100000:2.0:1789760753.729690:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4130700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512663808:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760753.729765:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25677824, mtime: 0x6aad94f1, atime: 0x0, ctime: 0x6aad94f1, blocks: 0xc807 00010000:00010000:2.0:1789760753.729775:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2ec95 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.729792:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=733s v=(0 0) 00010000:00010000:2.0:1789760753.729796:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760753.729802:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2ec95 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25677824->25681919] (req 25677824->25681919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.729822:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2ec95 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25677824->25681919] (req 25677824->25681919) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760753.781345:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973588 is committed 00002000:00010000:2.0:1789760753.781420:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25677824 -> 25681920 00010000:00010000:2.0:1789760753.781435:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2ec95 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25677824->25681919] (req 25677824->25681919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.781449:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2ec95 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25677824->25681919] (req 25677824->25681919) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.781459:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2ec95 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25677824->25681919] (req 25677824->25681919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.781466:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760753.781471:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2ec95 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25677824->25681919] (req 25677824->25681919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.781480:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2ec95 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25677824->25681919] (req 25677824->25681919) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.781498:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2ec95 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25677824->25681919] (req 25677824->25681919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.781505:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2ec95 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25677824->25681919] (req 25677824->25681919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760753.781630:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4130700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512663808:12345-192.168.203.56@tcp:4:dd.0 Request processed in 51935us (52982us total) trans 4294973588 rc 0/0 00000100:00100000:2.0:1789760753.781640:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6299 00000100:00100000:2.0:1789760753.781650:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 733251378257 time 733293978522 state 0x2 ref 0 00000100:00100000:1.0:1789760753.786668:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760753.787866:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512663936 00000100:00100000:2.0:1789760753.787878:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512663936 00000100:00100000:2.0:1789760753.788011:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 733251378257 time 733293978522 state 0x0 ref 1 00000100:00100000:2.0:1789760753.788027:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 733294238938 time 733293978522 state 0x1 ref 1 00000100:00100000:2.0:1789760753.788070:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 733294238938 time 733352368953 state 0x0 ref 1 00000100:00100000:2.0:1789760753.788083:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4131180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512663936:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760753.788130:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25681920, mtime: 0x6aad94f1, atime: 0x0, ctime: 0x6aad94f1, blocks: 0xc807 00010000:00010000:2.0:1789760753.788139:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2ec9c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.788154:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=733s v=(0 0) 00010000:00010000:2.0:1789760753.788159:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760753.788165:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2ec9c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25681920->25686015] (req 25681920->25686015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.788178:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2ec9c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25681920->25686015] (req 25681920->25686015) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760753.854072:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973589 is committed 00002000:00010000:2.0:1789760753.854213:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25681920 -> 25686016 00010000:00010000:2.0:1789760753.854229:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2ec9c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25681920->25686015] (req 25681920->25686015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.854252:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2ec9c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25681920->25686015] (req 25681920->25686015) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.854263:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2ec9c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25681920->25686015] (req 25681920->25686015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.854270:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760753.854276:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2ec9c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25681920->25686015] (req 25681920->25686015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.854285:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2ec9c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25681920->25686015] (req 25681920->25686015) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.854310:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2ec9c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25681920->25686015] (req 25681920->25686015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.854319:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2ec9c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25681920->25686015] (req 25681920->25686015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760753.854405:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4131180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512663936:12345-192.168.203.56@tcp:4:dd.0 Request processed in 66317us (67737us total) trans 4294973589 rc 0/0 00000100:00100000:2.0:1789760753.854418:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6300 00000100:00100000:2.0:1789760753.854427:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 733294238938 time 733352368953 state 0x2 ref 0 00000100:00100000:1.0:1789760753.863261:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760753.865540:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512664064 00000100:00100000:2.0:1789760753.865552:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512664064 00000100:00100000:2.0:1789760753.865609:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 733294238938 time 733352368953 state 0x0 ref 1 00000100:00100000:2.0:1789760753.865623:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 733352629369 time 733352368953 state 0x1 ref 1 00000100:00100000:2.0:1789760753.865714:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 733352629369 time 733430013795 state 0x0 ref 1 00000100:00100000:2.0:1789760753.865727:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4130a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512664064:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760753.865777:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25686016, mtime: 0x6aad94f1, atime: 0x0, ctime: 0x6aad94f1, blocks: 0xc807 00010000:00010000:2.0:1789760753.865786:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2eca3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.865800:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=733s v=(0 0) 00010000:00010000:2.0:1789760753.865804:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760753.865821:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2eca3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25686016->25690111] (req 25686016->25690111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.865835:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2eca3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25686016->25690111] (req 25686016->25690111) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760753.899383:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973590 is committed 00002000:00010000:2.0:1789760753.899528:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25686016 -> 25690112 00010000:00010000:2.0:1789760753.899543:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2eca3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25686016->25690111] (req 25686016->25690111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.899557:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2eca3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25686016->25690111] (req 25686016->25690111) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.899577:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2eca3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25686016->25690111] (req 25686016->25690111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.899585:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760753.899590:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2eca3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25686016->25690111] (req 25686016->25690111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.899600:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2eca3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25686016->25690111] (req 25686016->25690111) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.899618:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2eca3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25686016->25690111] (req 25686016->25690111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.899625:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2eca3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25686016->25690111] (req 25686016->25690111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760753.899746:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4130a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512664064:12345-192.168.203.56@tcp:4:dd.0 Request processed in 34016us (36499us total) trans 4294973590 rc 0/0 00000100:00100000:2.0:1789760753.899756:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6301 00000100:00100000:2.0:1789760753.899765:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 733352629369 time 733430013795 state 0x2 ref 0 00000100:00100000:1.0:1789760753.904056:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760753.904142:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512664192 00000100:00100000:2.0:1789760753.904150:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512664192 00000100:00100000:2.0:1789760753.904198:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 733352629369 time 733430013795 state 0x0 ref 1 00000100:00100000:2.0:1789760753.904212:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 733430274211 time 733430013795 state 0x1 ref 1 00000100:00100000:2.0:1789760753.904240:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 733430274211 time 733468539869 state 0x0 ref 1 00000100:00100000:2.0:1789760753.904260:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4132d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512664192:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760753.904399:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25690112, mtime: 0x6aad94f1, atime: 0x0, ctime: 0x6aad94f1, blocks: 0xc807 00010000:00010000:2.0:1789760753.904410:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2ecaa lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.904425:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=733s v=(0 0) 00010000:00010000:2.0:1789760753.904430:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760753.904436:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2ecaa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25690112->25694207] (req 25690112->25694207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.904450:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2ecaa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25690112->25694207] (req 25690112->25694207) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760753.941553:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973591 is committed 00002000:00010000:2.0:1789760753.943453:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25690112 -> 25694208 00010000:00010000:2.0:1789760753.943473:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2ecaa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25690112->25694207] (req 25690112->25694207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.943487:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2ecaa lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25690112->25694207] (req 25690112->25694207) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.943503:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2ecaa lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25690112->25694207] (req 25690112->25694207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.943510:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760753.943515:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2ecaa lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25690112->25694207] (req 25690112->25694207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.943524:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2ecaa lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25690112->25694207] (req 25690112->25694207) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.943543:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2ecaa lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25690112->25694207] (req 25690112->25694207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.943551:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2ecaa lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25690112->25694207] (req 25690112->25694207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760753.943649:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4132d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512664192:12345-192.168.203.56@tcp:4:dd.0 Request processed in 39386us (39596us total) trans 4294973591 rc 0/0 00000100:00100000:2.0:1789760753.943659:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6302 00000100:00100000:2.0:1789760753.943669:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 733430274211 time 733468539869 state 0x2 ref 0 00000100:00100000:0.0:1789760753.947565:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760753.947612:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512664320 00000100:00100000:2.0:1789760753.947622:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512664320 00000100:00100000:2.0:1789760753.947675:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 733430274211 time 733468539869 state 0x0 ref 1 00000100:00100000:2.0:1789760753.947691:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 733468800285 time 733468539869 state 0x1 ref 1 00000100:00100000:2.0:1789760753.947747:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 733468800285 time 733512046759 state 0x0 ref 1 00000100:00100000:2.0:1789760753.947759:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac531f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512664320:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760753.947808:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25694208, mtime: 0x6aad94f1, atime: 0x0, ctime: 0x6aad94f1, blocks: 0xc807 00010000:00010000:2.0:1789760753.947817:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2ecb1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.947831:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=733s v=(0 0) 00010000:00010000:2.0:1789760753.947836:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760753.947843:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2ecb1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25694208->25698303] (req 25694208->25698303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.947857:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2ecb1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25694208->25698303] (req 25694208->25698303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760753.986556:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973592 is committed 00002000:00010000:2.0:1789760753.986716:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25694208 -> 25698304 00010000:00010000:2.0:1789760753.986731:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2ecb1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25694208->25698303] (req 25694208->25698303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.986744:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2ecb1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25694208->25698303] (req 25694208->25698303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.986753:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2ecb1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25694208->25698303] (req 25694208->25698303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.986916:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760753.986923:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2ecb1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25694208->25698303] (req 25694208->25698303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.986933:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2ecb1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25694208->25698303] (req 25694208->25698303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.986954:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2ecb1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25694208->25698303] (req 25694208->25698303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.986962:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2ecb1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25694208->25698303] (req 25694208->25698303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760753.987068:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac531f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512664320:12345-192.168.203.56@tcp:4:dd.0 Request processed in 39304us (39505us total) trans 4294973592 rc 0/0 00000100:00100000:2.0:1789760753.987078:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6303 00000100:00100000:2.0:1789760753.987088:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 733468800285 time 733512046759 state 0x2 ref 0 00000100:00100000:1.0:1789760753.994377:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760753.994487:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512664448 00000100:00100000:2.0:1789760753.994497:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512664448 00000100:00100000:2.0:1789760753.994549:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 733468800285 time 733512046759 state 0x0 ref 1 00000100:00100000:2.0:1789760753.994565:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 733512307175 time 733512046759 state 0x1 ref 1 00000100:00100000:2.0:1789760753.994596:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 733512307175 time 733558895447 state 0x0 ref 1 00000100:00100000:2.0:1789760753.994610:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4132300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512664448:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760753.994657:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25698304, mtime: 0x6aad94f1, atime: 0x0, ctime: 0x6aad94f1, blocks: 0xc807 00010000:00010000:2.0:1789760753.994667:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2ecb8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.994682:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=733s v=(0 0) 00010000:00010000:2.0:1789760753.994686:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760753.994692:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2ecb8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25698304->25702399] (req 25698304->25702399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.994734:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2ecb8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25698304->25702399] (req 25698304->25702399) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.034965:0:5836:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:2.0:1789760754.041316:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25698304 -> 25702400 00010000:00010000:2.0:1789760754.041335:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2ecb8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25698304->25702399] (req 25698304->25702399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.041350:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2ecb8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25698304->25702399] (req 25698304->25702399) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.041359:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2ecb8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25698304->25702399] (req 25698304->25702399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.041366:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760754.041371:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2ecb8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25698304->25702399] (req 25698304->25702399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.041380:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2ecb8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25698304->25702399] (req 25698304->25702399) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.041407:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2ecb8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25698304->25702399] (req 25698304->25702399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.041415:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2ecb8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25698304->25702399] (req 25698304->25702399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760754.041513:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4132300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2360:x1876699512664448:12345-192.168.203.56@tcp:4:dd.0 Request processed in 46901us (47139us total) trans 4294973593 rc 0/0 00000100:00100000:2.0:1789760754.041524:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6304 00000100:00100000:2.0:1789760754.041535:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 733512307175 time 733558895447 state 0x2 ref 0 00000001:00080000:3.0:1789760754.041610:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973593 is committed 00000100:00100000:1.0:1789760754.043872:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760754.043948:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512664576 00000100:00100000:2.0:1789760754.043957:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512664576 00000100:00100000:2.0:1789760754.044008:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 733512307175 time 733558895447 state 0x0 ref 1 00000100:00100000:2.0:1789760754.044024:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 733559155863 time 733558895447 state 0x1 ref 1 00000100:00100000:2.0:1789760754.044059:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 733559155863 time 733608359530 state 0x0 ref 1 00000100:00100000:2.0:1789760754.044071:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4131f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512664576:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760754.044116:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25702400, mtime: 0x6aad94f1, atime: 0x0, ctime: 0x6aad94f1, blocks: 0xc807 00010000:00010000:2.0:1789760754.044124:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2ecbf lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.044138:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=733s v=(0 0) 00010000:00010000:2.0:1789760754.044143:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760754.044149:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2ecbf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25702400->25706495] (req 25702400->25706495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.044161:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2ecbf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25702400->25706495] (req 25702400->25706495) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760754.091321:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973594 is committed 00002000:00010000:0.0:1789760754.091457:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25702400 -> 25706496 00010000:00010000:0.0:1789760754.091472:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2ecbf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25702400->25706495] (req 25702400->25706495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.091484:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2ecbf lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25702400->25706495] (req 25702400->25706495) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.091494:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2ecbf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25702400->25706495] (req 25702400->25706495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.091502:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760754.091507:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2ecbf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25702400->25706495] (req 25702400->25706495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.091516:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2ecbf lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25702400->25706495] (req 25702400->25706495) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.091534:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2ecbf lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25702400->25706495] (req 25702400->25706495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.091542:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2ecbf lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25702400->25706495] (req 25702400->25706495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760754.091778:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4131f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512664576:12345-192.168.203.56@tcp:4:dd.0 Request processed in 47704us (47909us total) trans 4294973594 rc 0/0 00000100:00100000:0.0:1789760754.091789:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6305 00000100:00100000:0.0:1789760754.091799:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 733559155863 time 733608359530 state 0x2 ref 0 00000100:00100000:2.0:1789760754.096725:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760754.096802:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512664704 00000100:00100000:3.0:1789760754.096813:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512664704 00000100:00100000:3.0:1789760754.096866:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 733559155863 time 733608359530 state 0x0 ref 1 00000100:00100000:3.0:1789760754.096881:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 733608619946 time 733608359530 state 0x1 ref 1 00000100:00100000:3.0:1789760754.096909:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 733608619946 time 733661209021 state 0x0 ref 1 00000100:00100000:3.0:1789760754.096933:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5305f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512664704:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760754.096982:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25706496, mtime: 0x6aad94f1, atime: 0x0, ctime: 0x6aad94f1, blocks: 0xc807 00010000:00010000:3.0:1789760754.096991:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2ecc6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.097006:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=733s v=(0 0) 00010000:00010000:3.0:1789760754.097010:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760754.097017:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2ecc6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25706496->25710591] (req 25706496->25710591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.097032:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2ecc6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25706496->25710591] (req 25706496->25710591) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760754.129727:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25706496 -> 25710592 00010000:00010000:3.0:1789760754.129747:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2ecc6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25706496->25710591] (req 25706496->25710591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.129761:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2ecc6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25706496->25710591] (req 25706496->25710591) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.129771:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2ecc6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25706496->25710591] (req 25706496->25710591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.129778:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760754.129783:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2ecc6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25706496->25710591] (req 25706496->25710591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.129792:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2ecc6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25706496->25710591] (req 25706496->25710591) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.129812:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2ecc6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25706496->25710591] (req 25706496->25710591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.129819:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2ecc6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25706496->25710591] (req 25706496->25710591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760754.129928:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5305f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2361:x1876699512664704:12345-192.168.203.56@tcp:4:dd.0 Request processed in 32981us (33196us total) trans 4294973595 rc 0/0 00000100:00100000:3.0:1789760754.129939:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6306 00000100:00100000:3.0:1789760754.129948:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 733608619946 time 733661209021 state 0x2 ref 0 00000001:00080000:0.0:1789760754.130349:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973595 is committed 00000100:00100000:1.0:1789760754.136186:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760754.136998:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512664832 00000100:00100000:2.0:1789760754.137009:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512664832 00000100:00100000:2.0:1789760754.137068:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 733608619946 time 733661209021 state 0x0 ref 1 00000100:00100000:2.0:1789760754.137084:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 733661469437 time 733661209021 state 0x1 ref 1 00000100:00100000:2.0:1789760754.137118:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 733661469437 time 733701417416 state 0x0 ref 1 00000100:00100000:2.0:1789760754.137131:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4130380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512664832:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760754.137179:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25710592, mtime: 0x6aad94f1, atime: 0x0, ctime: 0x6aad94f1, blocks: 0xc807 00010000:00010000:2.0:1789760754.137188:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2eccd lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.137204:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=733s v=(0 0) 00010000:00010000:2.0:1789760754.137208:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760754.137214:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2eccd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25710592->25714687] (req 25710592->25714687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.137228:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2eccd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25710592->25714687] (req 25710592->25714687) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760754.177678:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973596 is committed 00002000:00010000:1.0:1789760754.177871:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25710592 -> 25714688 00010000:00010000:1.0:1789760754.177887:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2eccd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25710592->25714687] (req 25710592->25714687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.177909:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2eccd lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25710592->25714687] (req 25710592->25714687) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.177919:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2eccd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25710592->25714687] (req 25710592->25714687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.177927:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760754.177932:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2eccd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25710592->25714687] (req 25710592->25714687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.177940:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2eccd lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25710592->25714687] (req 25710592->25714687) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.177970:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2eccd lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25710592->25714687] (req 25710592->25714687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.177983:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2eccd lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25710592->25714687] (req 25710592->25714687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760754.178059:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4130380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512664832:12345-192.168.203.56@tcp:4:dd.0 Request processed in 40925us (41877us total) trans 4294973596 rc 0/0 00000100:00100000:1.0:1789760754.178069:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6307 00000100:00100000:1.0:1789760754.178079:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 733661469437 time 733701417416 state 0x2 ref 0 00000100:00100000:1.0:1789760754.181050:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760754.181182:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512664960 00000100:00100000:0.0:1789760754.181191:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512664960 00000100:00100000:0.0:1789760754.181255:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 733661469437 time 733701417416 state 0x0 ref 1 00000100:00100000:0.0:1789760754.181398:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 733701677832 time 733701417416 state 0x1 ref 1 00000100:00100000:0.0:1789760754.181422:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 733701677832 time 733745723260 state 0x0 ref 1 00000100:00100000:0.0:1789760754.181433:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4133b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512664960:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760754.181473:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25714688, mtime: 0x6aad94f1, atime: 0x0, ctime: 0x6aad94f1, blocks: 0xc807 00010000:00010000:0.0:1789760754.181479:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2ecd4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.181490:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=733s v=(0 0) 00010000:00010000:0.0:1789760754.181494:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760754.181498:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2ecd4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25714688->25718783] (req 25714688->25718783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.181512:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2ecd4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25714688->25718783] (req 25714688->25718783) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760754.223973:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973597 is committed 00002000:00010000:1.0:1789760754.224092:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25714688 -> 25718784 00010000:00010000:1.0:1789760754.224107:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2ecd4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25714688->25718783] (req 25714688->25718783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.224120:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2ecd4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25714688->25718783] (req 25714688->25718783) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.224129:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2ecd4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25714688->25718783] (req 25714688->25718783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.224137:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760754.224142:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2ecd4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25714688->25718783] (req 25714688->25718783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.224150:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2ecd4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25714688->25718783] (req 25714688->25718783) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.224177:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2ecd4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25714688->25718783] (req 25714688->25718783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.224186:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2ecd4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25714688->25718783] (req 25714688->25718783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760754.224293:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4133b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512664960:12345-192.168.203.56@tcp:4:dd.0 Request processed in 42857us (43247us total) trans 4294973597 rc 0/0 00000100:00100000:1.0:1789760754.224304:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6308 00000100:00100000:1.0:1789760754.224313:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 733701677832 time 733745723260 state 0x2 ref 0 00000100:00100000:2.0:1789760754.230027:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760754.230116:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512665088 00000100:00100000:3.0:1789760754.230125:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512665088 00000100:00100000:3.0:1789760754.230179:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 733701677832 time 733745723260 state 0x0 ref 1 00000100:00100000:3.0:1789760754.230195:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 733745983676 time 733745723260 state 0x1 ref 1 00000100:00100000:3.0:1789760754.230228:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 733745983676 time 733794527652 state 0x0 ref 1 00000100:00100000:3.0:1789760754.230252:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5307480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512665088:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760754.230299:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25718784, mtime: 0x6aad94f1, atime: 0x0, ctime: 0x6aad94f1, blocks: 0xc807 00010000:00010000:3.0:1789760754.230309:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2ecdb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.230324:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=733s v=(0 0) 00010000:00010000:3.0:1789760754.230328:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760754.230334:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2ecdb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25718784->25722879] (req 25718784->25722879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.230347:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2ecdb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25718784->25722879] (req 25718784->25722879) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760754.273893:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973598 is committed 00002000:00010000:3.0:1789760754.274097:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25718784 -> 25722880 00010000:00010000:3.0:1789760754.274114:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2ecdb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25718784->25722879] (req 25718784->25722879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.274128:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2ecdb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25718784->25722879] (req 25718784->25722879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.274138:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2ecdb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25718784->25722879] (req 25718784->25722879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.274146:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760754.274151:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2ecdb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25718784->25722879] (req 25718784->25722879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.274161:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2ecdb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25718784->25722879] (req 25718784->25722879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.274182:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2ecdb lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25718784->25722879] (req 25718784->25722879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.274190:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2ecdb lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25718784->25722879] (req 25718784->25722879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760754.274345:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5307480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512665088:12345-192.168.203.56@tcp:4:dd.0 Request processed in 44101us (44322us total) trans 4294973598 rc 0/0 00000100:00100000:3.0:1789760754.274355:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6309 00000100:00100000:3.0:1789760754.274364:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 733745983676 time 733794527652 state 0x2 ref 0 00000100:00100000:2.0:1789760754.278528:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760754.278634:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512665216 00000100:00100000:1.0:1789760754.278644:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512665216 00000100:00100000:1.0:1789760754.278732:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 733745983676 time 733794527652 state 0x0 ref 1 00000100:00100000:1.0:1789760754.278748:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 733794788068 time 733794527652 state 0x1 ref 1 00000100:00100000:1.0:1789760754.278777:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 733794788068 time 733843076790 state 0x0 ref 1 00000100:00100000:1.0:1789760754.278789:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5304700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512665216:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760754.278843:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25722880, mtime: 0x6aad94f1, atime: 0x0, ctime: 0x6aad94f1, blocks: 0xc807 00010000:00010000:1.0:1789760754.278852:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2ece2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.278868:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=733s v=(0 0) 00010000:00010000:1.0:1789760754.278873:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760754.278880:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2ece2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25722880->25726975] (req 25722880->25726975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.278896:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2ece2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25722880->25726975] (req 25722880->25726975) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760754.320251:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973599 is committed 00002000:00010000:1.0:1789760754.320427:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25722880 -> 25726976 00010000:00010000:1.0:1789760754.320443:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2ece2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25722880->25726975] (req 25722880->25726975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.320455:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2ece2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25722880->25726975] (req 25722880->25726975) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.320465:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2ece2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25722880->25726975] (req 25722880->25726975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.320473:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760754.320478:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2ece2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25722880->25726975] (req 25722880->25726975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.320487:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2ece2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25722880->25726975] (req 25722880->25726975) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.320508:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2ece2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25722880->25726975] (req 25722880->25726975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.320516:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2ece2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25722880->25726975] (req 25722880->25726975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760754.320618:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5304700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512665216:12345-192.168.203.56@tcp:4:dd.0 Request processed in 41826us (42094us total) trans 4294973599 rc 0/0 00000100:00100000:1.0:1789760754.320628:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6310 00000100:00100000:1.0:1789760754.322575:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 733794788068 time 733843076790 state 0x2 ref 0 00000100:00100000:1.0:1789760754.323661:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760754.323762:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512665344 00000100:00100000:3.0:1789760754.323772:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512665344 00000100:00100000:3.0:1789760754.323823:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 733794788068 time 733843076790 state 0x0 ref 1 00000100:00100000:3.0:1789760754.323838:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 733843337206 time 733843076790 state 0x1 ref 1 00000100:00100000:3.0:1789760754.323867:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 733843337206 time 733888167118 state 0x0 ref 1 00000100:00100000:3.0:1789760754.323879:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4133800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512665344:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760754.323935:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25726976, mtime: 0x6aad94f1, atime: 0x0, ctime: 0x6aad94f1, blocks: 0xc807 00010000:00010000:3.0:1789760754.323945:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2ece9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.323961:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=733s v=(0 0) 00010000:00010000:3.0:1789760754.323966:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760754.323973:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2ece9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25726976->25731071] (req 25726976->25731071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.323988:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2ece9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25726976->25731071] (req 25726976->25731071) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760754.366215:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25726976 -> 25731072 00010000:00010000:1.0:1789760754.366234:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2ece9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25726976->25731071] (req 25726976->25731071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.366257:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2ece9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25726976->25731071] (req 25726976->25731071) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.366267:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2ece9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25726976->25731071] (req 25726976->25731071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.366274:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760754.366279:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2ece9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25726976->25731071] (req 25726976->25731071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.366288:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2ece9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25726976->25731071] (req 25726976->25731071) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.366305:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2ece9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25726976->25731071] (req 25726976->25731071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.366318:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2ece9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25726976->25731071] (req 25726976->25731071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760754.366426:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4133800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2361:x1876699512665344:12345-192.168.203.56@tcp:4:dd.0 Request processed in 42544us (42765us total) trans 4294973600 rc 0/0 00000100:00100000:1.0:1789760754.366436:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6311 00000100:00100000:1.0:1789760754.366447:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 733843337206 time 733888167118 state 0x2 ref 0 00000001:00080000:0.0:1789760754.367571:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973600 is committed 00000100:00100000:3.0:1789760754.369757:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760754.369852:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512665472 00000100:00100000:2.0:1789760754.369861:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512665472 00000100:00100000:2.0:1789760754.369920:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 733843337206 time 733888167118 state 0x0 ref 1 00000100:00100000:2.0:1789760754.369935:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 733888427534 time 733888167118 state 0x1 ref 1 00000100:00100000:2.0:1789760754.369961:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 733888427534 time 733934260843 state 0x0 ref 1 00000100:00100000:2.0:1789760754.369972:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac3352a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512665472:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760754.370018:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25731072, mtime: 0x6aad94f1, atime: 0x0, ctime: 0x6aad94f1, blocks: 0xc807 00010000:00010000:2.0:1789760754.370027:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2ecf0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.370041:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=733s v=(0 0) 00010000:00010000:2.0:1789760754.370045:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760754.370051:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2ecf0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25731072->25735167] (req 25731072->25735167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.370064:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2ecf0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25731072->25735167] (req 25731072->25735167) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760754.415042:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973601 is committed 00002000:00010000:3.0:1789760754.415200:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25731072 -> 25735168 00010000:00010000:3.0:1789760754.415217:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2ecf0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25731072->25735167] (req 25731072->25735167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.415231:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2ecf0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25731072->25735167] (req 25731072->25735167) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.415251:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2ecf0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25731072->25735167] (req 25731072->25735167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.415258:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760754.415263:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2ecf0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25731072->25735167] (req 25731072->25735167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.415273:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2ecf0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25731072->25735167] (req 25731072->25735167) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.415296:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2ecf0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25731072->25735167] (req 25731072->25735167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.415305:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2ecf0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25731072->25735167] (req 25731072->25735167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760754.415407:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac3352a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512665472:12345-192.168.203.56@tcp:4:dd.0 Request processed in 45430us (45652us total) trans 4294973601 rc 0/0 00000100:00100000:3.0:1789760754.415418:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6312 00000100:00100000:3.0:1789760754.415429:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 733888427534 time 733934260843 state 0x2 ref 0 00000100:00100000:3.0:1789760754.418127:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760754.418377:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512665600 00000100:00100000:2.0:1789760754.418388:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512665600 00000100:00100000:2.0:1789760754.418443:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 733888427534 time 733934260843 state 0x0 ref 1 00000100:00100000:2.0:1789760754.418459:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 733934521259 time 733934260843 state 0x1 ref 1 00000100:00100000:2.0:1789760754.418494:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 733934521259 time 733982793053 state 0x0 ref 1 00000100:00100000:2.0:1789760754.418509:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac3353480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512665600:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760754.418561:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25735168, mtime: 0x6aad94f1, atime: 0x0, ctime: 0x6aad94f1, blocks: 0xc807 00010000:00010000:2.0:1789760754.418570:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2ecf7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.418586:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=733s v=(0 0) 00010000:00010000:2.0:1789760754.418591:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760754.418597:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2ecf7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25735168->25739263] (req 25735168->25739263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.418610:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2ecf7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25735168->25739263] (req 25735168->25739263) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760754.485414:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25735168 -> 25739264 00010000:00010000:3.0:1789760754.485433:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2ecf7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25735168->25739263] (req 25735168->25739263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.485447:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2ecf7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25735168->25739263] (req 25735168->25739263) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.485456:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2ecf7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25735168->25739263] (req 25735168->25739263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.485464:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760754.485469:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2ecf7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25735168->25739263] (req 25735168->25739263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.485482:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2ecf7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25735168->25739263] (req 25735168->25739263) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.485503:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2ecf7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25735168->25739263] (req 25735168->25739263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.485511:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2ecf7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25735168->25739263] (req 25735168->25739263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760754.485609:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac3353480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2361:x1876699512665600:12345-192.168.203.56@tcp:4:dd.0 Request processed in 67098us (67487us total) trans 4294973602 rc 0/0 00000100:00100000:3.0:1789760754.485619:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6313 00000100:00100000:3.0:1789760754.485628:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 733934521259 time 733982793053 state 0x2 ref 0 00000001:00080000:0.0:1789760754.488470:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973602 is committed 00000100:00100000:2.0:1789760754.488872:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760754.488959:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512665728 00000100:00100000:1.0:1789760754.488968:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512665728 00000100:00100000:1.0:1789760754.489020:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 733934521259 time 733982793053 state 0x0 ref 1 00000100:00100000:1.0:1789760754.489034:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 733983053469 time 733982793053 state 0x1 ref 1 00000100:00100000:1.0:1789760754.489091:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 733983053469 time 734053389849 state 0x0 ref 1 00000100:00100000:1.0:1789760754.489103:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5304380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512665728:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760754.489147:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25739264, mtime: 0x6aad94f1, atime: 0x0, ctime: 0x6aad94f1, blocks: 0xc807 00010000:00010000:1.0:1789760754.489156:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2ecfe lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.489227:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=734s v=(0 0) 00010000:00010000:1.0:1789760754.489233:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760754.489247:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2ecfe lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25739264->25743359] (req 25739264->25743359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.489268:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2ecfe lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25739264->25743359] (req 25739264->25743359) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760754.539460:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25739264 -> 25743360 00002000:00010000:1.0:1789760754.539469:0:6710:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760753 -> 1789760754 00002000:00010000:1.0:1789760754.539471:0:6710:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760753 -> 1789760754 00010000:00010000:1.0:1789760754.539487:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2ecfe lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25739264->25743359] (req 25739264->25743359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.539500:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2ecfe lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25739264->25743359] (req 25739264->25743359) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.539508:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2ecfe lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25739264->25743359] (req 25739264->25743359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.539516:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760754.539521:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2ecfe lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25739264->25743359] (req 25739264->25743359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.539529:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2ecfe lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25739264->25743359] (req 25739264->25743359) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.539549:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2ecfe lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25739264->25743359] (req 25739264->25743359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.539560:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2ecfe lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25739264->25743359] (req 25739264->25743359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760754.539656:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5304380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2360:x1876699512665728:12345-192.168.203.56@tcp:4:dd.0 Request processed in 50549us (50783us total) trans 4294973603 rc 0/0 00000100:00100000:1.0:1789760754.539666:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6314 00000100:00100000:1.0:1789760754.539676:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 733983053469 time 734053389849 state 0x2 ref 0 00000001:00080000:0.0:1789760754.540206:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973603 is committed 00000100:00100000:0.0:1789760754.549411:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760754.549498:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512665856 00000100:00100000:3.0:1789760754.549509:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512665856 00000100:00100000:3.0:1789760754.549564:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 733983053469 time 734053389849 state 0x0 ref 1 00000100:00100000:3.0:1789760754.549581:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 734053650265 time 734053389849 state 0x1 ref 1 00000100:00100000:3.0:1789760754.549620:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 734053650265 time 734113919942 state 0x0 ref 1 00000100:00100000:3.0:1789760754.549633:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac531c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512665856:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760754.549684:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25743360, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0xc807 00010000:00010000:3.0:1789760754.549694:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2ed05 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.549784:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=734s v=(0 0) 00010000:00010000:3.0:1789760754.549788:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760754.549795:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2ed05 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25743360->25747455] (req 25743360->25747455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.549809:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2ed05 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25743360->25747455] (req 25743360->25747455) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760754.622837:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973604 is committed 00002000:00010000:3.0:1789760754.622939:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25743360 -> 25747456 00010000:00010000:3.0:1789760754.622955:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2ed05 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25743360->25747455] (req 25743360->25747455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.622967:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2ed05 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25743360->25747455] (req 25743360->25747455) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.622976:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2ed05 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25743360->25747455] (req 25743360->25747455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.622983:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760754.622989:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2ed05 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25743360->25747455] (req 25743360->25747455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.622999:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2ed05 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25743360->25747455] (req 25743360->25747455) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.623019:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2ed05 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25743360->25747455] (req 25743360->25747455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.623028:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2ed05 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25743360->25747455] (req 25743360->25747455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760754.623122:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac531c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512665856:12345-192.168.203.56@tcp:4:dd.0 Request processed in 73486us (73717us total) trans 4294973604 rc 0/0 00000100:00100000:3.0:1789760754.623133:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6315 00000100:00100000:3.0:1789760754.623144:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 734053650265 time 734113919942 state 0x2 ref 0 00000100:00100000:3.0:1789760754.633785:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760754.633934:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512665984 00000100:00100000:2.0:1789760754.633949:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512665984 00000100:00100000:2.0:1789760754.634015:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 734053650265 time 734113919942 state 0x0 ref 1 00000100:00100000:2.0:1789760754.634030:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 734114180358 time 734113919942 state 0x1 ref 1 00000100:00100000:2.0:1789760754.634118:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 734114180358 time 734198416548 state 0x0 ref 1 00000100:00100000:2.0:1789760754.634133:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac3352300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512665984:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760754.634329:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25747456, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0xc807 00010000:00010000:2.0:1789760754.634340:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2ed0c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.634356:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=734s v=(0 0) 00010000:00010000:2.0:1789760754.634361:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760754.634367:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2ed0c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25747456->25751551] (req 25747456->25751551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.634381:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2ed0c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25747456->25751551] (req 25747456->25751551) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760754.671730:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973605 is committed 00002000:00010000:2.0:1789760754.671825:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25747456 -> 25751552 00010000:00010000:2.0:1789760754.671846:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2ed0c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25747456->25751551] (req 25747456->25751551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.671863:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2ed0c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25747456->25751551] (req 25747456->25751551) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.671872:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2ed0c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25747456->25751551] (req 25747456->25751551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.671886:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760754.671893:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2ed0c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25747456->25751551] (req 25747456->25751551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.671903:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2ed0c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25747456->25751551] (req 25747456->25751551) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.672112:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2ed0c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25747456->25751551] (req 25747456->25751551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.672121:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2ed0c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25747456->25751551] (req 25747456->25751551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760754.672289:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac3352300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512665984:12345-192.168.203.56@tcp:4:dd.0 Request processed in 38153us (38512us total) trans 4294973605 rc 0/0 00000100:00100000:2.0:1789760754.672302:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6316 00000100:00100000:2.0:1789760754.672314:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 734114180358 time 734198416548 state 0x2 ref 0 00000100:00100000:2.0:1789760754.677083:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760754.677206:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512666112 00000100:00100000:1.0:1789760754.677219:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512666112 00000100:00100000:1.0:1789760754.677295:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 734114180358 time 734198416548 state 0x0 ref 1 00000100:00100000:1.0:1789760754.677313:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 734198676964 time 734198416548 state 0x1 ref 1 00000100:00100000:1.0:1789760754.677347:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 734198676964 time 734241645919 state 0x0 ref 1 00000100:00100000:1.0:1789760754.677361:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5306680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512666112:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760754.677421:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25751552, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0xc807 00010000:00010000:1.0:1789760754.677433:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2ed13 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.677453:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=734s v=(0 0) 00010000:00010000:1.0:1789760754.677459:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760754.677466:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2ed13 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25751552->25755647] (req 25751552->25755647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.677487:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2ed13 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25751552->25755647] (req 25751552->25755647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760754.717790:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973606 is committed 00002000:00010000:1.0:1789760754.718023:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25751552 -> 25755648 00010000:00010000:1.0:1789760754.718045:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2ed13 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25751552->25755647] (req 25751552->25755647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.718060:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2ed13 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25751552->25755647] (req 25751552->25755647) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.718071:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2ed13 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25751552->25755647] (req 25751552->25755647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.718079:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760754.718085:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2ed13 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25751552->25755647] (req 25751552->25755647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.718096:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2ed13 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25751552->25755647] (req 25751552->25755647) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.718122:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2ed13 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25751552->25755647] (req 25751552->25755647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.718131:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2ed13 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25751552->25755647] (req 25751552->25755647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760754.718305:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5306680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512666112:12345-192.168.203.56@tcp:4:dd.0 Request processed in 40940us (41230us total) trans 4294973606 rc 0/0 00000100:00100000:1.0:1789760754.718317:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6317 00000100:00100000:1.0:1789760754.718330:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 734198676964 time 734241645919 state 0x2 ref 0 00000100:00100000:2.0:1789760754.721481:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760754.721583:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512666240 00000100:00100000:3.0:1789760754.721595:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512666240 00000100:00100000:3.0:1789760754.721874:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 734198676964 time 734241645919 state 0x0 ref 1 00000100:00100000:3.0:1789760754.721895:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 734241906335 time 734241645919 state 0x1 ref 1 00000100:00100000:3.0:1789760754.722061:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 734241906335 time 734286359015 state 0x0 ref 1 00000100:00100000:3.0:1789760754.722080:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5307b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512666240:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760754.722166:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25755648, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0xc807 00010000:00010000:3.0:1789760754.722178:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2ed1a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.722200:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=734s v=(0 0) 00010000:00010000:3.0:1789760754.722206:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760754.722215:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2ed1a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25755648->25759743] (req 25755648->25759743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.722247:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2ed1a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25755648->25759743] (req 25755648->25759743) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760754.759674:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25755648 -> 25759744 00010000:00010000:2.0:1789760754.759728:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2ed1a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25755648->25759743] (req 25755648->25759743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.759744:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2ed1a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25755648->25759743] (req 25755648->25759743) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.759754:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2ed1a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25755648->25759743] (req 25755648->25759743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.759762:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760754.759768:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2ed1a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25755648->25759743] (req 25755648->25759743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.759777:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2ed1a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25755648->25759743] (req 25755648->25759743) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.759802:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2ed1a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25755648->25759743] (req 25755648->25759743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.759809:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2ed1a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25755648->25759743] (req 25755648->25759743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760754.760231:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5307b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2359:x1876699512666240:12345-192.168.203.56@tcp:4:dd.0 Request processed in 38148us (38758us total) trans 4294973607 rc 0/0 00000100:00100000:2.0:1789760754.760253:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6318 00000100:00100000:2.0:1789760754.760267:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 734241906335 time 734286359015 state 0x2 ref 0 00000001:00080000:0.0:1789760754.760434:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973607 is committed 00000100:00100000:3.0:1789760754.765549:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760754.765749:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512666368 00000100:00100000:2.0:1789760754.765764:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512666368 00000100:00100000:2.0:1789760754.765833:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 734241906335 time 734286359015 state 0x0 ref 1 00000100:00100000:2.0:1789760754.765851:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 734286619431 time 734286359015 state 0x1 ref 1 00000100:00100000:2.0:1789760754.765886:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 734286619431 time 734330185581 state 0x0 ref 1 00000100:00100000:2.0:1789760754.765899:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac3353800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512666368:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760754.765968:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25759744, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0xc807 00010000:00010000:2.0:1789760754.765979:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2ed21 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.766008:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=734s v=(0 0) 00010000:00010000:2.0:1789760754.766014:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760754.766021:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2ed21 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25759744->25763839] (req 25759744->25763839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.766038:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2ed21 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25759744->25763839] (req 25759744->25763839) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760754.804225:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973608 is committed 00002000:00010000:0.0:1789760754.804802:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25759744 -> 25763840 00010000:00010000:0.0:1789760754.804825:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2ed21 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25759744->25763839] (req 25759744->25763839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.804840:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2ed21 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25759744->25763839] (req 25759744->25763839) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.804849:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2ed21 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25759744->25763839] (req 25759744->25763839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.804857:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760754.804863:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2ed21 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25759744->25763839] (req 25759744->25763839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.804874:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2ed21 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25759744->25763839] (req 25759744->25763839) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.804898:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2ed21 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25759744->25763839] (req 25759744->25763839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.804916:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2ed21 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25759744->25763839] (req 25759744->25763839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760754.805146:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac3353800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512666368:12345-192.168.203.56@tcp:4:dd.0 Request processed in 39240us (39612us total) trans 4294973608 rc 0/0 00000100:00100000:0.0:1789760754.805159:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6319 00000100:00100000:0.0:1789760754.805171:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 734286619431 time 734330185581 state 0x2 ref 0 00000100:00100000:1.0:1789760754.809523:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760754.809764:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512666496 00000100:00100000:3.0:1789760754.809847:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512666496 00000100:00100000:2.0:1789760754.809984:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 734286619431 time 734330185581 state 0x0 ref 1 00000100:00100000:2.0:1789760754.810003:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 734330445997 time 734330185581 state 0x1 ref 1 00000100:00100000:2.0:1789760754.810062:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 734330445997 time 734374361296 state 0x0 ref 1 00000100:00100000:2.0:1789760754.810076:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febfbddc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512666496:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760754.810142:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25763840, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0xc807 00010000:00010000:2.0:1789760754.810151:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2ed28 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.810172:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=734s v=(0 0) 00010000:00010000:2.0:1789760754.810177:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760754.810184:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2ed28 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25763840->25767935] (req 25763840->25767935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.810200:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2ed28 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25763840->25767935] (req 25763840->25767935) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760754.861330:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973609 is committed 00002000:00010000:2.0:1789760754.862344:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25763840 -> 25767936 00010000:00010000:2.0:1789760754.862367:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2ed28 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25763840->25767935] (req 25763840->25767935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.862383:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2ed28 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25763840->25767935] (req 25763840->25767935) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.862393:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2ed28 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25763840->25767935] (req 25763840->25767935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.862401:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760754.862407:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2ed28 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25763840->25767935] (req 25763840->25767935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.862419:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2ed28 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25763840->25767935] (req 25763840->25767935) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.862444:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2ed28 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25763840->25767935] (req 25763840->25767935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.862455:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2ed28 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25763840->25767935] (req 25763840->25767935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760754.862584:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febfbddc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512666496:12345-192.168.203.56@tcp:4:dd.0 Request processed in 52502us (53069us total) trans 4294973609 rc 0/0 00000100:00100000:2.0:1789760754.862596:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6320 00000100:00100000:2.0:1789760754.862608:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 734330445997 time 734374361296 state 0x2 ref 0 00000100:00100000:3.0:1789760754.868949:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760754.869026:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512666624 00000100:00100000:2.0:1789760754.869050:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512666624 00000100:00100000:2.0:1789760754.869150:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 734330445997 time 734374361296 state 0x0 ref 1 00000100:00100000:2.0:1789760754.869171:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 734374621712 time 734374361296 state 0x1 ref 1 00000100:00100000:2.0:1789760754.869207:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 734374621712 time 734433506619 state 0x0 ref 1 00000100:00100000:2.0:1789760754.869223:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febfc0a3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512666624:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760754.869307:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25767936, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0xc807 00010000:00010000:2.0:1789760754.869322:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2ed2f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.869342:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=734s v=(0 0) 00010000:00010000:2.0:1789760754.869347:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760754.869355:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2ed2f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25767936->25772031] (req 25767936->25772031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.869380:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2ed2f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25767936->25772031] (req 25767936->25772031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760754.921313:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25767936 -> 25772032 00010000:00010000:3.0:1789760754.921448:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2ed2f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25767936->25772031] (req 25767936->25772031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.921462:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2ed2f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25767936->25772031] (req 25767936->25772031) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.921471:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2ed2f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25767936->25772031] (req 25767936->25772031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.921478:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760754.921483:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2ed2f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25767936->25772031] (req 25767936->25772031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.921492:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2ed2f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25767936->25772031] (req 25767936->25772031) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.921512:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2ed2f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25767936->25772031] (req 25767936->25772031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.921520:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2ed2f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25767936->25772031] (req 25767936->25772031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760754.921619:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febfc0a3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2359:x1876699512666624:12345-192.168.203.56@tcp:4:dd.0 Request processed in 52395us (52677us total) trans 4294973610 rc 0/0 00000100:00100000:3.0:1789760754.921630:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6321 00000100:00100000:3.0:1789760754.921639:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 734374621712 time 734433506619 state 0x2 ref 0 00000001:00080000:3.0:1789760754.921683:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973610 is committed 00000100:00100000:2.0:1789760754.925936:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760754.926043:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512666752 00000100:00100000:1.0:1789760754.926058:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512666752 00000100:00100000:1.0:1789760754.926118:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 734374621712 time 734433506619 state 0x0 ref 1 00000100:00100000:1.0:1789760754.926134:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 734433767035 time 734433506619 state 0x1 ref 1 00000100:00100000:1.0:1789760754.926168:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 734433767035 time 734490467327 state 0x0 ref 1 00000100:00100000:1.0:1789760754.926182:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5307100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512666752:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760754.926308:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25772032, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0xc807 00010000:00010000:1.0:1789760754.926318:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2ed36 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.926334:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=734s v=(0 0) 00010000:00010000:1.0:1789760754.926338:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760754.926406:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2ed36 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25772032->25776127] (req 25772032->25776127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.926425:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2ed36 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25772032->25776127] (req 25772032->25776127) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760754.949112:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760754.949152:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760754.949222:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760754.949257:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512666880 00000100:00100000:3.0:1789760754.949265:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512666880 00000100:00100000:3.0:1789760754.949316:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 402 00000100:00100000:3.0:1789760754.949326:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febe0503050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:e6cf7267-9b19-462a-a263-14115df15dab+13:2361:x1876699512666880:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:3.0:1789760754.949388:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febe0503050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:e6cf7267-9b19-462a-a263-14115df15dab+13:2361:x1876699512666880:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 64us (236us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760754.949396:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 402 00000100:00100000:0.0:1789760754.952291:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512667008 00000100:00100000:0.0:1789760754.952548:0:5847:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512667008 00000100:00080000:2.0:1789760754.953001:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789760754.953277:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512667136 00000100:00100000:0.0:1789760754.953282:0:5847:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512667136 00000100:00080000:2.0:1789760754.953658:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789760754.953691:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 484 00000100:00100000:0.0:1789760754.953897:0:5847:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febfc0a3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+8:2361:x1876699512667008:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:2.0:1789760754.953908:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 485 00000100:00100000:2.0:1789760754.953919:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febfb894700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:d2c40150-e97b-40da-952b-cb9f18bf7e96+5:2361:x1876699512667136:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:2.0:1789760754.953996:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febfb894700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:d2c40150-e97b-40da-952b-cb9f18bf7e96+5:2361:x1876699512667136:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 80us (4774us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760754.954004:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 485 00000100:00100000:0.0:1789760754.954086:0:5847:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febfc0a3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+8:2361:x1876699512667008:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 194us (4981us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760754.954093:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 484 00002000:00010000:1.0:1789760754.963732:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25772032 -> 25776128 00010000:00010000:1.0:1789760754.963751:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2ed36 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25772032->25776127] (req 25772032->25776127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.963765:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2ed36 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25772032->25776127] (req 25772032->25776127) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.963774:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2ed36 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25772032->25776127] (req 25772032->25776127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.963781:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760754.963786:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2ed36 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25772032->25776127] (req 25772032->25776127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.963794:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2ed36 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25772032->25776127] (req 25772032->25776127) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.963883:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2ed36 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25772032->25776127] (req 25772032->25776127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.963892:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2ed36 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25772032->25776127] (req 25772032->25776127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760754.964047:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5307100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2359:x1876699512666752:12345-192.168.203.56@tcp:4:dd.0 Request processed in 37863us (38120us total) trans 4294973611 rc 0/0 00000100:00100000:1.0:1789760754.964057:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6322 00000100:00100000:1.0:1789760754.964066:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 734433767035 time 734490467327 state 0x2 ref 0 00000001:00080000:3.0:1789760754.964142:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973611 is committed 00000100:00100000:3.0:1789760754.967454:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760754.967788:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512667264 00000100:00100000:2.0:1789760754.967801:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512667264 00000100:00100000:2.0:1789760754.967874:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 734433767035 time 734490467327 state 0x0 ref 1 00000100:00100000:2.0:1789760754.967893:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 734490727743 time 734490467327 state 0x1 ref 1 00000100:00100000:2.0:1789760754.968054:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 734490727743 time 734532352132 state 0x0 ref 1 00000100:00100000:2.0:1789760754.968070:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febfb897100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512667264:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760754.968136:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25776128, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0xc807 00010000:00010000:2.0:1789760754.968146:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2ed3d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.968165:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=734s v=(0 0) 00010000:00010000:2.0:1789760754.968172:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760754.968180:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2ed3d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25776128->25780223] (req 25776128->25780223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.968196:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2ed3d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25776128->25780223] (req 25776128->25780223) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760755.012196:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25776128 -> 25780224 00010000:00010000:2.0:1789760755.012216:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2ed3d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25776128->25780223] (req 25776128->25780223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.012229:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2ed3d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25776128->25780223] (req 25776128->25780223) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.012249:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2ed3d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25776128->25780223] (req 25776128->25780223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.012257:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760755.012262:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2ed3d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25776128->25780223] (req 25776128->25780223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.012270:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2ed3d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25776128->25780223] (req 25776128->25780223) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.012289:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2ed3d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25776128->25780223] (req 25776128->25780223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.012310:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2ed3d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25776128->25780223] (req 25776128->25780223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760755.012404:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febfb897100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2361:x1876699512667264:12345-192.168.203.56@tcp:4:dd.0 Request processed in 44332us (44960us total) trans 4294973612 rc 0/0 00000100:00100000:2.0:1789760755.012414:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6323 00000100:00100000:2.0:1789760755.012424:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 734490727743 time 734532352132 state 0x2 ref 0 00000001:00080000:3.0:1789760755.012645:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973612 is committed 00000100:00100000:2.0:1789760755.020986:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760755.021116:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512667392 00000100:00100000:3.0:1789760755.021135:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512667392 00000100:00080000:2.0:1789760755.021204:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1789760755.021372:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 734490727743 time 734532352132 state 0x0 ref 1 00000100:00100000:3.0:1789760755.021394:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 734532612548 time 734532352132 state 0x1 ref 1 00000100:00100000:3.0:1789760755.021429:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 734532612548 time 734585728256 state 0x0 ref 1 00000100:00100000:3.0:1789760755.021450:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5304a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512667392:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760755.021516:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25780224, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0xc807 00010000:00010000:3.0:1789760755.021528:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2ed44 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.021548:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=734s v=(0 0) 00010000:00010000:3.0:1789760755.021554:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760755.021563:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2ed44 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25780224->25784319] (req 25780224->25784319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.021580:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2ed44 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25780224->25784319] (req 25780224->25784319) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.060095:0:5835:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:2.0:1789760755.070743:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25780224 -> 25784320 00010000:00010000:2.0:1789760755.070771:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2ed44 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25780224->25784319] (req 25780224->25784319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.070787:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2ed44 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25780224->25784319] (req 25780224->25784319) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.070798:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2ed44 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25780224->25784319] (req 25780224->25784319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.070806:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760755.070811:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2ed44 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25780224->25784319] (req 25780224->25784319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.070822:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2ed44 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25780224->25784319] (req 25780224->25784319) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.070846:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2ed44 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25780224->25784319] (req 25780224->25784319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.070856:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2ed44 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25780224->25784319] (req 25780224->25784319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760755.071103:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5304a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2360:x1876699512667392:12345-192.168.203.56@tcp:4:dd.0 Request processed in 49653us (50117us total) trans 4294973613 rc 0/0 00000100:00100000:2.0:1789760755.071116:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6324 00000100:00100000:2.0:1789760755.071129:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 734532612548 time 734585728256 state 0x2 ref 0 00000100:00100000:3.0:1789760755.075624:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000001:00080000:0.0:1789760755.075884:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973613 is committed 00000100:00100000:1.0:1789760755.076023:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512667520 00000100:00100000:1.0:1789760755.076036:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512667520 00000100:00100000:1.0:1789760755.076106:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 734532612548 time 734585728256 state 0x0 ref 1 00000100:00100000:1.0:1789760755.076125:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 734585988672 time 734585728256 state 0x1 ref 1 00000100:00100000:1.0:1789760755.076150:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 734585988672 time 734640450327 state 0x0 ref 1 00000100:00100000:1.0:1789760755.076165:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febfb895880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512667520:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760755.076229:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25784320, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0xc807 00010000:00010000:1.0:1789760755.076248:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2ed4b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.076269:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=734s v=(0 0) 00010000:00010000:1.0:1789760755.076275:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760755.076284:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2ed4b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25784320->25788415] (req 25784320->25788415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.076301:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2ed4b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25784320->25788415] (req 25784320->25788415) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760755.124370:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973614 is committed 00002000:00010000:0.0:1789760755.124593:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25784320 -> 25788416 00010000:00010000:0.0:1789760755.124613:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2ed4b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25784320->25788415] (req 25784320->25788415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.125619:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2ed4b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25784320->25788415] (req 25784320->25788415) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.125691:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2ed4b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25784320->25788415] (req 25784320->25788415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.125993:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760755.126016:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2ed4b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25784320->25788415] (req 25784320->25788415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.126029:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2ed4b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25784320->25788415] (req 25784320->25788415) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.126083:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2ed4b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25784320->25788415] (req 25784320->25788415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.126093:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2ed4b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25784320->25788415] (req 25784320->25788415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760755.126337:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febfb895880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512667520:12345-192.168.203.56@tcp:4:dd.0 Request processed in 50167us (50719us total) trans 4294973614 rc 0/0 00000100:00100000:0.0:1789760755.126364:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6325 00000100:00100000:0.0:1789760755.126375:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 734585988672 time 734640450327 state 0x2 ref 0 00000100:00100000:3.0:1789760755.129691:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760755.129788:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512667648 00000100:00100000:0.0:1789760755.129801:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512667648 00000100:00100000:0.0:1789760755.129875:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 734585988672 time 734640450327 state 0x0 ref 1 00000100:00100000:0.0:1789760755.129894:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 734640710743 time 734640450327 state 0x1 ref 1 00000100:00100000:0.0:1789760755.129937:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 734640710743 time 734694236531 state 0x0 ref 1 00000100:00100000:0.0:1789760755.129951:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febe7907100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512667648:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760755.130011:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25788416, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0xc807 00010000:00010000:0.0:1789760755.130021:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2ed52 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.130041:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=734s v=(0 0) 00010000:00010000:0.0:1789760755.130046:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760755.130055:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2ed52 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25788416->25792511] (req 25788416->25792511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.130071:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2ed52 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25788416->25792511] (req 25788416->25792511) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760755.180138:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25788416 -> 25792512 00010000:00010000:2.0:1789760755.180166:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2ed52 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25788416->25792511] (req 25788416->25792511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.180182:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2ed52 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25788416->25792511] (req 25788416->25792511) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.180191:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2ed52 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25788416->25792511] (req 25788416->25792511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.180199:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760755.180204:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2ed52 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25788416->25792511] (req 25788416->25792511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.180214:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2ed52 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25788416->25792511] (req 25788416->25792511) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.180257:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2ed52 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25788416->25792511] (req 25788416->25792511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.180266:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2ed52 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25788416->25792511] (req 25788416->25792511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760755.180391:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febe7907100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2360:x1876699512667648:12345-192.168.203.56@tcp:4:dd.0 Request processed in 50433us (50706us total) trans 4294973615 rc 0/0 00000100:00100000:2.0:1789760755.180403:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6326 00000100:00100000:2.0:1789760755.180415:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 734640710743 time 734694236531 state 0x2 ref 0 00000001:00080000:2.0:1789760755.180960:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973615 is committed 00000100:00100000:1.0:1789760755.183865:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760755.184022:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512667776 00000100:00100000:3.0:1789760755.184036:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512667776 00000100:00100000:3.0:1789760755.184110:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 734640710743 time 734694236531 state 0x0 ref 1 00000100:00100000:3.0:1789760755.184128:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 734694496947 time 734694236531 state 0x1 ref 1 00000100:00100000:3.0:1789760755.184168:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 734694496947 time 734748466956 state 0x0 ref 1 00000100:00100000:3.0:1789760755.184182:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac28f0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512667776:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760755.184258:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25792512, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0xc807 00010000:00010000:3.0:1789760755.184269:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2ed59 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.184287:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=734s v=(0 0) 00010000:00010000:3.0:1789760755.184292:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760755.184300:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2ed59 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25792512->25796607] (req 25792512->25796607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.184316:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2ed59 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25792512->25796607] (req 25792512->25796607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760755.225836:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973616 is committed 00002000:00010000:1.0:1789760755.225985:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25792512 -> 25796608 00010000:00010000:1.0:1789760755.226005:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2ed59 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25792512->25796607] (req 25792512->25796607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.226020:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2ed59 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25792512->25796607] (req 25792512->25796607) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.226031:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2ed59 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25792512->25796607] (req 25792512->25796607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.226039:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760755.226046:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2ed59 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25792512->25796607] (req 25792512->25796607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.226056:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2ed59 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25792512->25796607] (req 25792512->25796607) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.226084:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2ed59 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25792512->25796607] (req 25792512->25796607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.226092:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2ed59 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25792512->25796607] (req 25792512->25796607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760755.226216:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac28f0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512667776:12345-192.168.203.56@tcp:4:dd.0 Request processed in 42029us (42357us total) trans 4294973616 rc 0/0 00000100:00100000:1.0:1789760755.226229:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6327 00000100:00100000:1.0:1789760755.226251:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 734694496947 time 734748466956 state 0x2 ref 0 00000100:00100000:1.0:1789760755.233384:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760755.233484:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512667904 00000100:00100000:0.0:1789760755.233497:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512667904 00000100:00100000:0.0:1789760755.233573:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 734694496947 time 734748466956 state 0x0 ref 1 00000100:00100000:0.0:1789760755.233595:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 734748727372 time 734748466956 state 0x1 ref 1 00000100:00100000:0.0:1789760755.233630:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 734748727372 time 734797929795 state 0x0 ref 1 00000100:00100000:0.0:1789760755.233648:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac38b6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512667904:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760755.234293:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25796608, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0xc807 00010000:00010000:0.0:1789760755.234307:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2ed60 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.234327:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=734s v=(0 0) 00010000:00010000:0.0:1789760755.234333:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760755.234341:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2ed60 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25796608->25800703] (req 25796608->25800703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.234359:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2ed60 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25796608->25800703] (req 25796608->25800703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760755.269051:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25796608 -> 25800704 00010000:00010000:0.0:1789760755.269077:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2ed60 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25796608->25800703] (req 25796608->25800703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.269093:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2ed60 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25796608->25800703] (req 25796608->25800703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.269102:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2ed60 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25796608->25800703] (req 25796608->25800703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.269110:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760755.269115:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2ed60 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25796608->25800703] (req 25796608->25800703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.269125:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2ed60 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25796608->25800703] (req 25796608->25800703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.269148:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2ed60 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25796608->25800703] (req 25796608->25800703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.269156:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2ed60 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25796608->25800703] (req 25796608->25800703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760755.269289:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac38b6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2360:x1876699512667904:12345-192.168.203.56@tcp:4:dd.0 Request processed in 35638us (35914us total) trans 4294973617 rc 0/0 00000100:00100000:0.0:1789760755.269302:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6328 00000100:00100000:0.0:1789760755.269314:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 734748727372 time 734797929795 state 0x2 ref 0 00000001:00080000:3.0:1789760755.271000:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973617 is committed 00000100:00100000:3.0:1789760755.276403:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760755.277412:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512668032 00000100:00100000:2.0:1789760755.277428:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512668032 00000100:00100000:3.0:1789760755.278145:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 734748727372 time 734797929795 state 0x0 ref 1 00000100:00100000:3.0:1789760755.278168:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 734798190211 time 734797929795 state 0x1 ref 1 00000100:00100000:3.0:1789760755.278263:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 734798190211 time 734842561804 state 0x0 ref 1 00000100:00100000:3.0:1789760755.278280:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac437f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512668032:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760755.278352:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25800704, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0xc807 00010000:00010000:3.0:1789760755.278363:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2ed67 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.278383:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=734s v=(0 0) 00010000:00010000:3.0:1789760755.278388:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760755.278395:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2ed67 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25800704->25804799] (req 25800704->25804799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.278412:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2ed67 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25800704->25804799] (req 25800704->25804799) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760755.308514:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973618 is committed 00002000:00010000:1.0:1789760755.308690:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25800704 -> 25804800 00010000:00010000:1.0:1789760755.308839:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2ed67 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25800704->25804799] (req 25800704->25804799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.308854:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2ed67 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25800704->25804799] (req 25800704->25804799) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.308863:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2ed67 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25800704->25804799] (req 25800704->25804799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.308871:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760755.308877:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2ed67 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25800704->25804799] (req 25800704->25804799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.308886:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2ed67 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25800704->25804799] (req 25800704->25804799) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.308975:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2ed67 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25800704->25804799] (req 25800704->25804799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.308985:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2ed67 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25800704->25804799] (req 25800704->25804799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760755.309207:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac437f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512668032:12345-192.168.203.56@tcp:4:dd.0 Request processed in 30924us (32811us total) trans 4294973618 rc 0/0 00000100:00100000:1.0:1789760755.309221:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6329 00000100:00100000:1.0:1789760755.309242:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 734798190211 time 734842561804 state 0x2 ref 0 00000100:00100000:1.0:1789760755.312306:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760755.314456:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512668160 00000100:00100000:3.0:1789760755.314479:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512668160 00000100:00100000:3.0:1789760755.314571:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 734798190211 time 734842561804 state 0x0 ref 1 00000100:00100000:3.0:1789760755.314591:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 734842822220 time 734842561804 state 0x1 ref 1 00000100:00100000:3.0:1789760755.315392:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 734842822220 time 734879683722 state 0x0 ref 1 00000100:00100000:3.0:1789760755.315419:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febff932a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512668160:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760755.315489:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25804800, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0xc807 00010000:00010000:3.0:1789760755.315501:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2ed6e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.315527:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=734s v=(0 0) 00010000:00010000:3.0:1789760755.315532:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760755.315540:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2ed6e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25804800->25808895] (req 25804800->25808895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.315557:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2ed6e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25804800->25808895] (req 25804800->25808895) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760755.369354:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973619 is committed 00002000:00010000:2.0:1789760755.369748:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25804800 -> 25808896 00010000:00010000:2.0:1789760755.369768:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2ed6e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25804800->25808895] (req 25804800->25808895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.369783:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2ed6e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25804800->25808895] (req 25804800->25808895) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.369793:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2ed6e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25804800->25808895] (req 25804800->25808895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.369801:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760755.369807:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2ed6e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25804800->25808895] (req 25804800->25808895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.369818:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2ed6e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25804800->25808895] (req 25804800->25808895) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.369845:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2ed6e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25804800->25808895] (req 25804800->25808895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.369855:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2ed6e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25804800->25808895] (req 25804800->25808895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760755.370038:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febff932a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512668160:12345-192.168.203.56@tcp:4:dd.0 Request processed in 54622us (57740us total) trans 4294973619 rc 0/0 00000100:00100000:2.0:1789760755.370051:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6330 00000100:00100000:2.0:1789760755.370063:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 734842822220 time 734879683722 state 0x2 ref 0 00000100:00100000:1.0:1789760755.380978:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760755.381113:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512668288 00000100:00100000:0.0:1789760755.381127:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512668288 00000100:00100000:0.0:1789760755.381201:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 734842822220 time 734879683722 state 0x0 ref 1 00000100:00100000:0.0:1789760755.381220:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 734879944138 time 734879683722 state 0x1 ref 1 00000100:00100000:0.0:1789760755.381298:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 734879944138 time 734945597247 state 0x0 ref 1 00000100:00100000:0.0:1789760755.381313:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febff930e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512668288:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760755.381376:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25808896, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0xc807 00010000:00010000:0.0:1789760755.381387:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2ed75 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.381407:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=734s v=(0 0) 00010000:00010000:0.0:1789760755.381412:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760755.381420:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2ed75 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25808896->25812991] (req 25808896->25812991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.381435:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2ed75 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25808896->25812991] (req 25808896->25812991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760755.466835:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973620 is committed 00002000:00010000:2.0:1789760755.467033:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25808896 -> 25812992 00010000:00010000:2.0:1789760755.467053:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2ed75 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25808896->25812991] (req 25808896->25812991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.467069:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2ed75 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25808896->25812991] (req 25808896->25812991) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.467079:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2ed75 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25808896->25812991] (req 25808896->25812991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.467087:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760755.467094:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2ed75 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25808896->25812991] (req 25808896->25812991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.467105:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2ed75 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25808896->25812991] (req 25808896->25812991) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.467133:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2ed75 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25808896->25812991] (req 25808896->25812991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.467141:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2ed75 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25808896->25812991] (req 25808896->25812991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760755.467262:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febff930e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512668288:12345-192.168.203.56@tcp:4:dd.0 Request processed in 85946us (86369us total) trans 4294973620 rc 0/0 00000100:00100000:2.0:1789760755.467273:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6331 00000100:00100000:2.0:1789760755.467285:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 734879944138 time 734945597247 state 0x2 ref 0 00000100:00100000:3.0:1789760755.474095:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760755.474205:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512668416 00000100:00100000:1.0:1789760755.474218:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512668416 00000100:00100000:1.0:1789760755.474301:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 734879944138 time 734945597247 state 0x0 ref 1 00000100:00100000:1.0:1789760755.474321:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 734945857663 time 734945597247 state 0x1 ref 1 00000100:00100000:1.0:1789760755.474358:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 734945857663 time 735038657601 state 0x0 ref 1 00000100:00100000:1.0:1789760755.474373:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac437fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512668416:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760755.474442:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25812992, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0xc807 00010000:00010000:1.0:1789760755.474455:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2ed7c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.474475:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=735s v=(0 0) 00010000:00010000:1.0:1789760755.474481:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760755.474489:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2ed7c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25812992->25817087] (req 25812992->25817087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.474512:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2ed7c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25812992->25817087] (req 25812992->25817087) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760755.521471:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973621 is committed 00002000:00010000:1.0:1789760755.521598:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25812992 -> 25817088 00010000:00010000:1.0:1789760755.521618:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2ed7c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25812992->25817087] (req 25812992->25817087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.521738:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2ed7c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25812992->25817087] (req 25812992->25817087) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.521750:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2ed7c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25812992->25817087] (req 25812992->25817087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.521758:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760755.521764:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2ed7c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25812992->25817087] (req 25812992->25817087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.521775:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2ed7c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25812992->25817087] (req 25812992->25817087) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.521800:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2ed7c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25812992->25817087] (req 25812992->25817087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.521808:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2ed7c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25812992->25817087] (req 25812992->25817087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760755.522197:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac437fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512668416:12345-192.168.203.56@tcp:4:dd.0 Request processed in 47819us (48109us total) trans 4294973621 rc 0/0 00000100:00100000:1.0:1789760755.522210:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6332 00000100:00100000:1.0:1789760755.522221:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 734945857663 time 735038657601 state 0x2 ref 0 00000100:00100000:1.0:1789760755.526810:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760755.527001:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512668544 00000100:00100000:2.0:1789760755.527015:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512668544 00000100:00100000:2.0:1789760755.527090:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 734945857663 time 735038657601 state 0x0 ref 1 00000100:00100000:2.0:1789760755.527109:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 735038918017 time 735038657601 state 0x1 ref 1 00000100:00100000:2.0:1789760755.527139:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 735038918017 time 735091438912 state 0x0 ref 1 00000100:00100000:2.0:1789760755.527152:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac403bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512668544:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760755.527216:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25817088, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0xc807 00010000:00010000:2.0:1789760755.527228:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2ed83 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.527359:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=735s v=(0 0) 00010000:00010000:2.0:1789760755.527364:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760755.527371:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2ed83 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25817088->25821183] (req 25817088->25821183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.527388:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2ed83 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25817088->25821183] (req 25817088->25821183) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760755.579142:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973622 is committed 00002000:00010000:3.0:1789760755.580185:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25817088 -> 25821184 00002000:00010000:3.0:1789760755.580192:0:13063:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760754 -> 1789760755 00002000:00010000:3.0:1789760755.580299:0:13063:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760754 -> 1789760755 00010000:00010000:3.0:1789760755.580318:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2ed83 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25817088->25821183] (req 25817088->25821183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.580332:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2ed83 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25817088->25821183] (req 25817088->25821183) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.580341:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2ed83 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25817088->25821183] (req 25817088->25821183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.580350:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760755.580355:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2ed83 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25817088->25821183] (req 25817088->25821183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.580366:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2ed83 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25817088->25821183] (req 25817088->25821183) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.580391:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2ed83 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25817088->25821183] (req 25817088->25821183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.580400:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2ed83 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25817088->25821183] (req 25817088->25821183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760755.580577:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac403bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512668544:12345-192.168.203.56@tcp:4:dd.0 Request processed in 53419us (53775us total) trans 4294973622 rc 0/0 00000100:00100000:3.0:1789760755.580591:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6333 00000100:00100000:3.0:1789760755.580603:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 735038918017 time 735091438912 state 0x2 ref 0 00000100:00100000:3.0:1789760755.586094:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760755.586311:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512668672 00000100:00100000:2.0:1789760755.586325:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512668672 00000100:00100000:2.0:1789760755.586398:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 735038918017 time 735091438912 state 0x0 ref 1 00000100:00100000:2.0:1789760755.586417:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 735091699328 time 735091438912 state 0x1 ref 1 00000100:00100000:2.0:1789760755.586456:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 735091699328 time 735150755692 state 0x0 ref 1 00000100:00100000:2.0:1789760755.586470:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac437d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512668672:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760755.586535:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25821184, mtime: 0x6aad94f3, atime: 0x0, ctime: 0x6aad94f3, blocks: 0xc807 00010000:00010000:2.0:1789760755.586547:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2ed8a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.586564:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=735s v=(0 0) 00010000:00010000:2.0:1789760755.586569:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760755.586576:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2ed8a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25821184->25825279] (req 25821184->25825279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.586592:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2ed8a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25821184->25825279] (req 25821184->25825279) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760755.618505:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973623 is committed 00002000:00010000:1.0:1789760755.618642:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25821184 -> 25825280 00010000:00010000:1.0:1789760755.618664:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2ed8a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25821184->25825279] (req 25821184->25825279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.618680:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2ed8a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25821184->25825279] (req 25821184->25825279) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.618691:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2ed8a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25821184->25825279] (req 25821184->25825279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.618881:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760755.618887:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2ed8a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25821184->25825279] (req 25821184->25825279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.618898:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2ed8a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25821184->25825279] (req 25821184->25825279) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.618937:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2ed8a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25821184->25825279] (req 25821184->25825279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.618946:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2ed8a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25821184->25825279] (req 25821184->25825279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760755.619080:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac437d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512668672:12345-192.168.203.56@tcp:4:dd.0 Request processed in 32605us (32994us total) trans 4294973623 rc 0/0 00000100:00100000:1.0:1789760755.619093:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6334 00000100:00100000:1.0:1789760755.619106:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 735091699328 time 735150755692 state 0x2 ref 0 00000100:00100000:0.0:1789760755.624019:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760755.624118:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512668800 00000100:00100000:1.0:1789760755.624133:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512668800 00000100:00100000:1.0:1789760755.624218:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 735091699328 time 735150755692 state 0x0 ref 1 00000100:00100000:1.0:1789760755.624250:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 735151016108 time 735150755692 state 0x1 ref 1 00000100:00100000:1.0:1789760755.624289:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 735151016108 time 735188587492 state 0x0 ref 1 00000100:00100000:1.0:1789760755.624304:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac531d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512668800:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760755.624371:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25825280, mtime: 0x6aad94f3, atime: 0x0, ctime: 0x6aad94f3, blocks: 0xc807 00010000:00010000:1.0:1789760755.624382:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2ed91 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.624401:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=735s v=(0 0) 00010000:00010000:1.0:1789760755.624406:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760755.624415:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2ed91 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25825280->25829375] (req 25825280->25829375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.624437:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2ed91 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25825280->25829375] (req 25825280->25829375) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760755.667531:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973624 is committed 00002000:00010000:2.0:1789760755.667616:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25825280 -> 25829376 00010000:00010000:2.0:1789760755.667637:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2ed91 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25825280->25829375] (req 25825280->25829375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.667651:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2ed91 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25825280->25829375] (req 25825280->25829375) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.667661:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2ed91 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25825280->25829375] (req 25825280->25829375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.667669:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760755.667675:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2ed91 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25825280->25829375] (req 25825280->25829375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.667685:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2ed91 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25825280->25829375] (req 25825280->25829375) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.667732:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2ed91 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25825280->25829375] (req 25825280->25829375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.667743:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2ed91 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25825280->25829375] (req 25825280->25829375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760755.667918:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac531d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512668800:12345-192.168.203.56@tcp:4:dd.0 Request processed in 43610us (43907us total) trans 4294973624 rc 0/0 00000100:00100000:2.0:1789760755.667932:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6335 00000100:00100000:2.0:1789760755.667943:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 735151016108 time 735188587492 state 0x2 ref 0 00000100:00100000:3.0:1789760755.674996:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760755.675258:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512668928 00000100:00100000:0.0:1789760755.675272:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512668928 00000100:00100000:0.0:1789760755.675357:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 735151016108 time 735188587492 state 0x0 ref 1 00000100:00100000:0.0:1789760755.675377:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 735188847908 time 735188587492 state 0x1 ref 1 00000100:00100000:0.0:1789760755.675477:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 735188847908 time 735239775984 state 0x0 ref 1 00000100:00100000:0.0:1789760755.675493:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac437df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512668928:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760755.675556:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25829376, mtime: 0x6aad94f3, atime: 0x0, ctime: 0x6aad94f3, blocks: 0xc807 00010000:00010000:0.0:1789760755.675567:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2ed98 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.675587:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=735s v=(0 0) 00010000:00010000:0.0:1789760755.675593:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760755.675602:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2ed98 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25829376->25833471] (req 25829376->25833471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.675632:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2ed98 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25829376->25833471] (req 25829376->25833471) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760755.731452:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973625 is committed 00002000:00010000:2.0:1789760755.731839:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25829376 -> 25833472 00010000:00010000:2.0:1789760755.731860:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2ed98 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25829376->25833471] (req 25829376->25833471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.731875:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2ed98 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25829376->25833471] (req 25829376->25833471) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.731885:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2ed98 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25829376->25833471] (req 25829376->25833471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.731892:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760755.731899:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2ed98 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25829376->25833471] (req 25829376->25833471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.731909:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2ed98 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25829376->25833471] (req 25829376->25833471) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.731934:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2ed98 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25829376->25833471] (req 25829376->25833471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.731948:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2ed98 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25829376->25833471] (req 25829376->25833471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760755.732080:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac437df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512668928:12345-192.168.203.56@tcp:4:dd.0 Request processed in 56584us (57095us total) trans 4294973625 rc 0/0 00000100:00100000:2.0:1789760755.732090:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6336 00000100:00100000:2.0:1789760755.732102:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 735188847908 time 735239775984 state 0x2 ref 0 00000100:00100000:3.0:1789760755.736208:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760755.743418:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512669056 00000100:00100000:1.0:1789760755.743633:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512669056 00000100:00100000:2.0:1789760755.743831:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 735188847908 time 735239775984 state 0x0 ref 1 00000100:00100000:2.0:1789760755.743854:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 735240036400 time 735239775984 state 0x1 ref 1 00000100:00100000:2.0:1789760755.743903:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 735240036400 time 735308202268 state 0x0 ref 1 00000100:00100000:2.0:1789760755.743919:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac437e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512669056:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760755.743986:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25833472, mtime: 0x6aad94f3, atime: 0x0, ctime: 0x6aad94f3, blocks: 0xc807 00010000:00010000:2.0:1789760755.743995:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2ed9f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.744014:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=735s v=(0 0) 00010000:00010000:2.0:1789760755.744018:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760755.744025:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2ed9f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25833472->25837567] (req 25833472->25837567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.744041:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2ed9f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25833472->25837567] (req 25833472->25837567) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760755.815746:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973626 is committed 00002000:00010000:2.0:1789760755.816064:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25833472 -> 25837568 00010000:00010000:2.0:1789760755.816086:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2ed9f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25833472->25837567] (req 25833472->25837567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.816102:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2ed9f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25833472->25837567] (req 25833472->25837567) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.816112:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2ed9f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25833472->25837567] (req 25833472->25837567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.816119:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760755.816125:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2ed9f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25833472->25837567] (req 25833472->25837567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.816135:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2ed9f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25833472->25837567] (req 25833472->25837567) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.816161:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2ed9f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25833472->25837567] (req 25833472->25837567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.816169:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2ed9f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25833472->25837567] (req 25833472->25837567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760755.816300:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac437e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512669056:12345-192.168.203.56@tcp:4:dd.0 Request processed in 72378us (80100us total) trans 4294973626 rc 0/0 00000100:00100000:2.0:1789760755.816315:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6337 00000100:00100000:2.0:1789760755.816327:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 735240036400 time 735308202268 state 0x2 ref 0 00000100:00100000:3.0:1789760755.824097:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760755.824349:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512669184 00000100:00100000:2.0:1789760755.824363:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512669184 00000100:00100000:2.0:1789760755.824440:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 735240036400 time 735308202268 state 0x0 ref 1 00000100:00100000:2.0:1789760755.824460:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 735308462684 time 735308202268 state 0x1 ref 1 00000100:00100000:2.0:1789760755.824490:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 735308462684 time 735388789069 state 0x0 ref 1 00000100:00100000:2.0:1789760755.824504:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2877100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512669184:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760755.824574:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25837568, mtime: 0x6aad94f3, atime: 0x0, ctime: 0x6aad94f3, blocks: 0xc807 00010000:00010000:2.0:1789760755.824585:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2eda6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.824606:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=735s v=(0 0) 00010000:00010000:2.0:1789760755.824611:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760755.824619:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2eda6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25837568->25841663] (req 25837568->25841663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.824637:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2eda6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25837568->25841663] (req 25837568->25841663) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760755.894625:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973627 is committed 00002000:00010000:2.0:1789760755.894980:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25837568 -> 25841664 00010000:00010000:2.0:1789760755.895004:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2eda6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25837568->25841663] (req 25837568->25841663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.895020:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2eda6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25837568->25841663] (req 25837568->25841663) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.895032:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2eda6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25837568->25841663] (req 25837568->25841663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.895052:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760755.895058:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2eda6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25837568->25841663] (req 25837568->25841663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.895069:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2eda6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25837568->25841663] (req 25837568->25841663) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.895095:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2eda6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25837568->25841663] (req 25837568->25841663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.895104:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2eda6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25837568->25841663] (req 25837568->25841663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760755.895238:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2877100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512669184:12345-192.168.203.56@tcp:4:dd.0 Request processed in 70719us (71139us total) trans 4294973627 rc 0/0 00000100:00100000:2.0:1789760755.895250:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6338 00000100:00100000:2.0:1789760755.895263:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 735308462684 time 735388789069 state 0x2 ref 0 00000100:00100000:2.0:1789760755.899177:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760755.899339:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512669312 00000100:00100000:0.0:1789760755.899354:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512669312 00000100:00100000:0.0:1789760755.899551:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 735308462684 time 735388789069 state 0x0 ref 1 00000100:00100000:0.0:1789760755.899566:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 735389049485 time 735388789069 state 0x1 ref 1 00000100:00100000:0.0:1789760755.899658:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 735389049485 time 735463956612 state 0x0 ref 1 00000100:00100000:0.0:1789760755.899674:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5307800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512669312:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760755.899761:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25841664, mtime: 0x6aad94f3, atime: 0x0, ctime: 0x6aad94f3, blocks: 0xc807 00010000:00010000:0.0:1789760755.899771:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2edad lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.899787:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=735s v=(0 0) 00010000:00010000:0.0:1789760755.899791:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760755.899798:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2edad lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25841664->25845759] (req 25841664->25845759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.899812:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2edad lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25841664->25845759] (req 25841664->25845759) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760755.924078:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973628 is committed 00002000:00010000:0.0:1789760755.924139:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25841664 -> 25845760 00010000:00010000:0.0:1789760755.924153:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2edad lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25841664->25845759] (req 25841664->25845759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.924167:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2edad lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25841664->25845759] (req 25841664->25845759) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.924176:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2edad lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25841664->25845759] (req 25841664->25845759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.924184:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760755.924189:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2edad lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25841664->25845759] (req 25841664->25845759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.924198:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2edad lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25841664->25845759] (req 25841664->25845759) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.924218:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2edad lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25841664->25845759] (req 25841664->25845759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.924239:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2edad lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25841664->25845759] (req 25841664->25845759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760755.924323:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5307800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512669312:12345-192.168.203.56@tcp:4:dd.0 Request processed in 24647us (25620us total) trans 4294973628 rc 0/0 00000100:00100000:0.0:1789760755.924334:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6339 00000100:00100000:0.0:1789760755.924344:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 735389049485 time 735463956612 state 0x2 ref 0 00000100:00100000:3.0:1789760755.929873:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760755.930002:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512669440 00000100:00100000:2.0:1789760755.930012:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512669440 00000100:00100000:2.0:1789760755.930075:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 735389049485 time 735463956612 state 0x0 ref 1 00000100:00100000:2.0:1789760755.930090:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 735464217028 time 735463956612 state 0x1 ref 1 00000100:00100000:2.0:1789760755.930150:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 735464217028 time 735494449365 state 0x0 ref 1 00000100:00100000:2.0:1789760755.930163:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2877b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512669440:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760755.930220:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25845760, mtime: 0x6aad94f3, atime: 0x0, ctime: 0x6aad94f3, blocks: 0xc807 00010000:00010000:2.0:1789760755.930237:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2edb4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.930254:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=735s v=(0 0) 00010000:00010000:2.0:1789760755.930259:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760755.930266:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2edb4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25845760->25849855] (req 25845760->25849855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.930280:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2edb4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25845760->25849855] (req 25845760->25849855) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760755.961846:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25845760 -> 25849856 00010000:00010000:2.0:1789760755.961864:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2edb4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25845760->25849855] (req 25845760->25849855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.961878:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2edb4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25845760->25849855] (req 25845760->25849855) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.961894:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2edb4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25845760->25849855] (req 25845760->25849855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.961913:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760755.961918:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2edb4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25845760->25849855] (req 25845760->25849855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.961928:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2edb4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25845760->25849855] (req 25845760->25849855) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.961948:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2edb4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25845760->25849855] (req 25845760->25849855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.961957:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2edb4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25845760->25849855] (req 25845760->25849855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760755.962549:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2877b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2361:x1876699512669440:12345-192.168.203.56@tcp:4:dd.0 Request processed in 31888us (32185us total) trans 4294973629 rc 0/0 00000100:00100000:2.0:1789760755.962563:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6340 00000100:00100000:2.0:1789760755.962573:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 735464217028 time 735494449365 state 0x2 ref 0 00000001:00080000:1.0:1789760755.962653:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973629 is committed 00000100:00100000:0.0:1789760755.967505:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760755.967581:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512669568 00000100:00100000:2.0:1789760755.967590:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512669568 00000100:00100000:2.0:1789760755.967639:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 735464217028 time 735494449365 state 0x0 ref 1 00000100:00100000:2.0:1789760755.967654:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 735494709781 time 735494449365 state 0x1 ref 1 00000100:00100000:2.0:1789760755.967755:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 735494709781 time 735532054310 state 0x0 ref 1 00000100:00100000:2.0:1789760755.967768:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac531e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512669568:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760755.967811:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25849856, mtime: 0x6aad94f3, atime: 0x0, ctime: 0x6aad94f3, blocks: 0xc807 00010000:00010000:2.0:1789760755.967821:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2edbb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.967835:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=735s v=(0 0) 00010000:00010000:2.0:1789760755.967840:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760755.967847:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2edbb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25849856->25853951] (req 25849856->25853951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.967862:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2edbb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25849856->25853951] (req 25849856->25853951) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760755.997409:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973630 is committed 00002000:00010000:3.0:1789760755.997424:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25849856 -> 25853952 00010000:00010000:3.0:1789760755.997440:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2edbb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25849856->25853951] (req 25849856->25853951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.997453:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2edbb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25849856->25853951] (req 25849856->25853951) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.997463:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2edbb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25849856->25853951] (req 25849856->25853951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.997471:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760755.997475:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2edbb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25849856->25853951] (req 25849856->25853951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.997485:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2edbb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25849856->25853951] (req 25849856->25853951) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.997504:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2edbb lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25849856->25853951] (req 25849856->25853951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.997517:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2edbb lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25849856->25853951] (req 25849856->25853951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760755.997609:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac531e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512669568:12345-192.168.203.56@tcp:4:dd.0 Request processed in 29837us (30108us total) trans 4294973630 rc 0/0 00000100:00100000:3.0:1789760755.997619:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6341 00000100:00100000:3.0:1789760755.997629:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 735494709781 time 735532054310 state 0x2 ref 0 00000100:00100000:3.0:1789760756.002808:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760756.002892:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512669696 00000100:00100000:0.0:1789760756.002907:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512669696 00000100:00100000:0.0:1789760756.002962:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 735494709781 time 735532054310 state 0x0 ref 1 00000100:00100000:0.0:1789760756.002977:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 735532314726 time 735532054310 state 0x1 ref 1 00000100:00100000:0.0:1789760756.003069:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 735532314726 time 735567367840 state 0x0 ref 1 00000100:00100000:0.0:1789760756.003083:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2874700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512669696:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760756.003131:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25853952, mtime: 0x6aad94f3, atime: 0x0, ctime: 0x6aad94f3, blocks: 0xc807 00010000:00010000:0.0:1789760756.003139:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2edc2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.003154:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=735s v=(0 0) 00010000:00010000:0.0:1789760756.003159:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760756.003166:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2edc2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25853952->25858047] (req 25853952->25858047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.003180:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2edc2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25853952->25858047] (req 25853952->25858047) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760756.040656:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973631 is committed 00002000:00010000:1.0:1789760756.040675:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25853952 -> 25858048 00010000:00010000:1.0:1789760756.040690:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2edc2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25853952->25858047] (req 25853952->25858047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.040731:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2edc2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25853952->25858047] (req 25853952->25858047) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.040740:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2edc2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25853952->25858047] (req 25853952->25858047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.040747:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760756.040751:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2edc2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25853952->25858047] (req 25853952->25858047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.040760:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2edc2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25853952->25858047] (req 25853952->25858047) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.040777:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2edc2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25853952->25858047] (req 25853952->25858047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.040789:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2edc2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25853952->25858047] (req 25853952->25858047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760756.040975:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2874700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512669696:12345-192.168.203.56@tcp:4:dd.0 Request processed in 37890us (38171us total) trans 4294973631 rc 0/0 00000100:00100000:1.0:1789760756.041266:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6342 00000100:00100000:1.0:1789760756.041277:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 735532314726 time 735567367840 state 0x2 ref 0 00000100:00100000:1.0:1789760756.044576:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760756.045415:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512669824 00000100:00100000:2.0:1789760756.045425:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512669824 00000100:00100000:2.0:1789760756.045478:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 735532314726 time 735567367840 state 0x0 ref 1 00000100:00100000:2.0:1789760756.045493:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 735567628256 time 735567367840 state 0x1 ref 1 00000100:00100000:2.0:1789760756.045573:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 735567628256 time 735609871238 state 0x0 ref 1 00000100:00100000:2.0:1789760756.045585:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac403b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512669824:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760756.045633:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25858048, mtime: 0x6aad94f3, atime: 0x0, ctime: 0x6aad94f3, blocks: 0xc807 00010000:00010000:2.0:1789760756.045643:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2edc9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.045660:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=735s v=(0 0) 00010000:00010000:2.0:1789760756.045664:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760756.045670:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2edc9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25858048->25862143] (req 25858048->25862143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.045682:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2edc9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25858048->25862143] (req 25858048->25862143) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.084437:0:7647:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:0.0:1789760756.088835:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973632 is committed 00002000:00010000:3.0:1789760756.089059:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25858048 -> 25862144 00010000:00010000:3.0:1789760756.089075:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2edc9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25858048->25862143] (req 25858048->25862143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.089088:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2edc9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25858048->25862143] (req 25858048->25862143) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.089097:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2edc9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25858048->25862143] (req 25858048->25862143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.089104:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760756.089109:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2edc9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25858048->25862143] (req 25858048->25862143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.089118:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2edc9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25858048->25862143] (req 25858048->25862143) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.090384:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2edc9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25858048->25862143] (req 25858048->25862143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.090395:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2edc9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25858048->25862143] (req 25858048->25862143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760756.090487:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac403b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512669824:12345-192.168.203.56@tcp:4:dd.0 Request processed in 44899us (45916us total) trans 4294973632 rc 0/0 00000100:00100000:3.0:1789760756.091257:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6343 00000100:00100000:3.0:1789760756.091269:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 735567628256 time 735609871238 state 0x2 ref 0 00000100:00100000:0.0:1789760756.096296:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760756.096409:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512669952 00000100:00100000:3.0:1789760756.096419:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512669952 00000100:00100000:3.0:1789760756.096474:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 735567628256 time 735609871238 state 0x0 ref 1 00000100:00100000:3.0:1789760756.096489:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 735610131654 time 735609871238 state 0x1 ref 1 00000100:00100000:3.0:1789760756.096519:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 735610131654 time 735660819109 state 0x0 ref 1 00000100:00100000:3.0:1789760756.096532:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac531ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512669952:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760756.096581:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25862144, mtime: 0x6aad94f3, atime: 0x0, ctime: 0x6aad94f3, blocks: 0xc807 00010000:00010000:3.0:1789760756.096591:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2edd0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.096606:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=735s v=(0 0) 00010000:00010000:3.0:1789760756.096612:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760756.096619:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2edd0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25862144->25866239] (req 25862144->25866239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.096646:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2edd0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25862144->25866239] (req 25862144->25866239) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760756.140450:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973633 is committed 00002000:00010000:3.0:1789760756.140527:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25862144 -> 25866240 00010000:00010000:3.0:1789760756.140541:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2edd0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25862144->25866239] (req 25862144->25866239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.140554:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2edd0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25862144->25866239] (req 25862144->25866239) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.140564:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2edd0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25862144->25866239] (req 25862144->25866239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.140572:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760756.140577:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2edd0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25862144->25866239] (req 25862144->25866239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.140586:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2edd0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25862144->25866239] (req 25862144->25866239) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.140603:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2edd0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25862144->25866239] (req 25862144->25866239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.140610:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2edd0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25862144->25866239] (req 25862144->25866239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760756.140865:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac531ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512669952:12345-192.168.203.56@tcp:4:dd.0 Request processed in 44329us (44572us total) trans 4294973633 rc 0/0 00000100:00100000:3.0:1789760756.140876:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6344 00000100:00100000:3.0:1789760756.140885:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 735610131654 time 735660819109 state 0x2 ref 0 00000100:00100000:0.0:1789760756.143451:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760756.143527:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512670080 00000100:00100000:1.0:1789760756.143536:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512670080 00000100:00100000:1.0:1789760756.143586:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 735610131654 time 735660819109 state 0x0 ref 1 00000100:00100000:1.0:1789760756.143600:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 735661079525 time 735660819109 state 0x1 ref 1 00000100:00100000:1.0:1789760756.143733:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 735661079525 time 735708031532 state 0x0 ref 1 00000100:00100000:1.0:1789760756.143747:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac531ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512670080:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760756.143794:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25866240, mtime: 0x6aad94f3, atime: 0x0, ctime: 0x6aad94f3, blocks: 0xc807 00010000:00010000:1.0:1789760756.143803:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2edd7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.143818:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=735s v=(0 0) 00010000:00010000:1.0:1789760756.143823:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760756.143830:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2edd7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25866240->25870335] (req 25866240->25870335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.143843:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2edd7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25866240->25870335] (req 25866240->25870335) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760756.175384:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973634 is committed 00002000:00010000:1.0:1789760756.175519:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25866240 -> 25870336 00010000:00010000:1.0:1789760756.175534:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2edd7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25866240->25870335] (req 25866240->25870335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.175547:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2edd7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25866240->25870335] (req 25866240->25870335) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.175557:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2edd7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25866240->25870335] (req 25866240->25870335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.175564:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760756.175569:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2edd7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25866240->25870335] (req 25866240->25870335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.175578:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2edd7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25866240->25870335] (req 25866240->25870335) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.175596:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2edd7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25866240->25870335] (req 25866240->25870335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.175604:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2edd7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25866240->25870335] (req 25866240->25870335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760756.175807:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac531ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512670080:12345-192.168.203.56@tcp:4:dd.0 Request processed in 32058us (32358us total) trans 4294973634 rc 0/0 00000100:00100000:1.0:1789760756.175825:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6345 00000100:00100000:1.0:1789760756.175835:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 735661079525 time 735708031532 state 0x2 ref 0 00000100:00100000:0.0:1789760756.181621:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760756.181719:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512670208 00000100:00100000:2.0:1789760756.181729:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512670208 00000100:00100000:2.0:1789760756.181780:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 735661079525 time 735708031532 state 0x0 ref 1 00000100:00100000:2.0:1789760756.181794:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 735708291948 time 735708031532 state 0x1 ref 1 00000100:00100000:2.0:1789760756.181824:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 735708291948 time 735746123680 state 0x0 ref 1 00000100:00100000:2.0:1789760756.181836:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac531e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512670208:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760756.181882:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25870336, mtime: 0x6aad94f3, atime: 0x0, ctime: 0x6aad94f3, blocks: 0xc807 00010000:00010000:2.0:1789760756.181890:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2edde lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.181906:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=735s v=(0 0) 00010000:00010000:2.0:1789760756.181914:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760756.181921:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2edde lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25870336->25874431] (req 25870336->25874431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.181935:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2edde lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25870336->25874431] (req 25870336->25874431) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760756.215562:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25870336 -> 25874432 00010000:00010000:2.0:1789760756.215593:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2edde lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25870336->25874431] (req 25870336->25874431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.215607:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2edde lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25870336->25874431] (req 25870336->25874431) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.215615:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2edde lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25870336->25874431] (req 25870336->25874431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.215622:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760756.215627:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2edde lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25870336->25874431] (req 25870336->25874431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.215636:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2edde lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25870336->25874431] (req 25870336->25874431) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.215654:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2edde lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25870336->25874431] (req 25870336->25874431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.215661:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2edde lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25870336->25874431] (req 25870336->25874431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760756.215779:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac531e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2361:x1876699512670208:12345-192.168.203.56@tcp:4:dd.0 Request processed in 33941us (34163us total) trans 4294973635 rc 0/0 00000100:00100000:2.0:1789760756.215800:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6346 00000100:00100000:2.0:1789760756.215810:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 735708291948 time 735746123680 state 0x2 ref 0 00000001:00080000:3.0:1789760756.216515:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973635 is committed 00000100:00100000:0.0:1789760756.223172:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760756.223265:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512670336 00000100:00100000:3.0:1789760756.223275:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512670336 00000100:00100000:3.0:1789760756.223329:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 735708291948 time 735746123680 state 0x0 ref 1 00000100:00100000:3.0:1789760756.223347:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 735746384096 time 735746123680 state 0x1 ref 1 00000100:00100000:3.0:1789760756.223380:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 735746384096 time 735787679518 state 0x0 ref 1 00000100:00100000:3.0:1789760756.223391:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac531dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512670336:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760756.223435:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25874432, mtime: 0x6aad94f3, atime: 0x0, ctime: 0x6aad94f3, blocks: 0xc807 00010000:00010000:3.0:1789760756.223443:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2ede5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.223457:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=735s v=(0 0) 00010000:00010000:3.0:1789760756.223461:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760756.223467:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2ede5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25874432->25878527] (req 25874432->25878527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.223479:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2ede5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25874432->25878527] (req 25874432->25878527) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760756.291113:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973636 is committed 00002000:00010000:3.0:1789760756.291300:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25874432 -> 25878528 00010000:00010000:3.0:1789760756.291323:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2ede5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25874432->25878527] (req 25874432->25878527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.291500:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2ede5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25874432->25878527] (req 25874432->25878527) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.291509:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2ede5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25874432->25878527] (req 25874432->25878527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.291521:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760756.291526:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2ede5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25874432->25878527] (req 25874432->25878527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.291535:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2ede5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25874432->25878527] (req 25874432->25878527) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.291627:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2ede5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25874432->25878527] (req 25874432->25878527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.291635:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2ede5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25874432->25878527] (req 25874432->25878527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760756.291907:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac531dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512670336:12345-192.168.203.56@tcp:4:dd.0 Request processed in 68504us (68730us total) trans 4294973636 rc 0/0 00000100:00100000:3.0:1789760756.291918:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6347 00000100:00100000:3.0:1789760756.291928:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 735746384096 time 735787679518 state 0x2 ref 0 00000100:00100000:0.0:1789760756.295526:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760756.300423:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512670464 00000100:00100000:1.0:1789760756.300436:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512670464 00000100:00100000:1.0:1789760756.303591:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 735746384096 time 735787679518 state 0x0 ref 1 00000100:00100000:1.0:1789760756.303610:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 735787939934 time 735787679518 state 0x1 ref 1 00000100:00100000:1.0:1789760756.303651:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 735787939934 time 735867950785 state 0x0 ref 1 00000100:00100000:1.0:1789760756.303665:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac531ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512670464:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760756.303732:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25878528, mtime: 0x6aad94f3, atime: 0x0, ctime: 0x6aad94f3, blocks: 0xc807 00010000:00010000:1.0:1789760756.303741:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2edec lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.303757:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=735s v=(0 0) 00010000:00010000:1.0:1789760756.303761:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760756.303767:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2edec lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25878528->25882623] (req 25878528->25882623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.303780:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2edec lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25878528->25882623] (req 25878528->25882623) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760756.372120:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973637 is committed 00002000:00010000:3.0:1789760756.372351:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25878528 -> 25882624 00010000:00010000:3.0:1789760756.372366:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2edec lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25878528->25882623] (req 25878528->25882623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.372379:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2edec lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25878528->25882623] (req 25878528->25882623) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.372394:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2edec lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25878528->25882623] (req 25878528->25882623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.372403:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760756.372408:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2edec lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25878528->25882623] (req 25878528->25882623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.372418:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2edec lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25878528->25882623] (req 25878528->25882623) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.372437:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2edec lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25878528->25882623] (req 25878528->25882623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.372445:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2edec lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25878528->25882623] (req 25878528->25882623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760756.372618:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac531ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512670464:12345-192.168.203.56@tcp:4:dd.0 Request processed in 68950us (77096us total) trans 4294973637 rc 0/0 00000100:00100000:3.0:1789760756.372628:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6348 00000100:00100000:3.0:1789760756.372638:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 735787939934 time 735867950785 state 0x2 ref 0 00000100:00100000:0.0:1789760756.376954:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760756.377043:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512670592 00000100:00100000:2.0:1789760756.377052:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512670592 00000100:00100000:2.0:1789760756.377101:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 735787939934 time 735867950785 state 0x0 ref 1 00000100:00100000:2.0:1789760756.377115:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 735868211201 time 735867950785 state 0x1 ref 1 00000100:00100000:2.0:1789760756.377194:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 735868211201 time 735941492893 state 0x0 ref 1 00000100:00100000:2.0:1789760756.377206:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac531f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512670592:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760756.377268:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25882624, mtime: 0x6aad94f3, atime: 0x0, ctime: 0x6aad94f3, blocks: 0xc807 00010000:00010000:2.0:1789760756.377277:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2edf3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.377292:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=735s v=(0 0) 00010000:00010000:2.0:1789760756.377296:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760756.377301:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2edf3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25882624->25886719] (req 25882624->25886719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.377326:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2edf3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25882624->25886719] (req 25882624->25886719) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760756.414550:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25882624 -> 25886720 00010000:00010000:2.0:1789760756.414567:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2edf3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25882624->25886719] (req 25882624->25886719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.414579:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2edf3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25882624->25886719] (req 25882624->25886719) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.414589:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2edf3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25882624->25886719] (req 25882624->25886719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.414597:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760756.414601:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2edf3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25882624->25886719] (req 25882624->25886719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.414610:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2edf3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25882624->25886719] (req 25882624->25886719) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.414629:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2edf3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25882624->25886719] (req 25882624->25886719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.414638:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2edf3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25882624->25886719] (req 25882624->25886719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760756.414829:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac531f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2361:x1876699512670592:12345-192.168.203.56@tcp:4:dd.0 Request processed in 37619us (37928us total) trans 4294973638 rc 0/0 00000100:00100000:2.0:1789760756.414844:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6349 00000100:00100000:2.0:1789760756.414854:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 735868211201 time 735941492893 state 0x2 ref 0 00000001:00080000:1.0:1789760756.415501:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973638 is committed 00000100:00100000:3.0:1789760756.418115:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760756.419082:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512670720 00000100:00100000:2.0:1789760756.419093:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512670720 00000100:00100000:2.0:1789760756.419149:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 735868211201 time 735941492893 state 0x0 ref 1 00000100:00100000:2.0:1789760756.419163:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 735941753309 time 735941492893 state 0x1 ref 1 00000100:00100000:2.0:1789760756.419185:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 735941753309 time 735983485543 state 0x0 ref 1 00000100:00100000:2.0:1789760756.419197:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2876a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512670720:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760756.419265:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25886720, mtime: 0x6aad94f3, atime: 0x0, ctime: 0x6aad94f3, blocks: 0xc807 00010000:00010000:2.0:1789760756.419274:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2edfa lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.419290:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=735s v=(0 0) 00010000:00010000:2.0:1789760756.419295:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760756.419301:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2edfa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25886720->25890815] (req 25886720->25890815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.419316:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2edfa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25886720->25890815] (req 25886720->25890815) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760756.464815:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25886720 -> 25890816 00010000:00010000:2.0:1789760756.464834:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2edfa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25886720->25890815] (req 25886720->25890815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.464848:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2edfa lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25886720->25890815] (req 25886720->25890815) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.464856:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2edfa lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25886720->25890815] (req 25886720->25890815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.464863:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760756.464868:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2edfa lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25886720->25890815] (req 25886720->25890815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.464877:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2edfa lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25886720->25890815] (req 25886720->25890815) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.464909:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2edfa lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25886720->25890815] (req 25886720->25890815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.464917:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2edfa lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25886720->25890815] (req 25886720->25890815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760756.465010:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2876a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2358:x1876699512670720:12345-192.168.203.56@tcp:4:dd.0 Request processed in 45810us (46897us total) trans 4294973639 rc 0/0 00000100:00100000:2.0:1789760756.465020:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6350 00000100:00100000:2.0:1789760756.465030:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 735941753309 time 735983485543 state 0x2 ref 0 00000001:00080000:3.0:1789760756.465995:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973639 is committed 00000100:00100000:2.0:1789760756.471218:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760756.471388:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512670848 00000100:00100000:3.0:1789760756.471398:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512670848 00000100:00100000:3.0:1789760756.471450:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 735941753309 time 735983485543 state 0x0 ref 1 00000100:00100000:3.0:1789760756.471465:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 735983745959 time 735983485543 state 0x1 ref 1 00000100:00100000:3.0:1789760756.471546:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 735983745959 time 736035844855 state 0x0 ref 1 00000100:00100000:3.0:1789760756.471561:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5305880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512670848:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760756.471610:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25890816, mtime: 0x6aad94f3, atime: 0x0, ctime: 0x6aad94f3, blocks: 0xc807 00010000:00010000:3.0:1789760756.471619:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2ee01 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.471635:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=736s v=(0 0) 00010000:00010000:3.0:1789760756.471639:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760756.471646:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2ee01 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25890816->25894911] (req 25890816->25894911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.471660:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2ee01 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25890816->25894911] (req 25890816->25894911) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760756.513249:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973640 is committed 00002000:00010000:2.0:1789760756.513434:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25890816 -> 25894912 00010000:00010000:2.0:1789760756.513449:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2ee01 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25890816->25894911] (req 25890816->25894911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.513462:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2ee01 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25890816->25894911] (req 25890816->25894911) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.513471:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2ee01 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25890816->25894911] (req 25890816->25894911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.513478:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760756.513483:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2ee01 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25890816->25894911] (req 25890816->25894911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.513492:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2ee01 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25890816->25894911] (req 25890816->25894911) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.513511:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2ee01 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25890816->25894911] (req 25890816->25894911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.513519:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2ee01 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25890816->25894911] (req 25890816->25894911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760756.513610:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5305880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512670848:12345-192.168.203.56@tcp:4:dd.0 Request processed in 42048us (42396us total) trans 4294973640 rc 0/0 00000100:00100000:2.0:1789760756.513629:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6351 00000100:00100000:2.0:1789760756.513638:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 735983745959 time 736035844855 state 0x2 ref 0 00000100:00100000:1.0:1789760756.517476:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760756.517896:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512670976 00000100:00100000:2.0:1789760756.517912:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512670976 00000100:00100000:2.0:1789760756.517965:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 735983745959 time 736035844855 state 0x0 ref 1 00000100:00100000:2.0:1789760756.517979:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 736036105271 time 736035844855 state 0x1 ref 1 00000100:00100000:2.0:1789760756.518007:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 736036105271 time 736082307624 state 0x0 ref 1 00000100:00100000:2.0:1789760756.518019:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4038000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512670976:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760756.518479:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25894912, mtime: 0x6aad94f3, atime: 0x0, ctime: 0x6aad94f3, blocks: 0xc807 00010000:00010000:2.0:1789760756.518489:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2ee08 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.518504:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=736s v=(0 0) 00010000:00010000:2.0:1789760756.518509:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760756.518515:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2ee08 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25894912->25899007] (req 25894912->25899007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.518530:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2ee08 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25894912->25899007] (req 25894912->25899007) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760756.569419:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973641 is committed 00002000:00010000:2.0:1789760756.569521:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25894912 -> 25899008 00002000:00010000:2.0:1789760756.569526:0:6708:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760755 -> 1789760756 00002000:00010000:2.0:1789760756.569528:0:6708:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760755 -> 1789760756 00010000:00010000:2.0:1789760756.569541:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2ee08 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25894912->25899007] (req 25894912->25899007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.569559:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2ee08 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25894912->25899007] (req 25894912->25899007) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.569568:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2ee08 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25894912->25899007] (req 25894912->25899007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.569575:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760756.569579:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2ee08 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25894912->25899007] (req 25894912->25899007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.569588:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2ee08 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25894912->25899007] (req 25894912->25899007) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.569607:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2ee08 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25894912->25899007] (req 25894912->25899007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.569627:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2ee08 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25894912->25899007] (req 25894912->25899007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760756.569989:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4038000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512670976:12345-192.168.203.56@tcp:4:dd.0 Request processed in 51965us (52517us total) trans 4294973641 rc 0/0 00000100:00100000:2.0:1789760756.569999:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6352 00000100:00100000:2.0:1789760756.570011:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 736036105271 time 736082307624 state 0x2 ref 0 00000100:00100000:0.0:1789760756.573841:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760756.574232:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512671104 00000100:00100000:2.0:1789760756.574242:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512671104 00000100:00100000:2.0:1789760756.574293:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 736036105271 time 736082307624 state 0x0 ref 1 00000100:00100000:2.0:1789760756.574309:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 736082568040 time 736082307624 state 0x1 ref 1 00000100:00100000:2.0:1789760756.574329:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 736082568040 time 736138629156 state 0x0 ref 1 00000100:00100000:2.0:1789760756.574341:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febea4b6450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512671104:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760756.574388:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25899008, mtime: 0x6aad94f4, atime: 0x0, ctime: 0x6aad94f4, blocks: 0xc807 00010000:00010000:2.0:1789760756.574397:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2ee0f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.574413:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=736s v=(0 0) 00010000:00010000:2.0:1789760756.574417:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760756.574425:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2ee0f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25899008->25903103] (req 25899008->25903103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.574438:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2ee0f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25899008->25903103] (req 25899008->25903103) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760756.607523:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973642 is committed 00002000:00010000:2.0:1789760756.607626:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25899008 -> 25903104 00010000:00010000:2.0:1789760756.607642:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2ee0f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25899008->25903103] (req 25899008->25903103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.607656:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2ee0f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25899008->25903103] (req 25899008->25903103) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.607672:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2ee0f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25899008->25903103] (req 25899008->25903103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.607678:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760756.607683:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2ee0f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25899008->25903103] (req 25899008->25903103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.607692:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2ee0f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25899008->25903103] (req 25899008->25903103) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.607740:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2ee0f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25899008->25903103] (req 25899008->25903103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.607749:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2ee0f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25899008->25903103] (req 25899008->25903103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760756.608068:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febea4b6450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512671104:12345-192.168.203.56@tcp:4:dd.0 Request processed in 33724us (34230us total) trans 4294973642 rc 0/0 00000100:00100000:2.0:1789760756.608078:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6353 00000100:00100000:2.0:1789760756.608088:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 736082568040 time 736138629156 state 0x2 ref 0 00000100:00100000:2.0:1789760756.612660:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760756.613487:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512671232 00000100:00100000:0.0:1789760756.613498:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512671232 00000100:00100000:0.0:1789760756.613553:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 736082568040 time 736138629156 state 0x0 ref 1 00000100:00100000:0.0:1789760756.613568:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 736138889572 time 736138629156 state 0x1 ref 1 00000100:00100000:0.0:1789760756.613597:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 736138889572 time 736177897409 state 0x0 ref 1 00000100:00100000:0.0:1789760756.613610:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5305f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512671232:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760756.613667:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25903104, mtime: 0x6aad94f4, atime: 0x0, ctime: 0x6aad94f4, blocks: 0xc807 00010000:00010000:0.0:1789760756.613676:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2ee16 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.613691:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=736s v=(0 0) 00010000:00010000:0.0:1789760756.613878:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760756.613886:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2ee16 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25903104->25907199] (req 25903104->25907199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.613909:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2ee16 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25903104->25907199] (req 25903104->25907199) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760756.651217:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973643 is committed 00002000:00010000:0.0:1789760756.651330:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25903104 -> 25907200 00010000:00010000:0.0:1789760756.651345:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2ee16 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25903104->25907199] (req 25903104->25907199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.651359:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2ee16 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25903104->25907199] (req 25903104->25907199) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.651367:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2ee16 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25903104->25907199] (req 25903104->25907199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.651374:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760756.651379:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2ee16 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25903104->25907199] (req 25903104->25907199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.651388:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2ee16 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25903104->25907199] (req 25903104->25907199) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.651407:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2ee16 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25903104->25907199] (req 25903104->25907199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.651415:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2ee16 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25903104->25907199] (req 25903104->25907199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760756.651510:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5305f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512671232:12345-192.168.203.56@tcp:4:dd.0 Request processed in 37898us (38854us total) trans 4294973643 rc 0/0 00000100:00100000:0.0:1789760756.651523:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6354 00000100:00100000:0.0:1789760756.651533:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 736138889572 time 736177897409 state 0x2 ref 0 00000100:00100000:0.0:1789760756.654571:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760756.654651:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512671360 00000100:00100000:3.0:1789760756.654660:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512671360 00000100:00100000:3.0:1789760756.654744:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 736138889572 time 736177897409 state 0x0 ref 1 00000100:00100000:3.0:1789760756.654758:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 736178157825 time 736177897409 state 0x1 ref 1 00000100:00100000:3.0:1789760756.654957:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 736178157825 time 736219255002 state 0x0 ref 1 00000100:00100000:3.0:1789760756.654972:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac531df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512671360:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760756.655016:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25907200, mtime: 0x6aad94f4, atime: 0x0, ctime: 0x6aad94f4, blocks: 0xc807 00010000:00010000:3.0:1789760756.655025:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2ee1d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.655040:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=736s v=(0 0) 00010000:00010000:3.0:1789760756.655123:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760756.655131:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2ee1d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25907200->25911295] (req 25907200->25911295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.655145:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2ee1d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25907200->25911295] (req 25907200->25911295) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760756.663916:0:3309:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8feac2875c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3309:1876699540226560:0@lo:400:kworker.0 00000100:00100000:3.0:1789760756.663998:0:3309:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760756.664396:0:3309:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8feac2876a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3309:1876699540227072:0@lo:400:kworker.0 00000100:00100000:3.0:1789760756.664429:0:3309:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760756.664463:0:3309:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1789760756.664469:0:3309:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8feac2877100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3309:1876699540226688:0@lo:400:kworker.0 00000100:00100000:3.0:1789760756.664490:0:3309:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760756.664528:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540226560 00000100:00100000:3.0:1789760756.664534:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540226560 00000100:00100000:3.0:1789760756.664575:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540226688 00000100:00100000:3.0:1789760756.664578:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540226688 00000100:00100000:3.0:1789760756.664606:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 486 00000100:00100000:3.0:1789760756.664615:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2fb6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3309:x1876699540226560:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760756.664665:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2fb6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3309:x1876699540226560:12345-0@lo:400:kworker.0 Request processed in 51us (667us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760756.664673:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 486 00000100:00100000:3.0:1789760756.664689:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 487 00000100:00100000:3.0:1789760756.664720:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac28a1500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3309:x1876699540226688:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760756.664749:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac28a1500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3309:x1876699540226688:12345-0@lo:400:kworker.0 Request processed in 54us (258us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760756.664754:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 487 00000100:00100000:3.0:1789760756.664770:0:6705:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540227072 00000100:00100000:3.0:1789760756.665023:0:6705:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540227072 00000100:00100000:3.0:1789760756.665136:0:6705:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 185 00000100:00100000:3.0:1789760756.665145:0:6705:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac28a2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3309:x1876699540227072:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760756.665179:0:6705:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac28a2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3309:x1876699540227072:12345-0@lo:400:kworker.0 Request processed in 36us (750us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760756.665185:0:6705:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 185 00000100:00080000:3.0:1789760756.665208:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:3.0:1789760756.665213:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789760756.665257:0:3309:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8feac2877100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3309:1876699540226688:0@lo:400:kworker.0 00000100:00100000:3.0:1789760756.665268:0:3309:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8feac2875c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3309:1876699540226560:0@lo:400:kworker.0 00000100:00100000:3.0:1789760756.665276:0:3309:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8feac2876a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3309:1876699540227072:0@lo:400:kworker.0 00000100:00100000:1.0:1789760756.665488:0:3310:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8feac28a3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3310:1876699540227200:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1789760756.665524:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760756.665606:0:3311:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8feac2877b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3311:1876699540226816:0@lo:400:kworker.0 00000100:00100000:0.0:1789760756.665648:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760756.665669:0:3311:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [3->2] 00000100:00100000:0.0:1789760756.665675:0:3311:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8feac2875880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:6df98d54-eb52-4c5c-8fc9-ee9d74f90f98:3311:1876699540226432:0@lo:400:kworker.0 00000100:00100000:0.0:1789760756.665720:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760756.665742:0:3311:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8feac2874700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3311:1876699540226944:0@lo:400:kworker.0 00000100:00100000:0.0:1789760756.665763:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760756.665801:0:6704:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540226816 00000100:00100000:0.0:1789760756.665805:0:6704:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540226816 00000100:00100000:0.0:1789760756.665837:0:6704:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 186 00000100:00100000:0.0:1789760756.665844:0:6704:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac531c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3311:x1876699540226816:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760756.665884:0:6704:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac531c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3311:x1876699540226816:12345-0@lo:400:kworker.0 Request processed in 36us (231us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760756.665890:0:6704:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 186 00000100:00100000:0.0:1789760756.665922:0:5842:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540226432 00000100:00100000:0.0:1789760756.665926:0:5842:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540226432 00000100:00080000:3.0:1789760756.666202:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1789760756.666213:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540226944 00000100:00100000:3.0:1789760756.666216:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540226944 00000100:00100000:3.0:1789760756.666258:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 488 00000100:00100000:3.0:1789760756.666265:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac531fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3311:x1876699540226944:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760756.666302:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac531fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3311:x1876699540226944:12345-0@lo:400:kworker.0 Request processed in 37us (538us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760756.666308:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 488 00000100:00080000:3.0:1789760756.666325:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789760756.666390:0:5842:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 403 00000100:00100000:0.0:1789760756.666397:0:5842:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febe00c5450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:6df98d54-eb52-4c5c-8fc9-ee9d74f90f98+19:3311:x1876699540226432:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760756.666428:0:5842:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febe00c5450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:6df98d54-eb52-4c5c-8fc9-ee9d74f90f98+19:3311:x1876699540226432:12345-0@lo:400:kworker.0 Request processed in 32us (709us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760756.666435:0:5842:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 403 00000100:00100000:0.0:1789760756.666474:0:3311:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8feac2875880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:6df98d54-eb52-4c5c-8fc9-ee9d74f90f98:3311:1876699540226432:0@lo:400:kworker.0 00000100:00100000:0.0:1789760756.666484:0:3311:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8feac2874700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3311:1876699540226944:0@lo:400:kworker.0 00000100:00100000:0.0:1789760756.666492:0:3311:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8feac2877b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3311:1876699540226816:0@lo:400:kworker.0 00000100:00100000:2.0:1789760756.668014:0:6707:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540227200 00000100:00100000:2.0:1789760756.668023:0:6707:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540227200 00000100:00100000:2.0:1789760756.668361:0:6707:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 202 00000100:00100000:2.0:1789760756.668372:0:6707:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac403b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3310:x1876699540227200:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1789760756.668394:0:6707:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 528023552 pending 1703936 free 3832545280 avail 3826253824 00000100:00100000:2.0:1789760756.668466:0:6707:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac403b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3310:x1876699540227200:12345-0@lo:13:osp-pre-0-0.0 Request processed in 96us (2941us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760756.668473:0:6707:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 202 00000100:00100000:0.0:1789760756.668737:0:3310:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8feac28a3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3310:1876699540227200:0@lo:13:osp-pre-0-0.0 00000001:00080000:1.0:1789760756.714191:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973644 is committed 00002000:00010000:2.0:1789760756.716268:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25907200 -> 25911296 00010000:00010000:2.0:1789760756.716287:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2ee1d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25907200->25911295] (req 25907200->25911295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.716300:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2ee1d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25907200->25911295] (req 25907200->25911295) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.716308:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2ee1d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25907200->25911295] (req 25907200->25911295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.716315:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760756.716320:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2ee1d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25907200->25911295] (req 25907200->25911295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.716329:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2ee1d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25907200->25911295] (req 25907200->25911295) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.716414:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2ee1d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25907200->25911295] (req 25907200->25911295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.716423:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2ee1d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25907200->25911295] (req 25907200->25911295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760756.716529:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac531df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512671360:12345-192.168.203.56@tcp:4:dd.0 Request processed in 61556us (61971us total) trans 4294973644 rc 0/0 00000100:00100000:2.0:1789760756.716539:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6355 00000100:00100000:2.0:1789760756.716548:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 736178157825 time 736219255002 state 0x2 ref 0 00000100:00100000:1.0:1789760756.724330:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760756.724525:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512671488 00000100:00100000:2.0:1789760756.724536:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512671488 00000100:00100000:2.0:1789760756.724588:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 736178157825 time 736219255002 state 0x0 ref 1 00000100:00100000:2.0:1789760756.724604:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 736219515418 time 736219255002 state 0x1 ref 1 00000100:00100000:2.0:1789760756.724635:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 736219515418 time 736288934706 state 0x0 ref 1 00000100:00100000:2.0:1789760756.724647:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4038700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512671488:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760756.724731:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25911296, mtime: 0x6aad94f4, atime: 0x0, ctime: 0x6aad94f4, blocks: 0xc807 00010000:00010000:2.0:1789760756.724740:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2ee24 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.724757:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=736s v=(0 0) 00010000:00010000:2.0:1789760756.724761:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760756.724767:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2ee24 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25911296->25915391] (req 25911296->25915391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.724782:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2ee24 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25911296->25915391] (req 25911296->25915391) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760756.786730:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25911296 -> 25915392 00010000:00010000:3.0:1789760756.786751:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2ee24 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25911296->25915391] (req 25911296->25915391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.786763:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2ee24 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25911296->25915391] (req 25911296->25915391) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.786784:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2ee24 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25911296->25915391] (req 25911296->25915391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.786792:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760756.786797:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2ee24 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25911296->25915391] (req 25911296->25915391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.786805:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2ee24 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25911296->25915391] (req 25911296->25915391) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.786833:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2ee24 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25911296->25915391] (req 25911296->25915391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.786840:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2ee24 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25911296->25915391] (req 25911296->25915391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760756.786944:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4038700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2359:x1876699512671488:12345-192.168.203.56@tcp:4:dd.0 Request processed in 62293us (62618us total) trans 4294973645 rc 0/0 00000100:00100000:3.0:1789760756.786954:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6356 00000100:00100000:3.0:1789760756.786964:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 736219515418 time 736288934706 state 0x2 ref 0 00000001:00080000:0.0:1789760756.788092:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973645 is committed 00000100:00100000:2.0:1789760756.793261:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760756.793463:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512671616 00000100:00100000:0.0:1789760756.793473:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512671616 00000100:00100000:0.0:1789760756.793529:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 736219515418 time 736288934706 state 0x0 ref 1 00000100:00100000:0.0:1789760756.793545:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 736289195122 time 736288934706 state 0x1 ref 1 00000100:00100000:0.0:1789760756.793579:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 736289195122 time 736357878219 state 0x0 ref 1 00000100:00100000:0.0:1789760756.793592:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5307480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512671616:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760756.793744:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25915392, mtime: 0x6aad94f4, atime: 0x0, ctime: 0x6aad94f4, blocks: 0xc807 00010000:00010000:0.0:1789760756.793754:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2ee2b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.793769:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=736s v=(0 0) 00010000:00010000:0.0:1789760756.793906:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760756.793913:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2ee2b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25915392->25919487] (req 25915392->25919487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.793928:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2ee2b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25915392->25919487] (req 25915392->25919487) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760756.864660:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25915392 -> 25919488 00010000:00010000:3.0:1789760756.864679:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2ee2b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25915392->25919487] (req 25915392->25919487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.864692:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2ee2b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25915392->25919487] (req 25915392->25919487) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.864719:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2ee2b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25915392->25919487] (req 25915392->25919487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.864726:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760756.864731:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2ee2b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25915392->25919487] (req 25915392->25919487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.864741:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2ee2b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25915392->25919487] (req 25915392->25919487) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.864759:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2ee2b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25915392->25919487] (req 25915392->25919487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.864772:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2ee2b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25915392->25919487] (req 25915392->25919487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760756.864862:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5307480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2361:x1876699512671616:12345-192.168.203.56@tcp:4:dd.0 Request processed in 71267us (71604us total) trans 4294973646 rc 0/0 00000100:00100000:3.0:1789760756.864873:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6357 00000100:00100000:3.0:1789760756.864883:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 736289195122 time 736357878219 state 0x2 ref 0 00000001:00080000:0.0:1789760756.865782:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973646 is committed 00000100:00100000:0.0:1789760756.868312:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760756.868406:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512671744 00000100:00100000:1.0:1789760756.868416:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512671744 00000100:00100000:1.0:1789760756.868477:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 736289195122 time 736357878219 state 0x0 ref 1 00000100:00100000:1.0:1789760756.868493:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 736358138635 time 736357878219 state 0x1 ref 1 00000100:00100000:1.0:1789760756.868524:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 736358138635 time 736432823756 state 0x0 ref 1 00000100:00100000:1.0:1789760756.868536:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac531f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512671744:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760756.868585:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25919488, mtime: 0x6aad94f4, atime: 0x0, ctime: 0x6aad94f4, blocks: 0xc807 00010000:00010000:1.0:1789760756.868595:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2ee32 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.868621:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=736s v=(0 0) 00010000:00010000:1.0:1789760756.868626:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760756.868632:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2ee32 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25919488->25923583] (req 25919488->25923583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.868646:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2ee32 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25919488->25923583] (req 25919488->25923583) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760756.903062:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25919488 -> 25923584 00010000:00010000:3.0:1789760756.903081:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2ee32 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25919488->25923583] (req 25919488->25923583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.903094:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2ee32 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25919488->25923583] (req 25919488->25923583) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.903103:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2ee32 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25919488->25923583] (req 25919488->25923583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.903110:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760756.903115:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2ee32 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25919488->25923583] (req 25919488->25923583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.903124:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2ee32 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25919488->25923583] (req 25919488->25923583) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.903147:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2ee32 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25919488->25923583] (req 25919488->25923583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.903155:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2ee32 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25919488->25923583] (req 25919488->25923583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760756.903264:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac531f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2360:x1876699512671744:12345-192.168.203.56@tcp:4:dd.0 Request processed in 34723us (34954us total) trans 4294973647 rc 0/0 00000100:00100000:3.0:1789760756.903274:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6358 00000100:00100000:3.0:1789760756.903283:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 736358138635 time 736432823756 state 0x2 ref 0 00000001:00080000:0.0:1789760756.903522:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973647 is committed 00000100:00100000:1.0:1789760756.907459:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760756.907561:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512671872 00000100:00100000:3.0:1789760756.907570:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512671872 00000100:00100000:3.0:1789760756.907680:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 736358138635 time 736432823756 state 0x0 ref 1 00000100:00100000:3.0:1789760756.908913:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 736433084172 time 736432823756 state 0x1 ref 1 00000100:00100000:3.0:1789760756.908951:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 736433084172 time 736473250702 state 0x0 ref 1 00000100:00100000:3.0:1789760756.908969:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac403aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512671872:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760756.909020:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25923584, mtime: 0x6aad94f4, atime: 0x0, ctime: 0x6aad94f4, blocks: 0xc807 00010000:00010000:3.0:1789760756.909030:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2ee39 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.909046:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=736s v=(0 0) 00010000:00010000:3.0:1789760756.909051:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760756.909057:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2ee39 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25923584->25927679] (req 25923584->25927679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.909071:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2ee39 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25923584->25927679] (req 25923584->25927679) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760756.941544:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973648 is committed 00002000:00010000:3.0:1789760756.941827:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25923584 -> 25927680 00010000:00010000:3.0:1789760756.941842:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2ee39 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25923584->25927679] (req 25923584->25927679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.941855:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2ee39 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25923584->25927679] (req 25923584->25927679) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.941863:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2ee39 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25923584->25927679] (req 25923584->25927679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.941870:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760756.941874:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2ee39 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25923584->25927679] (req 25923584->25927679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.941883:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2ee39 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25923584->25927679] (req 25923584->25927679) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.941995:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2ee39 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25923584->25927679] (req 25923584->25927679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.942003:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2ee39 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25923584->25927679] (req 25923584->25927679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760756.942089:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac403aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512671872:12345-192.168.203.56@tcp:4:dd.0 Request processed in 33123us (34635us total) trans 4294973648 rc 0/0 00000100:00100000:3.0:1789760756.942099:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6359 00000100:00100000:3.0:1789760756.942109:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 736433084172 time 736473250702 state 0x2 ref 0 00000100:00100000:2.0:1789760756.949581:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760756.949739:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512672000 00000100:00100000:3.0:1789760756.949750:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512672000 00000100:00100000:2.0:1789760756.949932:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 736433084172 time 736473250702 state 0x0 ref 1 00000100:00100000:2.0:1789760756.949948:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 736473511118 time 736473250702 state 0x1 ref 1 00000100:00100000:2.0:1789760756.949980:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 736473511118 time 736514280244 state 0x0 ref 1 00000100:00100000:2.0:1789760756.949994:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5304700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512672000:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760756.950040:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25927680, mtime: 0x6aad94f4, atime: 0x0, ctime: 0x6aad94f4, blocks: 0xc807 00010000:00010000:2.0:1789760756.950049:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2ee40 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.950064:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=736s v=(0 0) 00010000:00010000:2.0:1789760756.950068:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760756.950075:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2ee40 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25927680->25931775] (req 25927680->25931775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.950089:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2ee40 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25927680->25931775] (req 25927680->25931775) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760757.018956:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973649 is committed 00002000:00010000:3.0:1789760757.019116:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25927680 -> 25931776 00010000:00010000:3.0:1789760757.019132:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2ee40 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25927680->25931775] (req 25927680->25931775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.019144:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2ee40 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25927680->25931775] (req 25927680->25931775) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.019153:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2ee40 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25927680->25931775] (req 25927680->25931775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.019160:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760757.019165:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2ee40 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25927680->25931775] (req 25927680->25931775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.019174:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2ee40 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25927680->25931775] (req 25927680->25931775) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.019191:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2ee40 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25927680->25931775] (req 25927680->25931775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.019203:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2ee40 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25927680->25931775] (req 25927680->25931775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760757.019316:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5304700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512672000:12345-192.168.203.56@tcp:4:dd.0 Request processed in 69320us (69739us total) trans 4294973649 rc 0/0 00000100:00100000:3.0:1789760757.019326:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6360 00000100:00100000:3.0:1789760757.019335:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 736473511118 time 736514280244 state 0x2 ref 0 00000100:00100000:0.0:1789760757.030790:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760757.031739:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512672128 00000100:00100000:3.0:1789760757.031750:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512672128 00000100:00100000:3.0:1789760757.031815:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 736473511118 time 736514280244 state 0x0 ref 1 00000100:00100000:3.0:1789760757.031832:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 736514540660 time 736514280244 state 0x1 ref 1 00000100:00100000:3.0:1789760757.031862:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 736514540660 time 736596161927 state 0x0 ref 1 00000100:00100000:3.0:1789760757.031876:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac531c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512672128:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760757.031925:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25931776, mtime: 0x6aad94f4, atime: 0x0, ctime: 0x6aad94f4, blocks: 0xc807 00010000:00010000:3.0:1789760757.031934:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2ee47 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1789760757.032071:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:3.0:1789760757.032762:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=736s v=(0 0) 00010000:00010000:3.0:1789760757.032767:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760757.032774:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2ee47 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25931776->25935871] (req 25931776->25935871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.032794:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2ee47 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25931776->25935871] (req 25931776->25935871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760757.098257:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973650 is committed 00002000:00010000:0.0:1789760757.098377:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25931776 -> 25935872 00010000:00010000:0.0:1789760757.098395:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2ee47 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25931776->25935871] (req 25931776->25935871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.098408:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2ee47 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25931776->25935871] (req 25931776->25935871) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.098417:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2ee47 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25931776->25935871] (req 25931776->25935871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.098424:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760757.098429:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2ee47 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25931776->25935871] (req 25931776->25935871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.098439:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2ee47 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25931776->25935871] (req 25931776->25935871) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.098460:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2ee47 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25931776->25935871] (req 25931776->25935871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.098473:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2ee47 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25931776->25935871] (req 25931776->25935871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760757.098574:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac531c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512672128:12345-192.168.203.56@tcp:4:dd.0 Request processed in 66696us (67789us total) trans 4294973650 rc 0/0 00000100:00100000:0.0:1789760757.098583:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6361 00000100:00100000:0.0:1789760757.098593:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 736514540660 time 736596161927 state 0x2 ref 0 00000100:00100000:0.0:1789760757.105195:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760757.105429:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512672256 00000100:00100000:3.0:1789760757.105439:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512672256 00000100:00100000:3.0:1789760757.105494:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 736514540660 time 736596161927 state 0x0 ref 1 00000100:00100000:3.0:1789760757.105510:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 736596422343 time 736596161927 state 0x1 ref 1 00000100:00100000:3.0:1789760757.105574:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 736596422343 time 736669872913 state 0x0 ref 1 00000100:00100000:3.0:1789760757.105586:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac531d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512672256:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760757.105643:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25935872, mtime: 0x6aad94f4, atime: 0x0, ctime: 0x6aad94f4, blocks: 0xc807 00010000:00010000:3.0:1789760757.105652:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2ee4e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.105668:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=736s v=(0 0) 00010000:00010000:3.0:1789760757.105672:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760757.105679:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2ee4e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25935872->25939967] (req 25935872->25939967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.105737:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2ee4e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25935872->25939967] (req 25935872->25939967) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.110887:0:5836:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:2.0:1789760757.147316:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973651 is committed 00002000:00010000:0.0:1789760757.147514:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25935872 -> 25939968 00010000:00010000:0.0:1789760757.147530:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2ee4e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25935872->25939967] (req 25935872->25939967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.147544:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2ee4e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25935872->25939967] (req 25935872->25939967) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.147553:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2ee4e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25935872->25939967] (req 25935872->25939967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.147561:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760757.147566:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2ee4e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25935872->25939967] (req 25935872->25939967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.147575:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2ee4e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25935872->25939967] (req 25935872->25939967) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.147599:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2ee4e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25935872->25939967] (req 25935872->25939967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.147607:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2ee4e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25935872->25939967] (req 25935872->25939967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760757.147909:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac531d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512672256:12345-192.168.203.56@tcp:4:dd.0 Request processed in 42321us (42718us total) trans 4294973651 rc 0/0 00000100:00100000:0.0:1789760757.147921:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6362 00000100:00100000:0.0:1789760757.147931:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 736596422343 time 736669872913 state 0x2 ref 0 00000100:00100000:2.0:1789760757.152153:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760757.152270:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512672384 00000100:00100000:3.0:1789760757.152279:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512672384 00000100:00100000:3.0:1789760757.152334:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 736596422343 time 736669872913 state 0x0 ref 1 00000100:00100000:3.0:1789760757.152360:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 736670133329 time 736669872913 state 0x1 ref 1 00000100:00100000:3.0:1789760757.152389:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 736670133329 time 736716688942 state 0x0 ref 1 00000100:00100000:3.0:1789760757.152402:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5304380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512672384:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760757.152445:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25939968, mtime: 0x6aad94f4, atime: 0x0, ctime: 0x6aad94f4, blocks: 0xc807 00010000:00010000:3.0:1789760757.152454:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2ee55 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.152470:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=736s v=(0 0) 00010000:00010000:3.0:1789760757.152474:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760757.152480:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2ee55 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25939968->25944063] (req 25939968->25944063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.152493:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2ee55 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25939968->25944063] (req 25939968->25944063) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760757.171206:0:3312:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1789760757.171236:0:3312:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8feac5306680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3312:1876699540227328:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789760757.171300:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760757.171395:0:6707:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540227328 00000100:00100000:3.0:1789760757.171404:0:6707:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540227328 00000100:00100000:3.0:1789760757.171457:0:6707:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 203 00000100:00100000:3.0:1789760757.171476:0:6707:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5307b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3312:x1876699540227328:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1789760757.171510:0:6707:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3859808256 avail 3857711104 00000100:00100000:3.0:1789760757.171557:0:6707:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5307b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3312:x1876699540227328:12345-0@lo:13:osp-pre-1-0.0 Request processed in 82us (256us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760757.171564:0:6707:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 203 00000100:00100000:2.0:1789760757.171836:0:3312:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8feac5306680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3312:1876699540227328:0@lo:13:osp-pre-1-0.0 00000001:00080000:3.0:1789760757.198787:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973652 is committed 00002000:00010000:0.0:1789760757.198949:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25939968 -> 25944064 00010000:00010000:0.0:1789760757.198965:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2ee55 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25939968->25944063] (req 25939968->25944063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.198978:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2ee55 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25939968->25944063] (req 25939968->25944063) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.199043:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2ee55 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25939968->25944063] (req 25939968->25944063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.199051:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760757.199056:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2ee55 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25939968->25944063] (req 25939968->25944063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.199070:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2ee55 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25939968->25944063] (req 25939968->25944063) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.199089:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2ee55 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25939968->25944063] (req 25939968->25944063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.199096:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2ee55 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25939968->25944063] (req 25939968->25944063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760757.199187:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5304380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512672384:12345-192.168.203.56@tcp:4:dd.0 Request processed in 46784us (47037us total) trans 4294973652 rc 0/0 00000100:00100000:0.0:1789760757.199197:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6363 00000100:00100000:0.0:1789760757.199205:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 736670133329 time 736716688942 state 0x2 ref 0 00000100:00100000:1.0:1789760757.202323:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760757.202376:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512672512 00000100:00100000:0.0:1789760757.202387:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512672512 00000100:00100000:0.0:1789760757.202446:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 736670133329 time 736716688942 state 0x0 ref 1 00000100:00100000:0.0:1789760757.202460:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 736716949358 time 736716688942 state 0x1 ref 1 00000100:00100000:0.0:1789760757.202492:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 736716949358 time 736766791635 state 0x0 ref 1 00000100:00100000:0.0:1789760757.202505:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4039880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512672512:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760757.202551:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25944064, mtime: 0x6aad94f4, atime: 0x0, ctime: 0x6aad94f4, blocks: 0xc807 00010000:00010000:0.0:1789760757.202561:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2ee5c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.202577:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=736s v=(0 0) 00010000:00010000:0.0:1789760757.202581:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760757.202587:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2ee5c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25944064->25948159] (req 25944064->25948159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.202601:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2ee5c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25944064->25948159] (req 25944064->25948159) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760757.250418:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973653 is committed 00002000:00010000:1.0:1789760757.250539:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25944064 -> 25948160 00010000:00010000:1.0:1789760757.250554:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2ee5c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25944064->25948159] (req 25944064->25948159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.250567:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2ee5c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25944064->25948159] (req 25944064->25948159) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.250575:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2ee5c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25944064->25948159] (req 25944064->25948159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.250582:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760757.250586:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2ee5c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25944064->25948159] (req 25944064->25948159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.250594:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2ee5c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25944064->25948159] (req 25944064->25948159) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.250684:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2ee5c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25944064->25948159] (req 25944064->25948159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.250832:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2ee5c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25944064->25948159] (req 25944064->25948159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760757.251023:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4039880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512672512:12345-192.168.203.56@tcp:4:dd.0 Request processed in 48516us (48705us total) trans 4294973653 rc 0/0 00000100:00100000:1.0:1789760757.251033:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6364 00000100:00100000:1.0:1789760757.251042:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 736716949358 time 736766791635 state 0x2 ref 0 00000100:00100000:1.0:1789760757.253404:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760757.253471:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512672640 00000100:00100000:0.0:1789760757.253480:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512672640 00000100:00100000:0.0:1789760757.253527:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 736716949358 time 736766791635 state 0x0 ref 1 00000100:00100000:0.0:1789760757.253542:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 736767052051 time 736766791635 state 0x1 ref 1 00000100:00100000:0.0:1789760757.253571:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 736767052051 time 736817870854 state 0x0 ref 1 00000100:00100000:0.0:1789760757.253584:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4039180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512672640:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760757.253639:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25948160, mtime: 0x6aad94f4, atime: 0x0, ctime: 0x6aad94f4, blocks: 0xc807 00010000:00010000:0.0:1789760757.253648:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2ee63 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.253664:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=736s v=(0 0) 00010000:00010000:0.0:1789760757.253669:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760757.253675:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2ee63 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25948160->25952255] (req 25948160->25952255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.253907:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2ee63 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25948160->25952255] (req 25948160->25952255) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760757.288232:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973654 is committed 00002000:00010000:0.0:1789760757.288329:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25948160 -> 25952256 00010000:00010000:0.0:1789760757.288346:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2ee63 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25948160->25952255] (req 25948160->25952255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.288359:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2ee63 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25948160->25952255] (req 25948160->25952255) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.288368:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2ee63 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25948160->25952255] (req 25948160->25952255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.288375:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760757.288380:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2ee63 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25948160->25952255] (req 25948160->25952255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.288389:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2ee63 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25948160->25952255] (req 25948160->25952255) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.288408:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2ee63 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25948160->25952255] (req 25948160->25952255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.288416:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2ee63 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25948160->25952255] (req 25948160->25952255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760757.288549:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4039180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512672640:12345-192.168.203.56@tcp:4:dd.0 Request processed in 34962us (35148us total) trans 4294973654 rc 0/0 00000100:00100000:0.0:1789760757.288559:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6365 00000100:00100000:0.0:1789760757.288569:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 736767052051 time 736817870854 state 0x2 ref 0 00000100:00100000:2.0:1789760757.292274:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760757.292375:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512672768 00000100:00100000:0.0:1789760757.292384:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512672768 00000100:00100000:0.0:1789760757.292436:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 736767052051 time 736817870854 state 0x0 ref 1 00000100:00100000:0.0:1789760757.292450:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 736818131270 time 736817870854 state 0x1 ref 1 00000100:00100000:0.0:1789760757.292516:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 736818131270 time 736856814918 state 0x0 ref 1 00000100:00100000:0.0:1789760757.292529:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5306680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512672768:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760757.292579:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25952256, mtime: 0x6aad94f4, atime: 0x0, ctime: 0x6aad94f4, blocks: 0xc807 00010000:00010000:0.0:1789760757.292588:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2ee6a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.292604:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=736s v=(0 0) 00010000:00010000:0.0:1789760757.292677:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760757.292686:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2ee6a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25952256->25956351] (req 25952256->25956351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.296465:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2ee6a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25952256->25956351] (req 25952256->25956351) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760757.334270:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25952256 -> 25956352 00010000:00010000:1.0:1789760757.334291:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2ee6a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25952256->25956351] (req 25952256->25956351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.334305:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2ee6a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25952256->25956351] (req 25952256->25956351) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.334315:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2ee6a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25952256->25956351] (req 25952256->25956351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.334324:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760757.334330:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2ee6a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25952256->25956351] (req 25952256->25956351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.334339:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2ee6a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25952256->25956351] (req 25952256->25956351) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.334359:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2ee6a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25952256->25956351] (req 25952256->25956351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.334367:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2ee6a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25952256->25956351] (req 25952256->25956351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760757.334461:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5306680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2361:x1876699512672768:12345-192.168.203.56@tcp:4:dd.0 Request processed in 41929us (42191us total) trans 4294973655 rc 0/0 00000100:00100000:1.0:1789760757.334471:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6366 00000100:00100000:1.0:1789760757.334479:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 736818131270 time 736856814918 state 0x2 ref 0 00000001:00080000:2.0:1789760757.334553:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973655 is committed 00000100:00100000:1.0:1789760757.339837:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760757.339943:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512672896 00000100:00100000:2.0:1789760757.339954:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512672896 00000100:00100000:2.0:1789760757.340005:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 736818131270 time 736856814918 state 0x0 ref 1 00000100:00100000:2.0:1789760757.340021:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 736857075334 time 736856814918 state 0x1 ref 1 00000100:00100000:2.0:1789760757.340058:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 736857075334 time 736904357798 state 0x0 ref 1 00000100:00100000:2.0:1789760757.340071:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4039500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512672896:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760757.340117:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25956352, mtime: 0x6aad94f4, atime: 0x0, ctime: 0x6aad94f4, blocks: 0xc807 00010000:00010000:2.0:1789760757.340127:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2ee71 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.340142:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=736s v=(0 0) 00010000:00010000:2.0:1789760757.340146:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760757.340152:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2ee71 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25956352->25960447] (req 25956352->25960447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.340165:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2ee71 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25956352->25960447] (req 25956352->25960447) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760757.382662:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25956352 -> 25960448 00010000:00010000:2.0:1789760757.382682:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2ee71 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25956352->25960447] (req 25956352->25960447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.382721:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2ee71 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25956352->25960447] (req 25956352->25960447) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.382730:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2ee71 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25956352->25960447] (req 25956352->25960447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.382738:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760757.382743:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2ee71 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25956352->25960447] (req 25956352->25960447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.382752:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2ee71 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25956352->25960447] (req 25956352->25960447) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.382771:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2ee71 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25956352->25960447] (req 25956352->25960447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.382778:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2ee71 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25956352->25960447] (req 25956352->25960447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760757.382860:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4039500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2360:x1876699512672896:12345-192.168.203.56@tcp:4:dd.0 Request processed in 42786us (43026us total) trans 4294973656 rc 0/0 00000100:00100000:2.0:1789760757.382870:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6367 00000100:00100000:2.0:1789760757.382880:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 736857075334 time 736904357798 state 0x2 ref 0 00000001:00080000:1.0:1789760757.383401:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973656 is committed 00000100:00100000:3.0:1789760757.387533:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760757.387944:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512673024 00000100:00100000:0.0:1789760757.387954:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512673024 00000100:00100000:0.0:1789760757.388006:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 736857075334 time 736904357798 state 0x0 ref 1 00000100:00100000:0.0:1789760757.388022:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 736904618214 time 736904357798 state 0x1 ref 1 00000100:00100000:0.0:1789760757.388125:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 736904618214 time 736952423418 state 0x0 ref 1 00000100:00100000:0.0:1789760757.388139:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac28a3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512673024:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760757.388188:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25960448, mtime: 0x6aad94f4, atime: 0x0, ctime: 0x6aad94f4, blocks: 0xc807 00010000:00010000:0.0:1789760757.388197:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2ee78 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.388211:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=736s v=(0 0) 00010000:00010000:0.0:1789760757.388228:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760757.388235:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2ee78 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25960448->25964543] (req 25960448->25964543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.388249:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2ee78 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25960448->25964543] (req 25960448->25964543) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760757.431317:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973657 is committed 00002000:00010000:1.0:1789760757.431510:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25960448 -> 25964544 00010000:00010000:1.0:1789760757.431524:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2ee78 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25960448->25964543] (req 25960448->25964543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.431537:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2ee78 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25960448->25964543] (req 25960448->25964543) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.431545:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2ee78 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25960448->25964543] (req 25960448->25964543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.431551:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760757.431556:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2ee78 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25960448->25964543] (req 25960448->25964543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.431564:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2ee78 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25960448->25964543] (req 25960448->25964543) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.431581:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2ee78 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25960448->25964543] (req 25960448->25964543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.431588:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2ee78 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25960448->25964543] (req 25960448->25964543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760757.431686:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac28a3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512673024:12345-192.168.203.56@tcp:4:dd.0 Request processed in 43545us (44156us total) trans 4294973657 rc 0/0 00000100:00100000:1.0:1789760757.431725:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6368 00000100:00100000:1.0:1789760757.431736:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 736904618214 time 736952423418 state 0x2 ref 0 00000100:00100000:3.0:1789760757.434670:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760757.434914:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512673152 00000100:00100000:1.0:1789760757.434924:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512673152 00000100:00100000:1.0:1789760757.435041:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 736904618214 time 736952423418 state 0x0 ref 1 00000100:00100000:1.0:1789760757.435057:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 736952683834 time 736952423418 state 0x1 ref 1 00000100:00100000:1.0:1789760757.435086:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 736952683834 time 736999386337 state 0x0 ref 1 00000100:00100000:1.0:1789760757.435098:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2877b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512673152:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760757.435140:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25964544, mtime: 0x6aad94f4, atime: 0x0, ctime: 0x6aad94f4, blocks: 0xc807 00010000:00010000:1.0:1789760757.435148:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2ee7f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.435163:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=736s v=(0 0) 00010000:00010000:1.0:1789760757.435167:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760757.435173:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2ee7f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25964544->25968639] (req 25964544->25968639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.435187:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2ee7f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25964544->25968639] (req 25964544->25968639) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760757.465522:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973658 is committed 00002000:00010000:3.0:1789760757.465686:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25964544 -> 25968640 00010000:00010000:3.0:1789760757.465744:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2ee7f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25964544->25968639] (req 25964544->25968639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.465757:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2ee7f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25964544->25968639] (req 25964544->25968639) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.465767:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2ee7f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25964544->25968639] (req 25964544->25968639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.465775:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760757.465779:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2ee7f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25964544->25968639] (req 25964544->25968639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.465789:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2ee7f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25964544->25968639] (req 25964544->25968639) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.465811:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2ee7f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25964544->25968639] (req 25964544->25968639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.465823:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2ee7f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25964544->25968639] (req 25964544->25968639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760757.466114:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2877b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512673152:12345-192.168.203.56@tcp:4:dd.0 Request processed in 31011us (31445us total) trans 4294973658 rc 0/0 00000100:00100000:3.0:1789760757.466124:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6369 00000100:00100000:3.0:1789760757.466135:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 736952683834 time 736999386337 state 0x2 ref 0 00000100:00100000:2.0:1789760757.471583:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760757.471694:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512673280 00000100:00100000:1.0:1789760757.471915:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512673280 00000100:00100000:1.0:1789760757.472039:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 736952683834 time 736999386337 state 0x0 ref 1 00000100:00100000:1.0:1789760757.472054:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 736999646753 time 736999386337 state 0x1 ref 1 00000100:00100000:1.0:1789760757.472084:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 736999646753 time 737036384498 state 0x0 ref 1 00000100:00100000:1.0:1789760757.472096:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5307100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512673280:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760757.472143:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25968640, mtime: 0x6aad94f4, atime: 0x0, ctime: 0x6aad94f4, blocks: 0xc807 00010000:00010000:1.0:1789760757.472152:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2ee86 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.472166:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=737s v=(0 0) 00010000:00010000:1.0:1789760757.472170:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760757.472177:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2ee86 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25968640->25972735] (req 25968640->25972735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.472196:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2ee86 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25968640->25972735] (req 25968640->25972735) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760757.516874:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973659 is committed 00002000:00010000:2.0:1789760757.517116:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25968640 -> 25972736 00010000:00010000:2.0:1789760757.517193:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2ee86 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25968640->25972735] (req 25968640->25972735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.517206:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2ee86 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25968640->25972735] (req 25968640->25972735) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.517223:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2ee86 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25968640->25972735] (req 25968640->25972735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.517231:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760757.517235:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2ee86 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25968640->25972735] (req 25968640->25972735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.517244:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2ee86 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25968640->25972735] (req 25968640->25972735) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.517263:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2ee86 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25968640->25972735] (req 25968640->25972735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.517270:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2ee86 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25968640->25972735] (req 25968640->25972735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760757.517521:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5307100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512673280:12345-192.168.203.56@tcp:4:dd.0 Request processed in 45423us (45943us total) trans 4294973659 rc 0/0 00000100:00100000:2.0:1789760757.517531:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6370 00000100:00100000:2.0:1789760757.517541:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 736999646753 time 737036384498 state 0x2 ref 0 00000100:00100000:2.0:1789760757.521672:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760757.521855:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512673408 00000100:00100000:1.0:1789760757.521864:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512673408 00000100:00100000:1.0:1789760757.521913:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 736999646753 time 737036384498 state 0x0 ref 1 00000100:00100000:1.0:1789760757.522042:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 737036644914 time 737036384498 state 0x1 ref 1 00000100:00100000:1.0:1789760757.522073:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 737036644914 time 737086371696 state 0x0 ref 1 00000100:00100000:1.0:1789760757.522086:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5304a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512673408:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760757.522248:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25972736, mtime: 0x6aad94f4, atime: 0x0, ctime: 0x6aad94f4, blocks: 0xc807 00010000:00010000:1.0:1789760757.522258:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2ee8d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.522274:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=737s v=(0 0) 00010000:00010000:1.0:1789760757.522393:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760757.522399:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2ee8d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25972736->25976831] (req 25972736->25976831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.522413:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2ee8d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25972736->25976831] (req 25972736->25976831) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760757.552750:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973660 is committed 00002000:00010000:3.0:1789760757.552883:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25972736 -> 25976832 00002000:00010000:3.0:1789760757.552896:0:6710:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760756 -> 1789760757 00002000:00010000:3.0:1789760757.552900:0:6710:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760756 -> 1789760757 00010000:00010000:3.0:1789760757.552914:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2ee8d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25972736->25976831] (req 25972736->25976831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.552936:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2ee8d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25972736->25976831] (req 25972736->25976831) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.552947:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2ee8d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25972736->25976831] (req 25972736->25976831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.552954:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760757.552959:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2ee8d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25972736->25976831] (req 25972736->25976831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.552972:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2ee8d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25972736->25976831] (req 25972736->25976831) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.552993:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2ee8d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25972736->25976831] (req 25972736->25976831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.553001:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2ee8d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25972736->25976831] (req 25972736->25976831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760757.553093:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5304a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512673408:12345-192.168.203.56@tcp:4:dd.0 Request processed in 31004us (31425us total) trans 4294973660 rc 0/0 00000100:00100000:3.0:1789760757.553104:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6371 00000100:00100000:3.0:1789760757.553112:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 737036644914 time 737086371696 state 0x2 ref 0 00000100:00100000:3.0:1789760757.555560:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760757.555994:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512673536 00000100:00100000:1.0:1789760757.556004:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512673536 00000100:00100000:1.0:1789760757.556063:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 737036644914 time 737086371696 state 0x0 ref 1 00000100:00100000:1.0:1789760757.556080:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 737086632112 time 737086371696 state 0x1 ref 1 00000100:00100000:1.0:1789760757.556408:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 737086632112 time 737120706942 state 0x0 ref 1 00000100:00100000:1.0:1789760757.556422:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2874700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512673536:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760757.556471:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25976832, mtime: 0x6aad94f5, atime: 0x0, ctime: 0x6aad94f5, blocks: 0xc807 00010000:00010000:1.0:1789760757.556489:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2ee94 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.556506:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=737s v=(0 0) 00010000:00010000:1.0:1789760757.556510:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760757.556516:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2ee94 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25976832->25980927] (req 25976832->25980927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.556530:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2ee94 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25976832->25980927] (req 25976832->25980927) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760757.593327:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25976832 -> 25980928 00010000:00010000:3.0:1789760757.593347:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2ee94 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25976832->25980927] (req 25976832->25980927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.593361:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2ee94 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25976832->25980927] (req 25976832->25980927) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.593371:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2ee94 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25976832->25980927] (req 25976832->25980927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.593385:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760757.593390:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2ee94 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25976832->25980927] (req 25976832->25980927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.593400:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2ee94 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25976832->25980927] (req 25976832->25980927) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.593419:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2ee94 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25976832->25980927] (req 25976832->25980927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.593427:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2ee94 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25976832->25980927] (req 25976832->25980927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760757.593515:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2874700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2361:x1876699512673536:12345-192.168.203.56@tcp:4:dd.0 Request processed in 37090us (37957us total) trans 4294973661 rc 0/0 00000100:00100000:3.0:1789760757.593525:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6372 00000100:00100000:3.0:1789760757.593534:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 737086632112 time 737120706942 state 0x2 ref 0 00000001:00080000:0.0:1789760757.594226:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973661 is committed 00000100:00100000:2.0:1789760757.597427:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760757.597554:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512673664 00000100:00100000:1.0:1789760757.597564:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512673664 00000100:00100000:1.0:1789760757.597617:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 737086632112 time 737120706942 state 0x0 ref 1 00000100:00100000:1.0:1789760757.597632:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 737120967358 time 737120706942 state 0x1 ref 1 00000100:00100000:1.0:1789760757.597694:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 737120967358 time 737161993652 state 0x0 ref 1 00000100:00100000:1.0:1789760757.597729:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5307800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512673664:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760757.597775:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25980928, mtime: 0x6aad94f5, atime: 0x0, ctime: 0x6aad94f5, blocks: 0xc807 00010000:00010000:1.0:1789760757.597784:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2ee9b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.597800:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=737s v=(0 0) 00010000:00010000:1.0:1789760757.597804:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760757.597810:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2ee9b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25980928->25985023] (req 25980928->25985023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.597824:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2ee9b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25980928->25985023] (req 25980928->25985023) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760757.635690:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973662 is committed 00002000:00010000:2.0:1789760757.635767:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25980928 -> 25985024 00010000:00010000:2.0:1789760757.635783:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2ee9b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25980928->25985023] (req 25980928->25985023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.635796:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2ee9b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25980928->25985023] (req 25980928->25985023) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.635805:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2ee9b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25980928->25985023] (req 25980928->25985023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.635813:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760757.635817:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2ee9b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25980928->25985023] (req 25980928->25985023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.635826:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2ee9b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25980928->25985023] (req 25980928->25985023) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.635844:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2ee9b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25980928->25985023] (req 25980928->25985023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.635852:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2ee9b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25980928->25985023] (req 25980928->25985023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760757.635947:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5307800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512673664:12345-192.168.203.56@tcp:4:dd.0 Request processed in 38218us (38527us total) trans 4294973662 rc 0/0 00000100:00100000:2.0:1789760757.635958:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6373 00000100:00100000:2.0:1789760757.635967:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 737120967358 time 737161993652 state 0x2 ref 0 00000100:00100000:2.0:1789760757.646366:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760757.646456:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512673792 00000100:00100000:3.0:1789760757.646465:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512673792 00000100:00100000:3.0:1789760757.646524:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 737120967358 time 737161993652 state 0x0 ref 1 00000100:00100000:3.0:1789760757.646538:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 737162254068 time 737161993652 state 0x1 ref 1 00000100:00100000:3.0:1789760757.646565:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 737162254068 time 737210865475 state 0x0 ref 1 00000100:00100000:3.0:1789760757.646581:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5305880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512673792:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760757.646638:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25985024, mtime: 0x6aad94f5, atime: 0x0, ctime: 0x6aad94f5, blocks: 0xc807 00010000:00010000:3.0:1789760757.646647:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2eea2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.646662:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=737s v=(0 0) 00010000:00010000:3.0:1789760757.646666:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760757.646675:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2eea2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25985024->25989119] (req 25985024->25989119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.646688:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2eea2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25985024->25989119] (req 25985024->25989119) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760757.708549:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973663 is committed 00002000:00010000:2.0:1789760757.708770:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25985024 -> 25989120 00010000:00010000:2.0:1789760757.708794:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2eea2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25985024->25989119] (req 25985024->25989119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.708866:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2eea2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25985024->25989119] (req 25985024->25989119) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.708876:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2eea2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25985024->25989119] (req 25985024->25989119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.708884:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760757.708890:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2eea2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25985024->25989119] (req 25985024->25989119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.708899:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2eea2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25985024->25989119] (req 25985024->25989119) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.708919:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2eea2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25985024->25989119] (req 25985024->25989119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.708933:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2eea2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25985024->25989119] (req 25985024->25989119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760757.709037:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5305880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512673792:12345-192.168.203.56@tcp:4:dd.0 Request processed in 62452us (62674us total) trans 4294973663 rc 0/0 00000100:00100000:2.0:1789760757.709046:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6374 00000100:00100000:2.0:1789760757.709056:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 737162254068 time 737210865475 state 0x2 ref 0 00000100:00100000:3.0:1789760757.713548:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760757.715089:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512673920 00000100:00100000:1.0:1789760757.715102:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512673920 00000100:00100000:1.0:1789760757.715160:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 737162254068 time 737210865475 state 0x0 ref 1 00000100:00100000:1.0:1789760757.715176:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 737211125891 time 737210865475 state 0x1 ref 1 00000100:00100000:1.0:1789760757.715224:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 737211125891 time 737279515231 state 0x0 ref 1 00000100:00100000:1.0:1789760757.715238:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2875880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512673920:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760757.715287:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25989120, mtime: 0x6aad94f5, atime: 0x0, ctime: 0x6aad94f5, blocks: 0xc807 00010000:00010000:1.0:1789760757.715296:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2eea9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.715322:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=737s v=(0 0) 00010000:00010000:1.0:1789760757.715327:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760757.715333:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2eea9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25989120->25993215] (req 25989120->25993215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.715347:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2eea9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25989120->25993215] (req 25989120->25993215) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760757.749996:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973664 is committed 00002000:00010000:1.0:1789760757.750064:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25989120 -> 25993216 00010000:00010000:1.0:1789760757.750081:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2eea9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25989120->25993215] (req 25989120->25993215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.750095:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2eea9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25989120->25993215] (req 25989120->25993215) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.750104:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2eea9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25989120->25993215] (req 25989120->25993215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.750112:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760757.750116:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2eea9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25989120->25993215] (req 25989120->25993215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.750126:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2eea9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25989120->25993215] (req 25989120->25993215) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.750144:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2eea9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25989120->25993215] (req 25989120->25993215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.750152:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2eea9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25989120->25993215] (req 25989120->25993215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760757.750291:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2875880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512673920:12345-192.168.203.56@tcp:4:dd.0 Request processed in 35051us (36748us total) trans 4294973664 rc 0/0 00000100:00100000:1.0:1789760757.750300:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6375 00000100:00100000:1.0:1789760757.750308:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 737211125891 time 737279515231 state 0x2 ref 0 00000100:00100000:1.0:1789760757.756229:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760757.756494:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512674048 00000100:00100000:2.0:1789760757.756504:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512674048 00000100:00100000:2.0:1789760757.756553:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 737211125891 time 737279515231 state 0x0 ref 1 00000100:00100000:2.0:1789760757.757028:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 737279775647 time 737279515231 state 0x1 ref 1 00000100:00100000:2.0:1789760757.757098:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 737279775647 time 737321396989 state 0x0 ref 1 00000100:00100000:2.0:1789760757.757112:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4038a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512674048:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760757.757157:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25993216, mtime: 0x6aad94f5, atime: 0x0, ctime: 0x6aad94f5, blocks: 0xc807 00010000:00010000:2.0:1789760757.757166:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2eeb0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.757180:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=737s v=(0 0) 00010000:00010000:2.0:1789760757.757185:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760757.757190:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2eeb0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25993216->25997311] (req 25993216->25997311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.757231:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2eeb0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25993216->25997311] (req 25993216->25997311) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760757.791452:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973665 is committed 00002000:00010000:3.0:1789760757.791574:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25993216 -> 25997312 00010000:00010000:3.0:1789760757.791593:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2eeb0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25993216->25997311] (req 25993216->25997311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.791606:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2eeb0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25993216->25997311] (req 25993216->25997311) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.791615:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2eeb0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25993216->25997311] (req 25993216->25997311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.791622:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760757.791627:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2eeb0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25993216->25997311] (req 25993216->25997311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.791636:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2eeb0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25993216->25997311] (req 25993216->25997311) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.791655:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2eeb0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25993216->25997311] (req 25993216->25997311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.791666:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2eeb0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25993216->25997311] (req 25993216->25997311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760757.791777:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4038a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512674048:12345-192.168.203.56@tcp:4:dd.0 Request processed in 34663us (35561us total) trans 4294973665 rc 0/0 00000100:00100000:3.0:1789760757.791787:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6376 00000100:00100000:3.0:1789760757.791797:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 737279775647 time 737321396989 state 0x2 ref 0 00000100:00100000:2.0:1789760757.797504:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760757.797588:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512674176 00000100:00100000:0.0:1789760757.797602:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512674176 00000100:00100000:0.0:1789760757.797655:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 737279775647 time 737321396989 state 0x0 ref 1 00000100:00100000:0.0:1789760757.797670:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 737321657405 time 737321396989 state 0x1 ref 1 00000100:00100000:0.0:1789760757.797772:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 737321657405 time 737362070471 state 0x0 ref 1 00000100:00100000:0.0:1789760757.797786:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5306300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512674176:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760757.797834:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25997312, mtime: 0x6aad94f5, atime: 0x0, ctime: 0x6aad94f5, blocks: 0xc807 00010000:00010000:0.0:1789760757.797843:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2eeb7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.797858:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=737s v=(0 0) 00010000:00010000:0.0:1789760757.797862:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760757.797868:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2eeb7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25997312->26001407] (req 25997312->26001407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.797881:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2eeb7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25997312->26001407] (req 25997312->26001407) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760757.829554:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973666 is committed 00002000:00010000:1.0:1789760757.829645:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25997312 -> 26001408 00010000:00010000:1.0:1789760757.829660:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2eeb7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25997312->26001407] (req 25997312->26001407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.829672:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2eeb7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25997312->26001407] (req 25997312->26001407) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.834015:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2eeb7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25997312->26001407] (req 25997312->26001407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.834036:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760757.834041:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2eeb7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25997312->26001407] (req 25997312->26001407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.834053:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2eeb7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25997312->26001407] (req 25997312->26001407) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.834081:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2eeb7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25997312->26001407] (req 25997312->26001407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.834090:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2eeb7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25997312->26001407] (req 25997312->26001407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760757.834401:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5306300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512674176:12345-192.168.203.56@tcp:4:dd.0 Request processed in 36612us (36903us total) trans 4294973666 rc 0/0 00000100:00100000:1.0:1789760757.834411:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6377 00000100:00100000:1.0:1789760757.834420:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 737321657405 time 737362070471 state 0x2 ref 0 00000100:00100000:3.0:1789760757.840043:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760757.840647:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512674304 00000100:00100000:1.0:1789760757.840657:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512674304 00000100:00100000:1.0:1789760757.840790:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 737321657405 time 737362070471 state 0x0 ref 1 00000100:00100000:1.0:1789760757.840805:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 737362330887 time 737362070471 state 0x1 ref 1 00000100:00100000:0.0:1789760757.840947:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 737362330887 time 737405244656 state 0x0 ref 1 00000100:00100000:0.0:1789760757.840964:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2876a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512674304:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760757.841020:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26001408, mtime: 0x6aad94f5, atime: 0x0, ctime: 0x6aad94f5, blocks: 0xc807 00010000:00010000:0.0:1789760757.841029:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2eebe lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.841045:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=737s v=(0 0) 00010000:00010000:0.0:1789760757.841049:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760757.841055:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2eebe lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26001408->26005503] (req 26001408->26005503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.841069:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2eebe lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26001408->26005503] (req 26001408->26005503) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760757.876536:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973667 is committed 00002000:00010000:0.0:1789760757.876634:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26001408 -> 26005504 00010000:00010000:0.0:1789760757.876649:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2eebe lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26001408->26005503] (req 26001408->26005503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.876661:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2eebe lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26001408->26005503] (req 26001408->26005503) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.876670:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2eebe lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26001408->26005503] (req 26001408->26005503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.876677:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760757.876681:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2eebe lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26001408->26005503] (req 26001408->26005503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.876690:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2eebe lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26001408->26005503] (req 26001408->26005503) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.876824:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2eebe lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26001408->26005503] (req 26001408->26005503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.876834:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2eebe lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26001408->26005503] (req 26001408->26005503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760757.877414:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2876a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512674304:12345-192.168.203.56@tcp:4:dd.0 Request processed in 35970us (36897us total) trans 4294973667 rc 0/0 00000100:00100000:0.0:1789760757.877426:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6378 00000100:00100000:0.0:1789760757.877435:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 737362330887 time 737405244656 state 0x2 ref 0 00000100:00100000:3.0:1789760757.881393:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760757.881513:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512674432 00000100:00100000:2.0:1789760757.881527:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512674432 00000100:00100000:2.0:1789760757.881582:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 737362330887 time 737405244656 state 0x0 ref 1 00000100:00100000:2.0:1789760757.881597:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 737405505072 time 737405244656 state 0x1 ref 1 00000100:00100000:2.0:1789760757.881653:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 737405505072 time 737445952630 state 0x0 ref 1 00000100:00100000:2.0:1789760757.881665:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2875c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512674432:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760757.881757:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26005504, mtime: 0x6aad94f5, atime: 0x0, ctime: 0x6aad94f5, blocks: 0xc807 00010000:00010000:2.0:1789760757.881766:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2eec5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.881781:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=737s v=(0 0) 00010000:00010000:2.0:1789760757.881785:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760757.881791:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2eec5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26005504->26009599] (req 26005504->26009599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.881803:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2eec5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26005504->26009599] (req 26005504->26009599) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760757.923500:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973668 is committed 00002000:00010000:2.0:1789760757.923692:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26005504 -> 26009600 00010000:00010000:2.0:1789760757.923764:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2eec5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26005504->26009599] (req 26005504->26009599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.923777:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2eec5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26005504->26009599] (req 26005504->26009599) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.923786:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2eec5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26005504->26009599] (req 26005504->26009599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.923793:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760757.923799:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2eec5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26005504->26009599] (req 26005504->26009599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.923807:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2eec5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26005504->26009599] (req 26005504->26009599) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.923827:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2eec5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26005504->26009599] (req 26005504->26009599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.923834:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2eec5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26005504->26009599] (req 26005504->26009599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760757.924020:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2875c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512674432:12345-192.168.203.56@tcp:4:dd.0 Request processed in 42352us (42631us total) trans 4294973668 rc 0/0 00000100:00100000:2.0:1789760757.924034:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6379 00000100:00100000:2.0:1789760757.924044:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 737405505072 time 737445952630 state 0x2 ref 0 00000100:00100000:2.0:1789760757.934054:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760757.934146:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512674560 00000100:00100000:0.0:1789760757.934155:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512674560 00000100:00100000:0.0:1789760757.934208:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 737405505072 time 737445952630 state 0x0 ref 1 00000100:00100000:0.0:1789760757.934244:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 737446213046 time 737445952630 state 0x1 ref 1 00000100:00100000:0.0:1789760757.934275:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 737446213046 time 737498574832 state 0x0 ref 1 00000100:00100000:0.0:1789760757.934287:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5306a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512674560:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760757.934332:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26009600, mtime: 0x6aad94f5, atime: 0x0, ctime: 0x6aad94f5, blocks: 0xc807 00010000:00010000:0.0:1789760757.934340:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2eecc lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.934356:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=737s v=(0 0) 00010000:00010000:0.0:1789760757.934360:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760757.934366:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2eecc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26009600->26013695] (req 26009600->26013695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.934389:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2eecc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26009600->26013695] (req 26009600->26013695) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760757.962827:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973669 is committed 00002000:00010000:1.0:1789760757.963171:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26009600 -> 26013696 00010000:00010000:1.0:1789760757.963188:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2eecc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26009600->26013695] (req 26009600->26013695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.963200:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2eecc lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26009600->26013695] (req 26009600->26013695) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.963208:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2eecc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26009600->26013695] (req 26009600->26013695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.963230:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760757.963235:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2eecc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26009600->26013695] (req 26009600->26013695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.963245:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2eecc lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26009600->26013695] (req 26009600->26013695) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.963264:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2eecc lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26009600->26013695] (req 26009600->26013695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.963271:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2eecc lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26009600->26013695] (req 26009600->26013695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760757.963398:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5306a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512674560:12345-192.168.203.56@tcp:4:dd.0 Request processed in 29108us (29348us total) trans 4294973669 rc 0/0 00000100:00100000:1.0:1789760757.963408:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6380 00000100:00100000:1.0:1789760757.963418:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 737446213046 time 737498574832 state 0x2 ref 0 00000100:00100000:3.0:1789760757.970265:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760757.972674:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512674688 00000100:00100000:0.0:1789760757.972688:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512674688 00000100:00100000:0.0:1789760757.973008:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 737446213046 time 737498574832 state 0x0 ref 1 00000100:00100000:0.0:1789760757.973025:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 737498835248 time 737498574832 state 0x1 ref 1 00000100:00100000:0.0:1789760757.973112:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 737498835248 time 737537410944 state 0x0 ref 1 00000100:00100000:0.0:1789760757.973126:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2877100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512674688:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760757.973174:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26013696, mtime: 0x6aad94f5, atime: 0x0, ctime: 0x6aad94f5, blocks: 0xc807 00010000:00010000:0.0:1789760757.973182:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2eed3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.973197:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=737s v=(0 0) 00010000:00010000:0.0:1789760757.973202:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760757.973208:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2eed3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26013696->26017791] (req 26013696->26017791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.973231:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2eed3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26013696->26017791] (req 26013696->26017791) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760758.008687:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973670 is committed 00002000:00010000:0.0:1789760758.008763:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26013696 -> 26017792 00010000:00010000:0.0:1789760758.008778:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2eed3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26013696->26017791] (req 26013696->26017791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.008790:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2eed3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26013696->26017791] (req 26013696->26017791) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.008799:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2eed3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26013696->26017791] (req 26013696->26017791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.008807:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760758.008811:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2eed3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26013696->26017791] (req 26013696->26017791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.008820:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2eed3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26013696->26017791] (req 26013696->26017791) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.008838:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2eed3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26013696->26017791] (req 26013696->26017791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.008846:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2eed3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26013696->26017791] (req 26013696->26017791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760758.008941:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2877100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512674688:12345-192.168.203.56@tcp:4:dd.0 Request processed in 35815us (38681us total) trans 4294973670 rc 0/0 00000100:00100000:0.0:1789760758.008951:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6381 00000100:00100000:0.0:1789760758.008961:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 737498835248 time 737537410944 state 0x2 ref 0 00000100:00100000:2.0:1789760758.014638:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760758.015063:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512674816 00000100:00100000:0.0:1789760758.015073:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512674816 00000100:00100000:0.0:1789760758.015125:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 737498835248 time 737537410944 state 0x0 ref 1 00000100:00100000:0.0:1789760758.015141:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 737537671360 time 737537410944 state 0x1 ref 1 00000100:00100000:0.0:1789760758.015236:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 737537671360 time 737579534646 state 0x0 ref 1 00000100:00100000:0.0:1789760758.015250:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4130000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512674816:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760758.015297:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26017792, mtime: 0x6aad94f5, atime: 0x0, ctime: 0x6aad94f5, blocks: 0xc807 00010000:00010000:0.0:1789760758.015305:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2eeda lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.015320:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=737s v=(0 0) 00010000:00010000:0.0:1789760758.015325:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760758.015331:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2eeda lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26017792->26021887] (req 26017792->26021887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.015344:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2eeda lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26017792->26021887] (req 26017792->26021887) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760758.046428:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26017792 -> 26021888 00010000:00010000:0.0:1789760758.046447:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2eeda lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26017792->26021887] (req 26017792->26021887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.046460:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2eeda lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26017792->26021887] (req 26017792->26021887) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.046469:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2eeda lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26017792->26021887] (req 26017792->26021887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.046477:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760758.046481:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2eeda lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26017792->26021887] (req 26017792->26021887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.046491:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2eeda lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26017792->26021887] (req 26017792->26021887) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.046509:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2eeda lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26017792->26021887] (req 26017792->26021887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.046516:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2eeda lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26017792->26021887] (req 26017792->26021887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760758.047145:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973671 is committed 00000100:00100000:0.0:1789760758.047248:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4130000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512674816:12345-192.168.203.56@tcp:4:dd.0 Request processed in 31994us (32610us total) trans 4294973671 rc 0/0 00000100:00100000:0.0:1789760758.047258:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6382 00000100:00100000:0.0:1789760758.047277:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 737537671360 time 737579534646 state 0x2 ref 0 00000100:00100000:1.0:1789760758.055014:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760758.055119:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512674944 00000100:00100000:0.0:1789760758.055129:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512674944 00000100:00100000:0.0:1789760758.055183:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 737537671360 time 737579534646 state 0x0 ref 1 00000100:00100000:0.0:1789760758.055199:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 737579795062 time 737579534646 state 0x1 ref 1 00000100:00100000:0.0:1789760758.055237:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 737579795062 time 737619536822 state 0x0 ref 1 00000100:00100000:0.0:1789760758.055250:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac403ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512674944:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760758.055299:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26021888, mtime: 0x6aad94f5, atime: 0x0, ctime: 0x6aad94f5, blocks: 0xc807 00010000:00010000:0.0:1789760758.055309:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2eee1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.055325:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=737s v=(0 0) 00010000:00010000:0.0:1789760758.055330:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760758.055335:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2eee1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26021888->26025983] (req 26021888->26025983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.055349:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2eee1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26021888->26025983] (req 26021888->26025983) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760758.080999:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973672 is committed 00002000:00010000:1.0:1789760758.081100:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26021888 -> 26025984 00010000:00010000:1.0:1789760758.081113:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2eee1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26021888->26025983] (req 26021888->26025983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.081123:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2eee1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26021888->26025983] (req 26021888->26025983) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.081133:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2eee1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26021888->26025983] (req 26021888->26025983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.081138:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760758.081141:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2eee1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26021888->26025983] (req 26021888->26025983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.081147:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2eee1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26021888->26025983] (req 26021888->26025983) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.081160:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2eee1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26021888->26025983] (req 26021888->26025983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.081164:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2eee1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26021888->26025983] (req 26021888->26025983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760758.081250:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac403ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512674944:12345-192.168.203.56@tcp:4:dd.0 Request processed in 25997us (26241us total) trans 4294973672 rc 0/0 00000100:00100000:1.0:1789760758.081260:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6383 00000100:00100000:1.0:1789760758.081269:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 737579795062 time 737619536822 state 0x2 ref 0 00000100:00100000:1.0:1789760758.084971:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760758.085093:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512675072 00000100:00100000:0.0:1789760758.085102:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512675072 00000100:00100000:0.0:1789760758.085149:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 737579795062 time 737619536822 state 0x0 ref 1 00000100:00100000:0.0:1789760758.085164:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 737619797238 time 737619536822 state 0x1 ref 1 00000100:00100000:0.0:1789760758.085194:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 737619797238 time 737649493433 state 0x0 ref 1 00000100:00100000:0.0:1789760758.085205:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac403a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512675072:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760758.085256:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26025984, mtime: 0x6aad94f5, atime: 0x0, ctime: 0x6aad94f5, blocks: 0xc807 00010000:00010000:0.0:1789760758.085265:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2eee8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.085282:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=737s v=(0 0) 00010000:00010000:0.0:1789760758.085286:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760758.085292:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2eee8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26025984->26030079] (req 26025984->26030079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.085305:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2eee8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26025984->26030079] (req 26025984->26030079) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760758.107368:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973673 is committed 00002000:00010000:0.0:1789760758.107599:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26025984 -> 26030080 00010000:00010000:0.0:1789760758.107614:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2eee8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26025984->26030079] (req 26025984->26030079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.107626:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2eee8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26025984->26030079] (req 26025984->26030079) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.107635:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2eee8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26025984->26030079] (req 26025984->26030079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.107642:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760758.107647:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2eee8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26025984->26030079] (req 26025984->26030079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.107656:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2eee8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26025984->26030079] (req 26025984->26030079) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.107679:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2eee8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26025984->26030079] (req 26025984->26030079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.107686:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2eee8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26025984->26030079] (req 26025984->26030079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760758.107795:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac403a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512675072:12345-192.168.203.56@tcp:4:dd.0 Request processed in 22586us (22827us total) trans 4294973673 rc 0/0 00000100:00100000:0.0:1789760758.107804:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6384 00000100:00100000:0.0:1789760758.107813:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 737619797238 time 737649493433 state 0x2 ref 0 00000100:00100000:2.0:1789760758.110452:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760758.110538:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512675200 00000100:00100000:0.0:1789760758.110545:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512675200 00000100:00100000:0.0:1789760758.110587:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 737619797238 time 737649493433 state 0x0 ref 1 00000100:00100000:0.0:1789760758.110598:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 737649753849 time 737649493433 state 0x1 ref 1 00000100:00100000:0.0:1789760758.110660:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 737649753849 time 737674959474 state 0x0 ref 1 00000100:00100000:0.0:1789760758.110677:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4133480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512675200:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760758.110748:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26030080, mtime: 0x6aad94f5, atime: 0x0, ctime: 0x6aad94f5, blocks: 0xc807 00010000:00010000:0.0:1789760758.110756:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2eeef lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.110771:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=737s v=(0 0) 00010000:00010000:0.0:1789760758.110775:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760758.110782:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2eeef lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26030080->26034175] (req 26030080->26034175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.110795:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2eeef lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26030080->26034175] (req 26030080->26034175) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.133800:0:5835:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:3.0:1789760758.134796:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973674 is committed 00002000:00010000:1.0:1789760758.134908:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26030080 -> 26034176 00010000:00010000:1.0:1789760758.134924:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2eeef lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26030080->26034175] (req 26030080->26034175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.134941:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2eeef lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26030080->26034175] (req 26030080->26034175) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.134950:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2eeef lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26030080->26034175] (req 26030080->26034175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.134956:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760758.134961:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2eeef lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26030080->26034175] (req 26030080->26034175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.134969:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2eeef lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26030080->26034175] (req 26030080->26034175) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.134988:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2eeef lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26030080->26034175] (req 26030080->26034175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.134995:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2eeef lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26030080->26034175] (req 26030080->26034175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760758.135080:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4133480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512675200:12345-192.168.203.56@tcp:4:dd.0 Request processed in 24400us (24632us total) trans 4294973674 rc 0/0 00000100:00100000:1.0:1789760758.135089:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6385 00000100:00100000:1.0:1789760758.135098:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 737649753849 time 737674959474 state 0x2 ref 0 00000100:00100000:2.0:1789760758.139064:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760758.139144:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512675328 00000100:00100000:1.0:1789760758.139155:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512675328 00000100:00100000:1.0:1789760758.139220:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 737649753849 time 737674959474 state 0x0 ref 1 00000100:00100000:1.0:1789760758.139235:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 737675219890 time 737674959474 state 0x1 ref 1 00000100:00100000:1.0:1789760758.139265:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 737675219890 time 737703564461 state 0x0 ref 1 00000100:00100000:1.0:1789760758.139285:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4130700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512675328:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760758.139338:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26034176, mtime: 0x6aad94f5, atime: 0x0, ctime: 0x6aad94f5, blocks: 0xc807 00010000:00010000:1.0:1789760758.139347:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2eef6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.139362:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=737s v=(0 0) 00010000:00010000:1.0:1789760758.139367:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760758.139373:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2eef6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26034176->26038271] (req 26034176->26038271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.139387:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2eef6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26034176->26038271] (req 26034176->26038271) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760758.173085:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973675 is committed 00002000:00010000:1.0:1789760758.173949:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26034176 -> 26038272 00010000:00010000:1.0:1789760758.173966:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2eef6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26034176->26038271] (req 26034176->26038271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.173979:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2eef6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26034176->26038271] (req 26034176->26038271) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.173994:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2eef6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26034176->26038271] (req 26034176->26038271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.174002:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760758.174007:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2eef6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26034176->26038271] (req 26034176->26038271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.174017:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2eef6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26034176->26038271] (req 26034176->26038271) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.174036:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2eef6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26034176->26038271] (req 26034176->26038271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.174044:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2eef6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26034176->26038271] (req 26034176->26038271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760758.174143:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4130700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512675328:12345-192.168.203.56@tcp:4:dd.0 Request processed in 34864us (35084us total) trans 4294973675 rc 0/0 00000100:00100000:1.0:1789760758.174153:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6386 00000100:00100000:1.0:1789760758.174162:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 737675219890 time 737703564461 state 0x2 ref 0 00000100:00100000:3.0:1789760758.179667:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760758.179746:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512675456 00000100:00100000:0.0:1789760758.179756:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512675456 00000100:00100000:0.0:1789760758.180190:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 737675219890 time 737703564461 state 0x0 ref 1 00000100:00100000:0.0:1789760758.180206:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 737703824877 time 737703564461 state 0x1 ref 1 00000100:00100000:0.0:1789760758.180251:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 737703824877 time 737744551115 state 0x0 ref 1 00000100:00100000:0.0:1789760758.180263:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febc7b33480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512675456:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760758.180309:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26038272, mtime: 0x6aad94f5, atime: 0x0, ctime: 0x6aad94f5, blocks: 0xc807 00010000:00010000:0.0:1789760758.180317:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2eefd lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.180333:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=737s v=(0 0) 00010000:00010000:0.0:1789760758.180338:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760758.180345:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2eefd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26038272->26042367] (req 26038272->26042367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.180360:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2eefd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26038272->26042367] (req 26038272->26042367) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760758.209344:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973676 is committed 00002000:00010000:3.0:1789760758.209518:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26038272 -> 26042368 00010000:00010000:3.0:1789760758.209534:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2eefd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26038272->26042367] (req 26038272->26042367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.209547:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2eefd lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26038272->26042367] (req 26038272->26042367) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.209556:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2eefd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26038272->26042367] (req 26038272->26042367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.209563:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760758.209568:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2eefd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26038272->26042367] (req 26038272->26042367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.209577:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2eefd lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26038272->26042367] (req 26038272->26042367) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.209596:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2eefd lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26038272->26042367] (req 26038272->26042367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.209868:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2eefd lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26038272->26042367] (req 26038272->26042367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760758.209973:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febc7b33480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512675456:12345-192.168.203.56@tcp:4:dd.0 Request processed in 29707us (30311us total) trans 4294973676 rc 0/0 00000100:00100000:3.0:1789760758.209985:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6387 00000100:00100000:3.0:1789760758.209996:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 737703824877 time 737744551115 state 0x2 ref 0 00000100:00100000:0.0:1789760758.213997:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760758.217167:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512675584 00000100:00100000:1.0:1789760758.217180:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512675584 00000100:00100000:1.0:1789760758.217461:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 737703824877 time 737744551115 state 0x0 ref 1 00000100:00100000:1.0:1789760758.217479:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 737744811531 time 737744551115 state 0x1 ref 1 00000100:00100000:1.0:1789760758.217539:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 737744811531 time 737781838206 state 0x0 ref 1 00000100:00100000:1.0:1789760758.217551:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac531e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512675584:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760758.217602:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26042368, mtime: 0x6aad94f5, atime: 0x0, ctime: 0x6aad94f5, blocks: 0xc807 00010000:00010000:1.0:1789760758.217611:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2ef04 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.217627:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=737s v=(0 0) 00010000:00010000:1.0:1789760758.217632:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760758.217638:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2ef04 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26042368->26046463] (req 26042368->26046463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.217653:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2ef04 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26042368->26046463] (req 26042368->26046463) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760758.264266:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973677 is committed 00002000:00010000:3.0:1789760758.264323:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26042368 -> 26046464 00010000:00010000:3.0:1789760758.264343:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2ef04 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26042368->26046463] (req 26042368->26046463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.264355:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2ef04 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26042368->26046463] (req 26042368->26046463) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.264365:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2ef04 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26042368->26046463] (req 26042368->26046463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.264372:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760758.264377:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2ef04 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26042368->26046463] (req 26042368->26046463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.264385:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2ef04 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26042368->26046463] (req 26042368->26046463) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.264404:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2ef04 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26042368->26046463] (req 26042368->26046463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.264412:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2ef04 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26042368->26046463] (req 26042368->26046463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760758.264509:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac531e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512675584:12345-192.168.203.56@tcp:4:dd.0 Request processed in 46955us (50517us total) trans 4294973677 rc 0/0 00000100:00100000:3.0:1789760758.264522:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6388 00000100:00100000:3.0:1789760758.264531:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 737744811531 time 737781838206 state 0x2 ref 0 00000100:00100000:2.0:1789760758.267341:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760758.267449:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512675712 00000100:00100000:1.0:1789760758.267459:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512675712 00000100:00100000:1.0:1789760758.267513:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 737744811531 time 737781838206 state 0x0 ref 1 00000100:00100000:1.0:1789760758.267528:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 737782098622 time 737781838206 state 0x1 ref 1 00000100:00100000:1.0:1789760758.267592:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 737782098622 time 737831891796 state 0x0 ref 1 00000100:00100000:1.0:1789760758.267605:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4131180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512675712:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760758.267662:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26046464, mtime: 0x6aad94f5, atime: 0x0, ctime: 0x6aad94f5, blocks: 0xc807 00010000:00010000:1.0:1789760758.267672:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2ef0b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.267686:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=737s v=(0 0) 00010000:00010000:1.0:1789760758.267691:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760758.267737:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2ef0b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26046464->26050559] (req 26046464->26050559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.267752:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2ef0b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26046464->26050559] (req 26046464->26050559) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760758.301480:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973678 is committed 00002000:00010000:3.0:1789760758.301587:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26046464 -> 26050560 00010000:00010000:3.0:1789760758.301612:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2ef0b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26046464->26050559] (req 26046464->26050559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.301625:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2ef0b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26046464->26050559] (req 26046464->26050559) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.301634:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2ef0b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26046464->26050559] (req 26046464->26050559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.301641:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760758.301646:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2ef0b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26046464->26050559] (req 26046464->26050559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.301654:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2ef0b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26046464->26050559] (req 26046464->26050559) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.301673:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2ef0b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26046464->26050559] (req 26046464->26050559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.301680:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2ef0b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26046464->26050559] (req 26046464->26050559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760758.301809:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4131180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512675712:12345-192.168.203.56@tcp:4:dd.0 Request processed in 34201us (34472us total) trans 4294973678 rc 0/0 00000100:00100000:3.0:1789760758.301818:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6389 00000100:00100000:3.0:1789760758.301827:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 737782098622 time 737831891796 state 0x2 ref 0 00000100:00100000:1.0:1789760758.304953:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760758.305111:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512675840 00000100:00100000:3.0:1789760758.305122:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512675840 00000100:00100000:3.0:1789760758.305178:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 737782098622 time 737831891796 state 0x0 ref 1 00000100:00100000:3.0:1789760758.305193:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 737832152212 time 737831891796 state 0x1 ref 1 00000100:00100000:3.0:1789760758.305243:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 737832152212 time 737869542139 state 0x0 ref 1 00000100:00100000:3.0:1789760758.305256:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4039c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512675840:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760758.305305:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26050560, mtime: 0x6aad94f5, atime: 0x0, ctime: 0x6aad94f5, blocks: 0xc807 00010000:00010000:3.0:1789760758.305314:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2ef12 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.305330:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=737s v=(0 0) 00010000:00010000:3.0:1789760758.305334:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760758.305340:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2ef12 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26050560->26054655] (req 26050560->26054655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.305355:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2ef12 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26050560->26054655] (req 26050560->26054655) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760758.341012:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26050560 -> 26054656 00010000:00010000:3.0:1789760758.341033:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2ef12 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26050560->26054655] (req 26050560->26054655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.341047:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2ef12 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26050560->26054655] (req 26050560->26054655) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.341056:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2ef12 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26050560->26054655] (req 26050560->26054655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.341063:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760758.341068:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2ef12 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26050560->26054655] (req 26050560->26054655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.341078:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2ef12 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26050560->26054655] (req 26050560->26054655) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.341096:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2ef12 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26050560->26054655] (req 26050560->26054655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.341103:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2ef12 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26050560->26054655] (req 26050560->26054655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760758.341202:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4039c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2359:x1876699512675840:12345-192.168.203.56@tcp:4:dd.0 Request processed in 35943us (36253us total) trans 4294973679 rc 0/0 00000100:00100000:3.0:1789760758.341226:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6390 00000100:00100000:3.0:1789760758.341236:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 737832152212 time 737869542139 state 0x2 ref 0 00000001:00080000:2.0:1789760758.342619:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973679 is committed 00000100:00100000:2.0:1789760758.344355:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760758.344433:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512675968 00000100:00100000:3.0:1789760758.344442:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512675968 00000100:00100000:3.0:1789760758.344495:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 737832152212 time 737869542139 state 0x0 ref 1 00000100:00100000:3.0:1789760758.344510:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 737869802555 time 737869542139 state 0x1 ref 1 00000100:00100000:3.0:1789760758.344542:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 737869802555 time 737908841771 state 0x0 ref 1 00000100:00100000:3.0:1789760758.344554:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4130a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512675968:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760758.344607:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26054656, mtime: 0x6aad94f5, atime: 0x0, ctime: 0x6aad94f5, blocks: 0xc807 00010000:00010000:3.0:1789760758.344617:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2ef19 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.344632:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=737s v=(0 0) 00010000:00010000:3.0:1789760758.344637:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760758.344644:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2ef19 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26054656->26058751] (req 26054656->26058751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.344658:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2ef19 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26054656->26058751] (req 26054656->26058751) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760758.378383:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973680 is committed 00002000:00010000:3.0:1789760758.378496:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26054656 -> 26058752 00010000:00010000:3.0:1789760758.378512:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2ef19 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26054656->26058751] (req 26054656->26058751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.378525:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2ef19 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26054656->26058751] (req 26054656->26058751) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.378533:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2ef19 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26054656->26058751] (req 26054656->26058751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.378540:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760758.378545:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2ef19 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26054656->26058751] (req 26054656->26058751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.378553:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2ef19 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26054656->26058751] (req 26054656->26058751) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.378570:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2ef19 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26054656->26058751] (req 26054656->26058751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.378576:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2ef19 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26054656->26058751] (req 26054656->26058751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760758.378979:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4130a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512675968:12345-192.168.203.56@tcp:4:dd.0 Request processed in 34422us (34625us total) trans 4294973680 rc 0/0 00000100:00100000:3.0:1789760758.378990:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6391 00000100:00100000:3.0:1789760758.378999:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 737869802555 time 737908841771 state 0x2 ref 0 00000100:00100000:1.0:1789760758.382572:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760758.383027:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512676096 00000100:00100000:3.0:1789760758.383038:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512676096 00000100:00100000:3.0:1789760758.383091:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 737869802555 time 737908841771 state 0x0 ref 1 00000100:00100000:3.0:1789760758.383106:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 737909102187 time 737908841771 state 0x1 ref 1 00000100:00100000:3.0:1789760758.383137:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 737909102187 time 737947436797 state 0x0 ref 1 00000100:00100000:3.0:1789760758.383151:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4038e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512676096:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760758.383196:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26058752, mtime: 0x6aad94f5, atime: 0x0, ctime: 0x6aad94f5, blocks: 0xc807 00010000:00010000:3.0:1789760758.383205:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2ef20 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.383233:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=737s v=(0 0) 00010000:00010000:3.0:1789760758.383238:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760758.383245:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2ef20 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26058752->26062847] (req 26058752->26062847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.383258:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2ef20 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26058752->26062847] (req 26058752->26062847) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760758.414543:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973681 is committed 00002000:00010000:2.0:1789760758.414994:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26058752 -> 26062848 00010000:00010000:2.0:1789760758.415009:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2ef20 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26058752->26062847] (req 26058752->26062847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.415023:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2ef20 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26058752->26062847] (req 26058752->26062847) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.415032:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2ef20 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26058752->26062847] (req 26058752->26062847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.415040:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760758.415045:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2ef20 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26058752->26062847] (req 26058752->26062847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.415055:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2ef20 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26058752->26062847] (req 26058752->26062847) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.415075:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2ef20 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26058752->26062847] (req 26058752->26062847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.415083:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2ef20 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26058752->26062847] (req 26058752->26062847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760758.415172:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4038e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512676096:12345-192.168.203.56@tcp:4:dd.0 Request processed in 32020us (32605us total) trans 4294973681 rc 0/0 00000100:00100000:2.0:1789760758.415182:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6392 00000100:00100000:2.0:1789760758.415192:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 737909102187 time 737947436797 state 0x2 ref 0 00000100:00100000:1.0:1789760758.418090:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760758.418180:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512676224 00000100:00100000:3.0:1789760758.418190:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512676224 00000100:00100000:3.0:1789760758.418251:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 737909102187 time 737947436797 state 0x0 ref 1 00000100:00100000:3.0:1789760758.418266:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 737947697213 time 737947436797 state 0x1 ref 1 00000100:00100000:3.0:1789760758.418323:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 737947697213 time 737982622682 state 0x0 ref 1 00000100:00100000:3.0:1789760758.418340:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac403b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512676224:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760758.418385:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26062848, mtime: 0x6aad94f5, atime: 0x0, ctime: 0x6aad94f5, blocks: 0xc807 00010000:00010000:3.0:1789760758.418393:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2ef27 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.418408:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=737s v=(0 0) 00010000:00010000:3.0:1789760758.418412:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760758.418418:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2ef27 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26062848->26066943] (req 26062848->26066943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.418432:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2ef27 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26062848->26066943] (req 26062848->26066943) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760758.445694:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973682 is committed 00002000:00010000:0.0:1789760758.445726:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26062848 -> 26066944 00010000:00010000:0.0:1789760758.445741:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2ef27 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26062848->26066943] (req 26062848->26066943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.445754:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2ef27 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26062848->26066943] (req 26062848->26066943) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.445842:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2ef27 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26062848->26066943] (req 26062848->26066943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.445851:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760758.445857:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2ef27 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26062848->26066943] (req 26062848->26066943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.445866:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2ef27 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26062848->26066943] (req 26062848->26066943) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.445945:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2ef27 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26062848->26066943] (req 26062848->26066943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.445955:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2ef27 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26062848->26066943] (req 26062848->26066943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760758.446049:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac403b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512676224:12345-192.168.203.56@tcp:4:dd.0 Request processed in 27707us (27962us total) trans 4294973682 rc 0/0 00000100:00100000:0.0:1789760758.446060:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6393 00000100:00100000:0.0:1789760758.446069:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 737947697213 time 737982622682 state 0x2 ref 0 00000100:00100000:0.0:1789760758.449642:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760758.449681:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512676352 00000100:00100000:3.0:1789760758.449692:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512676352 00000100:00100000:3.0:1789760758.449785:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 737947697213 time 737982622682 state 0x0 ref 1 00000100:00100000:3.0:1789760758.449800:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 737982883098 time 737982622682 state 0x1 ref 1 00000100:00100000:3.0:1789760758.449826:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 737982883098 time 738014125755 state 0x0 ref 1 00000100:00100000:3.0:1789760758.449839:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac531ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512676352:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760758.449885:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26066944, mtime: 0x6aad94f5, atime: 0x0, ctime: 0x6aad94f5, blocks: 0xc807 00010000:00010000:3.0:1789760758.449894:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2ef2e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.449916:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=738s v=(0 0) 00010000:00010000:3.0:1789760758.449920:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760758.449927:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2ef2e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26066944->26071039] (req 26066944->26071039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.449940:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2ef2e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26066944->26071039] (req 26066944->26071039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760758.474580:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973683 is committed 00002000:00010000:1.0:1789760758.474673:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26066944 -> 26071040 00010000:00010000:1.0:1789760758.474686:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2ef2e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26066944->26071039] (req 26066944->26071039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.474818:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2ef2e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26066944->26071039] (req 26066944->26071039) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.474829:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2ef2e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26066944->26071039] (req 26066944->26071039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.474838:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760758.474843:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2ef2e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26066944->26071039] (req 26066944->26071039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.474853:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2ef2e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26066944->26071039] (req 26066944->26071039) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.474871:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2ef2e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26066944->26071039] (req 26066944->26071039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.474932:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2ef2e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26066944->26071039] (req 26066944->26071039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760758.475007:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac531ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512676352:12345-192.168.203.56@tcp:4:dd.0 Request processed in 25166us (25370us total) trans 4294973683 rc 0/0 00000100:00100000:1.0:1789760758.475017:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6394 00000100:00100000:1.0:1789760758.475025:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 737982883098 time 738014125755 state 0x2 ref 0 00000100:00100000:1.0:1789760758.478360:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760758.478837:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512676480 00000100:00100000:2.0:1789760758.478847:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512676480 00000100:00100000:2.0:1789760758.478898:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 737982883098 time 738014125755 state 0x0 ref 1 00000100:00100000:2.0:1789760758.478913:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 738014386171 time 738014125755 state 0x1 ref 1 00000100:00100000:2.0:1789760758.479024:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 738014386171 time 738043322342 state 0x0 ref 1 00000100:00100000:2.0:1789760758.479038:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4039f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512676480:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760758.479085:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26071040, mtime: 0x6aad94f5, atime: 0x0, ctime: 0x6aad94f5, blocks: 0xc807 00010000:00010000:2.0:1789760758.479095:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2ef35 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.479125:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=738s v=(0 0) 00010000:00010000:2.0:1789760758.479130:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760758.479136:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2ef35 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26071040->26075135] (req 26071040->26075135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.479150:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2ef35 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26071040->26075135] (req 26071040->26075135) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760758.502378:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973684 is committed 00002000:00010000:2.0:1789760758.502422:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26071040 -> 26075136 00010000:00010000:2.0:1789760758.502437:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2ef35 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26071040->26075135] (req 26071040->26075135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.502450:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2ef35 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26071040->26075135] (req 26071040->26075135) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.502459:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2ef35 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26071040->26075135] (req 26071040->26075135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.502466:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760758.502470:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2ef35 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26071040->26075135] (req 26071040->26075135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.502479:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2ef35 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26071040->26075135] (req 26071040->26075135) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.502645:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2ef35 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26071040->26075135] (req 26071040->26075135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.502651:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2ef35 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26071040->26075135] (req 26071040->26075135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760758.502759:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4039f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512676480:12345-192.168.203.56@tcp:4:dd.0 Request processed in 23719us (24407us total) trans 4294973684 rc 0/0 00000100:00100000:2.0:1789760758.502779:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6395 00000100:00100000:2.0:1789760758.502789:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 738014386171 time 738043322342 state 0x2 ref 0 00000100:00100000:3.0:1789760758.509856:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760758.510135:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512676608 00000100:00100000:0.0:1789760758.510145:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512676608 00000100:00100000:0.0:1789760758.510197:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 738014386171 time 738043322342 state 0x0 ref 1 00000100:00100000:0.0:1789760758.510225:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 738043582758 time 738043322342 state 0x1 ref 1 00000100:00100000:0.0:1789760758.510253:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 738043582758 time 738074552884 state 0x0 ref 1 00000100:00100000:0.0:1789760758.510266:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febc7b32680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512676608:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760758.510311:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26075136, mtime: 0x6aad94f5, atime: 0x0, ctime: 0x6aad94f5, blocks: 0xc807 00010000:00010000:0.0:1789760758.510321:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2ef3c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.510336:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=738s v=(0 0) 00010000:00010000:0.0:1789760758.510341:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760758.510347:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2ef3c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26075136->26079231] (req 26075136->26079231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.510362:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2ef3c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26075136->26079231] (req 26075136->26079231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760758.552629:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26075136 -> 26079232 00002000:00010000:1.0:1789760758.552638:0:6709:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760757 -> 1789760758 00002000:00010000:1.0:1789760758.552641:0:6709:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760757 -> 1789760758 00010000:00010000:1.0:1789760758.552654:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2ef3c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26075136->26079231] (req 26075136->26079231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.552666:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2ef3c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26075136->26079231] (req 26075136->26079231) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.552679:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2ef3c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26075136->26079231] (req 26075136->26079231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.552686:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760758.552690:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2ef3c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26075136->26079231] (req 26075136->26079231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.552726:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2ef3c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26075136->26079231] (req 26075136->26079231) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.552745:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2ef3c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26075136->26079231] (req 26075136->26079231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.552754:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2ef3c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26075136->26079231] (req 26075136->26079231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760758.552858:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febc7b32680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2359:x1876699512676608:12345-192.168.203.56@tcp:4:dd.0 Request processed in 42590us (43006us total) trans 4294973685 rc 0/0 00000100:00100000:1.0:1789760758.552869:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6396 00000100:00100000:1.0:1789760758.553040:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 738043582758 time 738074552884 state 0x2 ref 0 00000001:00080000:3.0:1789760758.553518:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973685 is committed 00000100:00100000:0.0:1789760758.556571:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760758.558016:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512676736 00000100:00100000:1.0:1789760758.558028:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512676736 00000100:00100000:1.0:1789760758.558083:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 738043582758 time 738074552884 state 0x0 ref 1 00000100:00100000:1.0:1789760758.558098:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 738074813300 time 738074552884 state 0x1 ref 1 00000100:00100000:1.0:1789760758.558131:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 738074813300 time 738122430749 state 0x0 ref 1 00000100:00100000:1.0:1789760758.558144:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac531ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512676736:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760758.558186:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26079232, mtime: 0x6aad94f6, atime: 0x0, ctime: 0x6aad94f6, blocks: 0xc807 00010000:00010000:1.0:1789760758.558194:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2ef43 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.558219:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=738s v=(0 0) 00010000:00010000:1.0:1789760758.558223:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760758.558230:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2ef43 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26079232->26083327] (req 26079232->26083327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.558244:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2ef43 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26079232->26083327] (req 26079232->26083327) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760758.605955:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973686 is committed 00002000:00010000:3.0:1789760758.605982:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26079232 -> 26083328 00010000:00010000:3.0:1789760758.605998:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2ef43 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26079232->26083327] (req 26079232->26083327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.606013:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2ef43 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26079232->26083327] (req 26079232->26083327) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.606022:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2ef43 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26079232->26083327] (req 26079232->26083327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.606029:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760758.606034:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2ef43 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26079232->26083327] (req 26079232->26083327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.606163:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2ef43 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26079232->26083327] (req 26079232->26083327) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.606300:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2ef43 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26079232->26083327] (req 26079232->26083327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.606310:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2ef43 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26079232->26083327] (req 26079232->26083327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760758.606403:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac531ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512676736:12345-192.168.203.56@tcp:4:dd.0 Request processed in 48257us (49837us total) trans 4294973686 rc 0/0 00000100:00100000:3.0:1789760758.606413:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6397 00000100:00100000:3.0:1789760758.606423:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 738074813300 time 738122430749 state 0x2 ref 0 00000100:00100000:0.0:1789760758.612900:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760758.613108:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512676864 00000100:00100000:2.0:1789760758.613118:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512676864 00000100:00100000:2.0:1789760758.613172:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 738074813300 time 738122430749 state 0x0 ref 1 00000100:00100000:2.0:1789760758.613188:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 738122691165 time 738122430749 state 0x1 ref 1 00000100:00100000:2.0:1789760758.613230:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 738122691165 time 738177529257 state 0x0 ref 1 00000100:00100000:2.0:1789760758.613244:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac531e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512676864:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760758.613598:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26083328, mtime: 0x6aad94f6, atime: 0x0, ctime: 0x6aad94f6, blocks: 0xc807 00010000:00010000:2.0:1789760758.613610:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2ef4a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.613626:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=738s v=(0 0) 00010000:00010000:2.0:1789760758.613631:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760758.613637:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2ef4a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26083328->26087423] (req 26083328->26087423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.613660:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2ef4a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26083328->26087423] (req 26083328->26087423) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760758.660481:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973687 is committed 00002000:00010000:3.0:1789760758.660619:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26083328 -> 26087424 00010000:00010000:3.0:1789760758.660633:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2ef4a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26083328->26087423] (req 26083328->26087423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.660783:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2ef4a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26083328->26087423] (req 26083328->26087423) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.660802:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2ef4a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26083328->26087423] (req 26083328->26087423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.660809:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760758.660814:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2ef4a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26083328->26087423] (req 26083328->26087423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.660827:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2ef4a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26083328->26087423] (req 26083328->26087423) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.660847:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2ef4a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26083328->26087423] (req 26083328->26087423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.660854:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2ef4a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26083328->26087423] (req 26083328->26087423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760758.660936:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac531e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512676864:12345-192.168.203.56@tcp:4:dd.0 Request processed in 47690us (48041us total) trans 4294973687 rc 0/0 00000100:00100000:3.0:1789760758.660946:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6398 00000100:00100000:3.0:1789760758.660955:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 738122691165 time 738177529257 state 0x2 ref 0 00000100:00100000:0.0:1789760758.670417:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760758.670511:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512676992 00000100:00100000:1.0:1789760758.670521:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512676992 00000100:00100000:1.0:1789760758.670583:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 738122691165 time 738177529257 state 0x0 ref 1 00000100:00100000:1.0:1789760758.670600:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 738177789673 time 738177529257 state 0x1 ref 1 00000100:00100000:1.0:1789760758.670639:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 738177789673 time 738234938344 state 0x0 ref 1 00000100:00100000:1.0:1789760758.670654:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac531dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512676992:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760758.670758:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26087424, mtime: 0x6aad94f6, atime: 0x0, ctime: 0x6aad94f6, blocks: 0xc807 00010000:00010000:1.0:1789760758.670774:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2ef51 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.670789:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=738s v=(0 0) 00010000:00010000:1.0:1789760758.670794:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760758.670800:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2ef51 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26087424->26091519] (req 26087424->26091519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.670816:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2ef51 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26087424->26091519] (req 26087424->26091519) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760758.716067:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26087424 -> 26091520 00000001:00080000:3.0:1789760758.716084:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973688 is committed 00010000:00010000:2.0:1789760758.716088:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2ef51 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26087424->26091519] (req 26087424->26091519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.716100:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2ef51 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26087424->26091519] (req 26087424->26091519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.716109:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2ef51 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26087424->26091519] (req 26087424->26091519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.716117:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760758.716122:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2ef51 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26087424->26091519] (req 26087424->26091519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.716131:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2ef51 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26087424->26091519] (req 26087424->26091519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.716151:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2ef51 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26087424->26091519] (req 26087424->26091519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.716159:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2ef51 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26087424->26091519] (req 26087424->26091519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760758.716262:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac531dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512676992:12345-192.168.203.56@tcp:4:dd.0 Request processed in 45606us (45851us total) trans 4294973688 rc 0/0 00000100:00100000:2.0:1789760758.716271:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6399 00000100:00100000:2.0:1789760758.716280:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 738177789673 time 738234938344 state 0x2 ref 0 00000100:00100000:0.0:1789760758.726620:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760758.728335:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512677120 00000100:00100000:3.0:1789760758.728349:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512677120 00000100:00100000:3.0:1789760758.728410:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 738177789673 time 738234938344 state 0x0 ref 1 00000100:00100000:3.0:1789760758.728425:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 738235198760 time 738234938344 state 0x1 ref 1 00000100:00100000:3.0:1789760758.728509:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 738235198760 time 738292807396 state 0x0 ref 1 00000100:00100000:3.0:1789760758.728525:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac531ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512677120:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760758.728574:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26091520, mtime: 0x6aad94f6, atime: 0x0, ctime: 0x6aad94f6, blocks: 0xc807 00010000:00010000:3.0:1789760758.728585:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2ef58 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.728601:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=738s v=(0 0) 00010000:00010000:3.0:1789760758.728606:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760758.728613:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2ef58 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26091520->26095615] (req 26091520->26095615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.728629:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2ef58 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26091520->26095615] (req 26091520->26095615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760758.773545:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973689 is committed 00002000:00010000:3.0:1789760758.773739:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26091520 -> 26095616 00010000:00010000:3.0:1789760758.773755:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2ef58 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26091520->26095615] (req 26091520->26095615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.773767:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2ef58 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26091520->26095615] (req 26091520->26095615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.773776:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2ef58 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26091520->26095615] (req 26091520->26095615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.773783:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760758.773788:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2ef58 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26091520->26095615] (req 26091520->26095615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.773796:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2ef58 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26091520->26095615] (req 26091520->26095615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.773821:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2ef58 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26091520->26095615] (req 26091520->26095615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.773829:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2ef58 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26091520->26095615] (req 26091520->26095615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760758.774071:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac531ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512677120:12345-192.168.203.56@tcp:4:dd.0 Request processed in 45545us (47454us total) trans 4294973689 rc 0/0 00000100:00100000:3.0:1789760758.774082:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6400 00000100:00100000:3.0:1789760758.774092:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 738235198760 time 738292807396 state 0x2 ref 0 00000100:00100000:0.0:1789760758.778337:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760758.780074:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512677248 00000100:00100000:3.0:1789760758.780088:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512677248 00000100:00100000:2.0:1789760758.782896:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 738235198760 time 738292807396 state 0x0 ref 1 00000100:00100000:2.0:1789760758.782925:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 738293067812 time 738292807396 state 0x1 ref 1 00000100:00100000:2.0:1789760758.782961:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 738293067812 time 738347261038 state 0x0 ref 1 00000100:00100000:2.0:1789760758.782975:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac531f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512677248:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760758.783022:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26095616, mtime: 0x6aad94f6, atime: 0x0, ctime: 0x6aad94f6, blocks: 0xc807 00010000:00010000:2.0:1789760758.783031:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2ef5f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.783046:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=738s v=(0 0) 00010000:00010000:2.0:1789760758.783050:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760758.783056:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2ef5f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26095616->26099711] (req 26095616->26099711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.783070:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2ef5f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26095616->26099711] (req 26095616->26099711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760758.867424:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973690 is committed 00002000:00010000:1.0:1789760758.869366:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26095616 -> 26099712 00010000:00010000:1.0:1789760758.869384:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2ef5f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26095616->26099711] (req 26095616->26099711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.869397:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2ef5f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26095616->26099711] (req 26095616->26099711) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.869407:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2ef5f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26095616->26099711] (req 26095616->26099711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.869414:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760758.869419:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2ef5f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26095616->26099711] (req 26095616->26099711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.869429:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2ef5f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26095616->26099711] (req 26095616->26099711) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.869452:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2ef5f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26095616->26099711] (req 26095616->26099711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.869460:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2ef5f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26095616->26099711] (req 26095616->26099711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760758.869564:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac531f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512677248:12345-192.168.203.56@tcp:4:dd.0 Request processed in 86588us (91233us total) trans 4294973690 rc 0/0 00000100:00100000:1.0:1789760758.869573:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6401 00000100:00100000:1.0:1789760758.869583:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 738293067812 time 738347261038 state 0x2 ref 0 00000100:00100000:1.0:1789760758.872081:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760758.872262:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512677376 00000100:00100000:2.0:1789760758.872271:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512677376 00000100:00100000:2.0:1789760758.872319:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 738293067812 time 738347261038 state 0x0 ref 1 00000100:00100000:2.0:1789760758.872333:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 738347521454 time 738347261038 state 0x1 ref 1 00000100:00100000:2.0:1789760758.872361:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 738347521454 time 738436660743 state 0x0 ref 1 00000100:00100000:2.0:1789760758.872372:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4038380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512677376:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760758.872415:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26099712, mtime: 0x6aad94f6, atime: 0x0, ctime: 0x6aad94f6, blocks: 0xc807 00010000:00010000:2.0:1789760758.872423:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2ef66 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.872437:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=738s v=(0 0) 00010000:00010000:2.0:1789760758.872442:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760758.872448:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2ef66 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26099712->26103807] (req 26099712->26103807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.872461:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2ef66 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26099712->26103807] (req 26099712->26103807) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760758.911516:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26099712 -> 26103808 00010000:00010000:2.0:1789760758.911537:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2ef66 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26099712->26103807] (req 26099712->26103807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.911557:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2ef66 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26099712->26103807] (req 26099712->26103807) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.911567:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2ef66 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26099712->26103807] (req 26099712->26103807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.911574:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760758.911579:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2ef66 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26099712->26103807] (req 26099712->26103807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.911589:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2ef66 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26099712->26103807] (req 26099712->26103807) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.911608:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2ef66 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26099712->26103807] (req 26099712->26103807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.911616:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2ef66 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26099712->26103807] (req 26099712->26103807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760758.911739:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4038380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2358:x1876699512677376:12345-192.168.203.56@tcp:4:dd.0 Request processed in 39363us (39660us total) trans 4294973691 rc 0/0 00000100:00100000:2.0:1789760758.911749:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6402 00000100:00100000:2.0:1789760758.911758:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 738347521454 time 738436660743 state 0x2 ref 0 00000001:00080000:0.0:1789760758.912135:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973691 is committed 00000100:00100000:0.0:1789760758.915388:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760758.918808:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512677504 00000100:00100000:3.0:1789760758.918822:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512677504 00000100:00100000:3.0:1789760758.918891:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 738347521454 time 738436660743 state 0x0 ref 1 00000100:00100000:3.0:1789760758.918910:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 738436921159 time 738436660743 state 0x1 ref 1 00000100:00100000:3.0:1789760758.918944:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 738436921159 time 738483244204 state 0x0 ref 1 00000100:00100000:3.0:1789760758.918958:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2fb8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512677504:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760758.919007:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26103808, mtime: 0x6aad94f6, atime: 0x0, ctime: 0x6aad94f6, blocks: 0xc807 00010000:00010000:3.0:1789760758.919017:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2ef6d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.919038:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=738s v=(0 0) 00010000:00010000:3.0:1789760758.919043:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760758.919049:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2ef6d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26103808->26107903] (req 26103808->26107903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.919063:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2ef6d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26103808->26107903] (req 26103808->26107903) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760758.989854:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973692 is committed 00002000:00010000:3.0:1789760758.990069:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26103808 -> 26107904 00010000:00010000:3.0:1789760758.990084:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2ef6d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26103808->26107903] (req 26103808->26107903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.990097:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2ef6d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26103808->26107903] (req 26103808->26107903) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.990106:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2ef6d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26103808->26107903] (req 26103808->26107903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.990114:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760758.990119:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2ef6d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26103808->26107903] (req 26103808->26107903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.990127:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2ef6d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26103808->26107903] (req 26103808->26107903) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.990145:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2ef6d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26103808->26107903] (req 26103808->26107903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.990155:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2ef6d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26103808->26107903] (req 26103808->26107903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760758.990250:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2fb8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512677504:12345-192.168.203.56@tcp:4:dd.0 Request processed in 71290us (74876us total) trans 4294973692 rc 0/0 00000100:00100000:3.0:1789760758.990261:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6403 00000100:00100000:3.0:1789760758.990271:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 738436921159 time 738483244204 state 0x2 ref 0 00000100:00100000:1.0:1789760758.993769:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760758.994908:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512677632 00000100:00100000:2.0:1789760758.994920:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512677632 00000100:00100000:2.0:1789760758.994979:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 738436921159 time 738483244204 state 0x0 ref 1 00000100:00100000:2.0:1789760758.994995:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 738483504620 time 738483244204 state 0x1 ref 1 00000100:00100000:2.0:1789760758.995062:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 738483504620 time 738559360924 state 0x0 ref 1 00000100:00100000:2.0:1789760758.995077:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac403a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512677632:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760758.995128:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26107904, mtime: 0x6aad94f6, atime: 0x0, ctime: 0x6aad94f6, blocks: 0xc807 00010000:00010000:2.0:1789760758.995137:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2ef74 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.995153:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=738s v=(0 0) 00010000:00010000:2.0:1789760758.995157:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760758.995163:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2ef74 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26107904->26111999] (req 26107904->26111999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.995176:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2ef74 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26107904->26111999] (req 26107904->26111999) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760759.045149:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26107904 -> 26112000 00010000:00010000:3.0:1789760759.045171:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2ef74 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26107904->26111999] (req 26107904->26111999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.045184:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2ef74 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26107904->26111999] (req 26107904->26111999) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.045193:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2ef74 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26107904->26111999] (req 26107904->26111999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.045201:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760759.045216:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2ef74 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26107904->26111999] (req 26107904->26111999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.045226:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2ef74 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26107904->26111999] (req 26107904->26111999) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.045249:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2ef74 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26107904->26111999] (req 26107904->26111999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.045257:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2ef74 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26107904->26111999] (req 26107904->26111999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760759.045353:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac403a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2359:x1876699512677632:12345-192.168.203.56@tcp:4:dd.0 Request processed in 50275us (51588us total) trans 4294973693 rc 0/0 00000100:00100000:3.0:1789760759.045364:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6404 00000100:00100000:3.0:1789760759.045374:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 738483504620 time 738559360924 state 0x2 ref 0 00000001:00080000:0.0:1789760759.046130:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973693 is committed 00000100:00100000:3.0:1789760759.049036:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760759.049145:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512677760 00000100:00100000:2.0:1789760759.049156:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512677760 00000100:00100000:2.0:1789760759.049278:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 738483504620 time 738559360924 state 0x0 ref 1 00000100:00100000:2.0:1789760759.049293:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 738559621340 time 738559360924 state 0x1 ref 1 00000100:00100000:2.0:1789760759.049322:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 738559621340 time 738613622144 state 0x0 ref 1 00000100:00100000:2.0:1789760759.049335:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febea899180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512677760:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760759.049384:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26112000, mtime: 0x6aad94f6, atime: 0x0, ctime: 0x6aad94f6, blocks: 0xc807 00010000:00010000:2.0:1789760759.049393:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2ef7b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.049411:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=738s v=(0 0) 00010000:00010000:2.0:1789760759.049415:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760759.049421:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2ef7b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26112000->26116095] (req 26112000->26116095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.049434:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2ef7b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26112000->26116095] (req 26112000->26116095) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1789760759.050948:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:2.0:1789760759.091523:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973694 is committed 00002000:00010000:0.0:1789760759.091742:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26112000 -> 26116096 00010000:00010000:0.0:1789760759.091759:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2ef7b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26112000->26116095] (req 26112000->26116095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.091772:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2ef7b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26112000->26116095] (req 26112000->26116095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.091781:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2ef7b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26112000->26116095] (req 26112000->26116095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.091789:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760759.091793:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2ef7b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26112000->26116095] (req 26112000->26116095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.091807:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2ef7b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26112000->26116095] (req 26112000->26116095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.091827:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2ef7b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26112000->26116095] (req 26112000->26116095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.091834:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2ef7b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26112000->26116095] (req 26112000->26116095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760759.091936:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febea899180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512677760:12345-192.168.203.56@tcp:4:dd.0 Request processed in 42596us (42904us total) trans 4294973694 rc 0/0 00000100:00100000:0.0:1789760759.091947:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6405 00000100:00100000:0.0:1789760759.091956:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 738559621340 time 738613622144 state 0x2 ref 0 00000100:00100000:1.0:1789760759.098980:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760759.099114:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512677888 00000100:00100000:3.0:1789760759.099124:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512677888 00000100:00100000:3.0:1789760759.099177:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 738559621340 time 738613622144 state 0x0 ref 1 00000100:00100000:3.0:1789760759.099193:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 738613882560 time 738613622144 state 0x1 ref 1 00000100:00100000:3.0:1789760759.099228:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 738613882560 time 738663527638 state 0x0 ref 1 00000100:00100000:3.0:1789760759.099241:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac403bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512677888:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760759.099287:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26116096, mtime: 0x6aad94f6, atime: 0x0, ctime: 0x6aad94f6, blocks: 0xc807 00010000:00010000:3.0:1789760759.099297:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2ef82 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.099318:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=738s v=(0 0) 00010000:00010000:3.0:1789760759.099323:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760759.099329:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2ef82 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26116096->26120191] (req 26116096->26120191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.099343:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2ef82 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26116096->26120191] (req 26116096->26120191) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760759.134662:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973695 is committed 00002000:00010000:1.0:1789760759.134789:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26116096 -> 26120192 00010000:00010000:1.0:1789760759.134805:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2ef82 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26116096->26120191] (req 26116096->26120191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.134818:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2ef82 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26116096->26120191] (req 26116096->26120191) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.134828:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2ef82 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26116096->26120191] (req 26116096->26120191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.134836:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760759.134841:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2ef82 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26116096->26120191] (req 26116096->26120191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.134850:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2ef82 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26116096->26120191] (req 26116096->26120191) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.134869:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2ef82 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26116096->26120191] (req 26116096->26120191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.134885:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2ef82 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26116096->26120191] (req 26116096->26120191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760759.134975:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac403bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512677888:12345-192.168.203.56@tcp:4:dd.0 Request processed in 35731us (36009us total) trans 4294973695 rc 0/0 00000100:00100000:1.0:1789760759.134985:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6406 00000100:00100000:1.0:1789760759.134994:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 738613882560 time 738663527638 state 0x2 ref 0 00000100:00100000:2.0:1789760759.138764:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760759.140307:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512678016 00000100:00100000:3.0:1789760759.140319:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512678016 00000100:00100000:3.0:1789760759.140424:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 738613882560 time 738663527638 state 0x0 ref 1 00000100:00100000:3.0:1789760759.140448:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 738663788054 time 738663527638 state 0x1 ref 1 00000100:00100000:3.0:1789760759.140477:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 738663788054 time 738704776754 state 0x0 ref 1 00000100:00100000:3.0:1789760759.140489:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4132d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512678016:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760759.140531:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26120192, mtime: 0x6aad94f6, atime: 0x0, ctime: 0x6aad94f6, blocks: 0xc807 00010000:00010000:3.0:1789760759.140540:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2ef89 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.140555:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=738s v=(0 0) 00010000:00010000:3.0:1789760759.140559:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760759.141045:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2ef89 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26120192->26124287] (req 26120192->26124287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.141070:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2ef89 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26120192->26124287] (req 26120192->26124287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.154938:0:7647:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:1.0:1789760759.195860:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973696 is committed 00002000:00010000:2.0:1789760759.196018:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26120192 -> 26124288 00010000:00010000:2.0:1789760759.196034:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2ef89 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26120192->26124287] (req 26120192->26124287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.196047:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2ef89 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26120192->26124287] (req 26120192->26124287) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.196056:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2ef89 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26120192->26124287] (req 26120192->26124287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.196069:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760759.196075:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2ef89 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26120192->26124287] (req 26120192->26124287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.196083:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2ef89 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26120192->26124287] (req 26120192->26124287) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.196100:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2ef89 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26120192->26124287] (req 26120192->26124287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.196107:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2ef89 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26120192->26124287] (req 26120192->26124287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760759.196229:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4132d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512678016:12345-192.168.203.56@tcp:4:dd.0 Request processed in 55737us (57479us total) trans 4294973696 rc 0/0 00000100:00100000:2.0:1789760759.196238:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6407 00000100:00100000:2.0:1789760759.196248:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 738663788054 time 738704776754 state 0x2 ref 0 00000100:00100000:2.0:1789760759.202061:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760759.202266:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512678144 00000100:00100000:0.0:1789760759.202275:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512678144 00000100:00100000:0.0:1789760759.202333:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 738663788054 time 738704776754 state 0x0 ref 1 00000100:00100000:0.0:1789760759.202347:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 738705037170 time 738704776754 state 0x1 ref 1 00000100:00100000:0.0:1789760759.202427:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 738705037170 time 738766725940 state 0x0 ref 1 00000100:00100000:0.0:1789760759.202440:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4132300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512678144:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760759.202489:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26124288, mtime: 0x6aad94f6, atime: 0x0, ctime: 0x6aad94f6, blocks: 0xc807 00010000:00010000:0.0:1789760759.202498:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2ef90 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.202518:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=738s v=(0 0) 00010000:00010000:0.0:1789760759.202523:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760759.202529:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2ef90 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26124288->26128383] (req 26124288->26128383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.202542:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2ef90 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26124288->26128383] (req 26124288->26128383) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760759.248354:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26124288 -> 26128384 00010000:00010000:2.0:1789760759.248373:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2ef90 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26124288->26128383] (req 26124288->26128383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.248387:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2ef90 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26124288->26128383] (req 26124288->26128383) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.248402:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2ef90 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26124288->26128383] (req 26124288->26128383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.248410:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760759.248415:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2ef90 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26124288->26128383] (req 26124288->26128383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.248425:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2ef90 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26124288->26128383] (req 26124288->26128383) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.248445:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2ef90 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26124288->26128383] (req 26124288->26128383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.248453:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2ef90 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26124288->26128383] (req 26124288->26128383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760759.248548:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4132300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2360:x1876699512678144:12345-192.168.203.56@tcp:4:dd.0 Request processed in 46104us (46490us total) trans 4294973697 rc 0/0 00000100:00100000:2.0:1789760759.248558:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6408 00000100:00100000:2.0:1789760759.248577:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 738705037170 time 738766725940 state 0x2 ref 0 00000001:00080000:3.0:1789760759.249490:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973697 is committed 00000100:00100000:1.0:1789760759.254285:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760759.254383:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512678272 00000100:00100000:3.0:1789760759.254392:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512678272 00000100:00100000:3.0:1789760759.254442:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 738705037170 time 738766725940 state 0x0 ref 1 00000100:00100000:3.0:1789760759.254458:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 738766986356 time 738766725940 state 0x1 ref 1 00000100:00100000:3.0:1789760759.254519:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 738766986356 time 738818817858 state 0x0 ref 1 00000100:00100000:3.0:1789760759.254532:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac403b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512678272:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760759.254580:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26128384, mtime: 0x6aad94f6, atime: 0x0, ctime: 0x6aad94f6, blocks: 0xc807 00010000:00010000:3.0:1789760759.254589:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2ef97 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.254616:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=738s v=(0 0) 00010000:00010000:3.0:1789760759.254620:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760759.254628:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2ef97 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26128384->26132479] (req 26128384->26132479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.254642:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2ef97 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26128384->26132479] (req 26128384->26132479) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760759.286920:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973698 is committed 00002000:00010000:1.0:1789760759.287063:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26128384 -> 26132480 00010000:00010000:1.0:1789760759.287077:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2ef97 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26128384->26132479] (req 26128384->26132479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.287087:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2ef97 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26128384->26132479] (req 26128384->26132479) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.287093:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2ef97 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26128384->26132479] (req 26128384->26132479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.287099:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760759.287103:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2ef97 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26128384->26132479] (req 26128384->26132479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.287108:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2ef97 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26128384->26132479] (req 26128384->26132479) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.287122:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2ef97 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26128384->26132479] (req 26128384->26132479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.287126:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2ef97 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26128384->26132479] (req 26128384->26132479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760759.287310:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac403b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512678272:12345-192.168.203.56@tcp:4:dd.0 Request processed in 32775us (33028us total) trans 4294973698 rc 0/0 00000100:00100000:1.0:1789760759.287321:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6409 00000100:00100000:1.0:1789760759.287331:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 738766986356 time 738818817858 state 0x2 ref 0 00000100:00100000:1.0:1789760759.290349:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760759.290646:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512678400 00000100:00100000:2.0:1789760759.290657:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512678400 00000100:00100000:2.0:1789760759.290830:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 738766986356 time 738818817858 state 0x0 ref 1 00000100:00100000:2.0:1789760759.290847:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 738819078274 time 738818817858 state 0x1 ref 1 00000100:00100000:2.0:1789760759.290885:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 738819078274 time 738855184480 state 0x0 ref 1 00000100:00100000:2.0:1789760759.290897:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febea4b7c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512678400:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760759.290942:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26132480, mtime: 0x6aad94f6, atime: 0x0, ctime: 0x6aad94f6, blocks: 0xc807 00010000:00010000:2.0:1789760759.290951:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2ef9e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.291106:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=738s v=(0 0) 00010000:00010000:2.0:1789760759.291110:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760759.291117:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2ef9e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26132480->26136575] (req 26132480->26136575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.291131:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2ef9e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26132480->26136575] (req 26132480->26136575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760759.321565:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973699 is committed 00002000:00010000:2.0:1789760759.321652:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26132480 -> 26136576 00010000:00010000:2.0:1789760759.321669:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2ef9e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26132480->26136575] (req 26132480->26136575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.321683:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2ef9e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26132480->26136575] (req 26132480->26136575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.321692:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2ef9e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26132480->26136575] (req 26132480->26136575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.321720:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760759.321726:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2ef9e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26132480->26136575] (req 26132480->26136575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.321737:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2ef9e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26132480->26136575] (req 26132480->26136575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.321757:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2ef9e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26132480->26136575] (req 26132480->26136575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.321765:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2ef9e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26132480->26136575] (req 26132480->26136575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760759.322028:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febea4b7c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512678400:12345-192.168.203.56@tcp:4:dd.0 Request processed in 31058us (31614us total) trans 4294973699 rc 0/0 00000100:00100000:2.0:1789760759.322040:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6410 00000100:00100000:2.0:1789760759.322051:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 738819078274 time 738855184480 state 0x2 ref 0 00000100:00100000:2.0:1789760759.327493:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760759.327669:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512678528 00000100:00100000:0.0:1789760759.327678:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512678528 00000100:00100000:1.0:1789760759.328088:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 738819078274 time 738855184480 state 0x0 ref 1 00000100:00100000:1.0:1789760759.328106:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 738855444896 time 738855184480 state 0x1 ref 1 00000100:00100000:1.0:1789760759.328181:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 738855444896 time 738892478921 state 0x0 ref 1 00000100:00100000:1.0:1789760759.328199:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4130000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512678528:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760759.328257:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26136576, mtime: 0x6aad94f6, atime: 0x0, ctime: 0x6aad94f6, blocks: 0xc807 00010000:00010000:1.0:1789760759.328266:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2efa5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.328283:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=738s v=(0 0) 00010000:00010000:1.0:1789760759.328287:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760759.328293:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2efa5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26136576->26140671] (req 26136576->26140671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.328307:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2efa5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26136576->26140671] (req 26136576->26140671) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760759.358140:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973700 is committed 00002000:00010000:3.0:1789760759.358263:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26136576 -> 26140672 00010000:00010000:3.0:1789760759.358279:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2efa5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26136576->26140671] (req 26136576->26140671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.358291:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2efa5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26136576->26140671] (req 26136576->26140671) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.358300:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2efa5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26136576->26140671] (req 26136576->26140671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.358307:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760759.358312:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2efa5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26136576->26140671] (req 26136576->26140671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.358322:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2efa5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26136576->26140671] (req 26136576->26140671) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.358345:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2efa5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26136576->26140671] (req 26136576->26140671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.358354:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2efa5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26136576->26140671] (req 26136576->26140671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760759.358477:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4130000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512678528:12345-192.168.203.56@tcp:4:dd.0 Request processed in 30278us (30991us total) trans 4294973700 rc 0/0 00000100:00100000:3.0:1789760759.358488:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6411 00000100:00100000:3.0:1789760759.358498:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 738855444896 time 738892478921 state 0x2 ref 0 00000100:00100000:1.0:1789760759.367937:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760759.368663:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512678656 00000100:00100000:2.0:1789760759.368674:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512678656 00000100:00100000:2.0:1789760759.368742:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 738855444896 time 738892478921 state 0x0 ref 1 00000100:00100000:2.0:1789760759.368757:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 738892739337 time 738892478921 state 0x1 ref 1 00000100:00100000:2.0:1789760759.368788:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 738892739337 time 738933088140 state 0x0 ref 1 00000100:00100000:2.0:1789760759.368799:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4038000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512678656:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760759.368844:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26140672, mtime: 0x6aad94f6, atime: 0x0, ctime: 0x6aad94f6, blocks: 0xc807 00010000:00010000:2.0:1789760759.368852:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2efac lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.368865:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=738s v=(0 0) 00010000:00010000:2.0:1789760759.368869:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760759.368874:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2efac lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26140672->26144767] (req 26140672->26144767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.368886:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2efac lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26140672->26144767] (req 26140672->26144767) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760759.405411:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973701 is committed 00002000:00010000:1.0:1789760759.405587:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26140672 -> 26144768 00010000:00010000:1.0:1789760759.405604:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2efac lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26140672->26144767] (req 26140672->26144767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.405618:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2efac lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26140672->26144767] (req 26140672->26144767) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.405627:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2efac lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26140672->26144767] (req 26140672->26144767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.405635:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760759.405641:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2efac lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26140672->26144767] (req 26140672->26144767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.405651:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2efac lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26140672->26144767] (req 26140672->26144767) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.405671:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2efac lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26140672->26144767] (req 26140672->26144767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.405679:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2efac lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26140672->26144767] (req 26140672->26144767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760759.405810:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4038000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512678656:12345-192.168.203.56@tcp:4:dd.0 Request processed in 37007us (37937us total) trans 4294973701 rc 0/0 00000100:00100000:1.0:1789760759.405821:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6412 00000100:00100000:1.0:1789760759.405831:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 738892739337 time 738933088140 state 0x2 ref 0 00000100:00100000:1.0:1789760759.411920:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760759.412069:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512678784 00000100:00100000:2.0:1789760759.412078:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512678784 00000100:00100000:2.0:1789760759.412137:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 738892739337 time 738933088140 state 0x0 ref 1 00000100:00100000:2.0:1789760759.412153:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 738933348556 time 738933088140 state 0x1 ref 1 00000100:00100000:2.0:1789760759.412317:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 738933348556 time 738976614956 state 0x0 ref 1 00000100:00100000:2.0:1789760759.412332:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4038700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512678784:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760759.412381:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26144768, mtime: 0x6aad94f6, atime: 0x0, ctime: 0x6aad94f6, blocks: 0xc807 00010000:00010000:2.0:1789760759.412390:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2efb3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.412405:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=738s v=(0 0) 00010000:00010000:2.0:1789760759.412410:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760759.412416:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2efb3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26144768->26148863] (req 26144768->26148863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.412430:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2efb3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26144768->26148863] (req 26144768->26148863) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760759.440744:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973702 is committed 00002000:00010000:1.0:1789760759.441033:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26144768 -> 26148864 00010000:00010000:1.0:1789760759.441047:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2efb3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26144768->26148863] (req 26144768->26148863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.441056:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2efb3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26144768->26148863] (req 26144768->26148863) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.441063:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2efb3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26144768->26148863] (req 26144768->26148863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.441116:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760759.441121:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2efb3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26144768->26148863] (req 26144768->26148863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.441128:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2efb3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26144768->26148863] (req 26144768->26148863) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.441146:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2efb3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26144768->26148863] (req 26144768->26148863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.441154:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2efb3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26144768->26148863] (req 26144768->26148863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760759.441252:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4038700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512678784:12345-192.168.203.56@tcp:4:dd.0 Request processed in 28919us (29337us total) trans 4294973702 rc 0/0 00000100:00100000:1.0:1789760759.441267:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6413 00000100:00100000:1.0:1789760759.441277:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 738933348556 time 738976614956 state 0x2 ref 0 00000100:00100000:2.0:1789760759.444219:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760759.444500:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512678912 00000100:00100000:3.0:1789760759.444510:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512678912 00000100:00100000:3.0:1789760759.444560:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 738933348556 time 738976614956 state 0x0 ref 1 00000100:00100000:3.0:1789760759.444575:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 738976875372 time 738976614956 state 0x1 ref 1 00000100:00100000:0.0:1789760759.444647:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 738976875372 time 739008942948 state 0x0 ref 1 00000100:00100000:0.0:1789760759.444664:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4133480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512678912:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760759.444734:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26148864, mtime: 0x6aad94f6, atime: 0x0, ctime: 0x6aad94f6, blocks: 0xc807 00010000:00010000:0.0:1789760759.444743:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2efba lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.444758:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=739s v=(0 0) 00010000:00010000:0.0:1789760759.444762:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760759.444769:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2efba lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26148864->26152959] (req 26148864->26152959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.444782:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2efba lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26148864->26152959] (req 26148864->26152959) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760759.474029:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973703 is committed 00002000:00010000:0.0:1789760759.474130:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26148864 -> 26152960 00010000:00010000:0.0:1789760759.474145:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2efba lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26148864->26152959] (req 26148864->26152959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.474158:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2efba lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26148864->26152959] (req 26148864->26152959) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.474169:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2efba lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26148864->26152959] (req 26148864->26152959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.474176:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760759.474181:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2efba lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26148864->26152959] (req 26148864->26152959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.474190:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2efba lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26148864->26152959] (req 26148864->26152959) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.474220:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2efba lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26148864->26152959] (req 26148864->26152959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.474229:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2efba lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26148864->26152959] (req 26148864->26152959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760759.474499:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4133480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512678912:12345-192.168.203.56@tcp:4:dd.0 Request processed in 29836us (30297us total) trans 4294973703 rc 0/0 00000100:00100000:0.0:1789760759.474513:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6414 00000100:00100000:0.0:1789760759.474523:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 738976875372 time 739008942948 state 0x2 ref 0 00000100:00100000:0.0:1789760759.477726:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760759.477811:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512679040 00000100:00100000:1.0:1789760759.477822:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512679040 00000100:00100000:1.0:1789760759.477882:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 738976875372 time 739008942948 state 0x0 ref 1 00000100:00100000:1.0:1789760759.477896:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 739009203364 time 739008942948 state 0x1 ref 1 00000100:00100000:1.0:1789760759.477923:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 739009203364 time 739042223087 state 0x0 ref 1 00000100:00100000:1.0:1789760759.477939:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2fba680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512679040:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760759.477985:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26152960, mtime: 0x6aad94f6, atime: 0x0, ctime: 0x6aad94f6, blocks: 0xc807 00010000:00010000:1.0:1789760759.477993:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2efc1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.478009:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=739s v=(0 0) 00010000:00010000:1.0:1789760759.478013:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760759.478020:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2efc1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26152960->26157055] (req 26152960->26157055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.478033:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2efc1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26152960->26157055] (req 26152960->26157055) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760759.509216:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973704 is committed 00002000:00010000:2.0:1789760759.510110:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26152960 -> 26157056 00010000:00010000:2.0:1789760759.510125:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2efc1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26152960->26157055] (req 26152960->26157055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.510140:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2efc1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26152960->26157055] (req 26152960->26157055) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.510150:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2efc1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26152960->26157055] (req 26152960->26157055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.510157:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760759.510162:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2efc1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26152960->26157055] (req 26152960->26157055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.510172:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2efc1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26152960->26157055] (req 26152960->26157055) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.510192:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2efc1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26152960->26157055] (req 26152960->26157055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.510214:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2efc1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26152960->26157055] (req 26152960->26157055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760759.510443:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2fba680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512679040:12345-192.168.203.56@tcp:4:dd.0 Request processed in 32500us (32752us total) trans 4294973704 rc 0/0 00000100:00100000:2.0:1789760759.510455:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6415 00000100:00100000:2.0:1789760759.510464:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 739009203364 time 739042223087 state 0x2 ref 0 00000100:00100000:1.0:1789760759.513477:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760759.513560:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512679168 00000100:00100000:2.0:1789760759.513570:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512679168 00000100:00100000:2.0:1789760759.513620:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 739009203364 time 739042223087 state 0x0 ref 1 00000100:00100000:2.0:1789760759.513635:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 739042483503 time 739042223087 state 0x1 ref 1 00000100:00100000:2.0:1789760759.513665:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 739042483503 time 739077964702 state 0x0 ref 1 00000100:00100000:2.0:1789760759.513677:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac403aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512679168:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760759.513756:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26157056, mtime: 0x6aad94f6, atime: 0x0, ctime: 0x6aad94f6, blocks: 0xc807 00010000:00010000:2.0:1789760759.513764:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2efc8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.513779:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=739s v=(0 0) 00010000:00010000:2.0:1789760759.513783:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760759.513789:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2efc8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26157056->26161151] (req 26157056->26161151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.513807:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2efc8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26157056->26161151] (req 26157056->26161151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760759.539980:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973705 is committed 00002000:00010000:2.0:1789760759.540057:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26157056 -> 26161152 00002000:00010000:2.0:1789760759.540060:0:6710:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760758 -> 1789760759 00002000:00010000:2.0:1789760759.540062:0:6710:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760758 -> 1789760759 00010000:00010000:2.0:1789760759.540073:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2efc8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26157056->26161151] (req 26157056->26161151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.540084:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2efc8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26157056->26161151] (req 26157056->26161151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.540093:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2efc8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26157056->26161151] (req 26157056->26161151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.540099:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760759.540104:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2efc8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26157056->26161151] (req 26157056->26161151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.540113:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2efc8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26157056->26161151] (req 26157056->26161151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.540133:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2efc8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26157056->26161151] (req 26157056->26161151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.540145:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2efc8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26157056->26161151] (req 26157056->26161151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760759.540322:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac403aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512679168:12345-192.168.203.56@tcp:4:dd.0 Request processed in 26643us (26849us total) trans 4294973705 rc 0/0 00000100:00100000:2.0:1789760759.540333:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6416 00000100:00100000:2.0:1789760759.540343:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 739042483503 time 739077964702 state 0x2 ref 0 00000100:00100000:3.0:1789760759.543033:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760759.543382:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512679296 00000100:00100000:0.0:1789760759.543391:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512679296 00000100:00100000:0.0:1789760759.543437:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 739042483503 time 739077964702 state 0x0 ref 1 00000100:00100000:0.0:1789760759.543448:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 739078225118 time 739077964702 state 0x1 ref 1 00000100:00100000:0.0:1789760759.543474:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 739078225118 time 739107773570 state 0x0 ref 1 00000100:00100000:0.0:1789760759.543486:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febea89b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512679296:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760759.543533:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26161152, mtime: 0x6aad94f7, atime: 0x0, ctime: 0x6aad94f7, blocks: 0xc807 00010000:00010000:0.0:1789760759.543542:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2efcf lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.543557:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=739s v=(0 0) 00010000:00010000:0.0:1789760759.543561:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760759.543567:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2efcf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26161152->26165247] (req 26161152->26165247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.543580:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2efcf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26161152->26165247] (req 26161152->26165247) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760759.574384:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973706 is committed 00002000:00010000:2.0:1789760759.574482:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26161152 -> 26165248 00010000:00010000:2.0:1789760759.574497:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2efcf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26161152->26165247] (req 26161152->26165247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.574511:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2efcf lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26161152->26165247] (req 26161152->26165247) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.574520:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2efcf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26161152->26165247] (req 26161152->26165247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.574527:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760759.574532:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2efcf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26161152->26165247] (req 26161152->26165247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.574541:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2efcf lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26161152->26165247] (req 26161152->26165247) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.574564:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2efcf lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26161152->26165247] (req 26161152->26165247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.574571:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2efcf lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26161152->26165247] (req 26161152->26165247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760759.574666:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febea89b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512679296:12345-192.168.203.56@tcp:4:dd.0 Request processed in 31175us (31635us total) trans 4294973706 rc 0/0 00000100:00100000:2.0:1789760759.574676:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6417 00000100:00100000:2.0:1789760759.574686:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 739078225118 time 739107773570 state 0x2 ref 0 00000100:00100000:0.0:1789760759.578235:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760759.578452:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512679424 00000100:00100000:2.0:1789760759.578463:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512679424 00000100:00100000:2.0:1789760759.578513:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 739078225118 time 739107773570 state 0x0 ref 1 00000100:00100000:2.0:1789760759.578527:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 739108033986 time 739107773570 state 0x1 ref 1 00000100:00100000:2.0:1789760759.578556:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 739108033986 time 739142856134 state 0x0 ref 1 00000100:00100000:2.0:1789760759.578568:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2fb8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512679424:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760759.578616:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26165248, mtime: 0x6aad94f7, atime: 0x0, ctime: 0x6aad94f7, blocks: 0xc807 00010000:00010000:2.0:1789760759.578626:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2efd6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.578641:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=739s v=(0 0) 00010000:00010000:2.0:1789760759.578646:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760759.578652:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2efd6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26165248->26169343] (req 26165248->26169343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.578665:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2efd6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26165248->26169343] (req 26165248->26169343) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760759.608861:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973707 is committed 00002000:00010000:0.0:1789760759.609025:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26165248 -> 26169344 00010000:00010000:0.0:1789760759.609040:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2efd6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26165248->26169343] (req 26165248->26169343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.609053:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2efd6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26165248->26169343] (req 26165248->26169343) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.609063:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2efd6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26165248->26169343] (req 26165248->26169343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.609151:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760759.609157:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2efd6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26165248->26169343] (req 26165248->26169343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.609167:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2efd6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26165248->26169343] (req 26165248->26169343) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.609186:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2efd6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26165248->26169343] (req 26165248->26169343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.609195:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2efd6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26165248->26169343] (req 26165248->26169343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760759.609288:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2fb8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512679424:12345-192.168.203.56@tcp:4:dd.0 Request processed in 30717us (31056us total) trans 4294973707 rc 0/0 00000100:00100000:0.0:1789760759.609298:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6418 00000100:00100000:0.0:1789760759.609308:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 739108033986 time 739142856134 state 0x2 ref 0 00000100:00100000:1.0:1789760759.614258:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760759.614446:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512679552 00000100:00100000:2.0:1789760759.614455:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512679552 00000100:00100000:2.0:1789760759.614505:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 739108033986 time 739142856134 state 0x0 ref 1 00000100:00100000:2.0:1789760759.614521:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 739143116550 time 739142856134 state 0x1 ref 1 00000100:00100000:2.0:1789760759.614551:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 739143116550 time 739178850577 state 0x0 ref 1 00000100:00100000:2.0:1789760759.614563:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4039880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512679552:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760759.614609:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26169344, mtime: 0x6aad94f7, atime: 0x0, ctime: 0x6aad94f7, blocks: 0xc807 00010000:00010000:2.0:1789760759.614618:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2efdd lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.614633:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=739s v=(0 0) 00010000:00010000:2.0:1789760759.614638:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760759.614645:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2efdd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26169344->26173439] (req 26169344->26173439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.614663:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2efdd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26169344->26173439] (req 26169344->26173439) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760759.652274:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973708 is committed 00002000:00010000:2.0:1789760759.652514:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26169344 -> 26173440 00010000:00010000:2.0:1789760759.652530:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2efdd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26169344->26173439] (req 26169344->26173439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.652543:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2efdd lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26169344->26173439] (req 26169344->26173439) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.652552:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2efdd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26169344->26173439] (req 26169344->26173439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.652559:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760759.652564:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2efdd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26169344->26173439] (req 26169344->26173439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.652571:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2efdd lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26169344->26173439] (req 26169344->26173439) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.652590:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2efdd lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26169344->26173439] (req 26169344->26173439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.652598:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2efdd lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26169344->26173439] (req 26169344->26173439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760759.652801:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4039880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512679552:12345-192.168.203.56@tcp:4:dd.0 Request processed in 38232us (38596us total) trans 4294973708 rc 0/0 00000100:00100000:2.0:1789760759.652818:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6419 00000100:00100000:2.0:1789760759.652830:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 739143116550 time 739178850577 state 0x2 ref 0 00000100:00100000:2.0:1789760759.656535:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760759.658643:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512679680 00000100:00100000:1.0:1789760759.658657:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512679680 00000100:00100000:1.0:1789760759.658788:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 739143116550 time 739178850577 state 0x0 ref 1 00000100:00100000:1.0:1789760759.658803:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 739179110993 time 739178850577 state 0x1 ref 1 00000100:00100000:1.0:1789760759.658834:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 739179110993 time 739223134021 state 0x0 ref 1 00000100:00100000:1.0:1789760759.658903:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4130700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512679680:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760759.659138:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26173440, mtime: 0x6aad94f7, atime: 0x0, ctime: 0x6aad94f7, blocks: 0xc807 00010000:00010000:1.0:1789760759.659149:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2efe4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.659164:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=739s v=(0 0) 00010000:00010000:1.0:1789760759.659168:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760759.659174:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2efe4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26173440->26177535] (req 26173440->26177535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.659187:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2efe4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26173440->26177535] (req 26173440->26177535) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760759.687919:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973709 is committed 00002000:00010000:2.0:1789760759.688236:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26173440 -> 26177536 00010000:00010000:2.0:1789760759.688251:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2efe4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26173440->26177535] (req 26173440->26177535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.688263:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2efe4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26173440->26177535] (req 26173440->26177535) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.688272:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2efe4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26173440->26177535] (req 26173440->26177535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.688280:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760759.688284:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2efe4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26173440->26177535] (req 26173440->26177535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.688292:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2efe4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26173440->26177535] (req 26173440->26177535) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.688383:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2efe4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26173440->26177535] (req 26173440->26177535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.688392:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2efe4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26173440->26177535] (req 26173440->26177535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760759.688483:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4130700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512679680:12345-192.168.203.56@tcp:4:dd.0 Request processed in 29634us (31952us total) trans 4294973709 rc 0/0 00000100:00100000:2.0:1789760759.688494:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6420 00000100:00100000:2.0:1789760759.688503:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 739179110993 time 739223134021 state 0x2 ref 0 00000100:00100000:1.0:1789760759.692006:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760759.692124:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512679808 00000100:00100000:0.0:1789760759.692133:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512679808 00000100:00100000:0.0:1789760759.692177:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 739179110993 time 739223134021 state 0x0 ref 1 00000100:00100000:0.0:1789760759.692190:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 739223394437 time 739223134021 state 0x1 ref 1 00000100:00100000:0.0:1789760759.692225:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 739223394437 time 739256524802 state 0x0 ref 1 00000100:00100000:0.0:1789760759.692237:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4039180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512679808:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760759.692289:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26177536, mtime: 0x6aad94f7, atime: 0x0, ctime: 0x6aad94f7, blocks: 0xc807 00010000:00010000:0.0:1789760759.692415:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2efeb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.692431:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=739s v=(0 0) 00010000:00010000:0.0:1789760759.692436:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760759.692442:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2efeb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26177536->26181631] (req 26177536->26181631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.692455:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2efeb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26177536->26181631] (req 26177536->26181631) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760759.723763:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973710 is committed 00002000:00010000:0.0:1789760759.723855:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26177536 -> 26181632 00010000:00010000:0.0:1789760759.723923:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2efeb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26177536->26181631] (req 26177536->26181631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.723942:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2efeb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26177536->26181631] (req 26177536->26181631) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.723950:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2efeb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26177536->26181631] (req 26177536->26181631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.723956:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760759.723961:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2efeb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26177536->26181631] (req 26177536->26181631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.723969:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2efeb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26177536->26181631] (req 26177536->26181631) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.723985:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2efeb lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26177536->26181631] (req 26177536->26181631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.723991:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2efeb lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26177536->26181631] (req 26177536->26181631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760759.724180:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4039180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512679808:12345-192.168.203.56@tcp:4:dd.0 Request processed in 31939us (32177us total) trans 4294973710 rc 0/0 00000100:00100000:0.0:1789760759.724191:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6421 00000100:00100000:0.0:1789760759.724222:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 739223394437 time 739256524802 state 0x2 ref 0 00000100:00100000:1.0:1789760759.726845:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760759.726952:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512679936 00000100:00100000:2.0:1789760759.726962:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512679936 00000100:00100000:2.0:1789760759.727014:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 739223394437 time 739256524802 state 0x0 ref 1 00000100:00100000:2.0:1789760759.727091:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 739256785218 time 739256524802 state 0x1 ref 1 00000100:00100000:2.0:1789760759.727124:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 739256785218 time 739291423749 state 0x0 ref 1 00000100:00100000:2.0:1789760759.727136:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4039500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512679936:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760759.727185:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26181632, mtime: 0x6aad94f7, atime: 0x0, ctime: 0x6aad94f7, blocks: 0xc807 00010000:00010000:2.0:1789760759.727193:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2eff2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.727223:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=739s v=(0 0) 00010000:00010000:2.0:1789760759.727227:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760759.727232:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2eff2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26181632->26185727] (req 26181632->26185727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.727246:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2eff2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26181632->26185727] (req 26181632->26185727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760759.768141:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26181632 -> 26185728 00010000:00010000:1.0:1789760759.768161:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2eff2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26181632->26185727] (req 26181632->26185727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.768174:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2eff2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26181632->26185727] (req 26181632->26185727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.768182:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2eff2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26181632->26185727] (req 26181632->26185727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.768190:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760759.768195:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2eff2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26181632->26185727] (req 26181632->26185727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.768219:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2eff2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26181632->26185727] (req 26181632->26185727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.768239:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2eff2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26181632->26185727] (req 26181632->26185727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.768246:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2eff2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26181632->26185727] (req 26181632->26185727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760759.768410:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4039500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2358:x1876699512679936:12345-192.168.203.56@tcp:4:dd.0 Request processed in 41270us (41568us total) trans 4294973711 rc 0/0 00000100:00100000:1.0:1789760759.768422:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6422 00000100:00100000:1.0:1789760759.768431:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 739256785218 time 739291423749 state 0x2 ref 0 00000001:00080000:3.0:1789760759.768510:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973711 is committed 00000100:00100000:2.0:1789760759.771869:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760759.772050:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512680064 00000100:00100000:1.0:1789760759.772059:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512680064 00000100:00100000:1.0:1789760759.772112:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 739256785218 time 739291423749 state 0x0 ref 1 00000100:00100000:1.0:1789760759.772127:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 739291684165 time 739291423749 state 0x1 ref 1 00000100:00100000:1.0:1789760759.772217:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 739291684165 time 739336515816 state 0x0 ref 1 00000100:00100000:1.0:1789760759.772230:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4131180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512680064:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760759.772279:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26185728, mtime: 0x6aad94f7, atime: 0x0, ctime: 0x6aad94f7, blocks: 0xc807 00010000:00010000:1.0:1789760759.772288:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2eff9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.772314:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=739s v=(0 0) 00010000:00010000:1.0:1789760759.772319:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760759.772325:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2eff9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26185728->26189823] (req 26185728->26189823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.772339:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2eff9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26185728->26189823] (req 26185728->26189823) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760759.810881:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973712 is committed 00002000:00010000:3.0:1789760759.813945:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26185728 -> 26189824 00010000:00010000:3.0:1789760759.813965:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2eff9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26185728->26189823] (req 26185728->26189823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.813979:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2eff9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26185728->26189823] (req 26185728->26189823) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.813988:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2eff9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26185728->26189823] (req 26185728->26189823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.813995:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760759.814001:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2eff9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26185728->26189823] (req 26185728->26189823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.814009:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2eff9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26185728->26189823] (req 26185728->26189823) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.814028:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2eff9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26185728->26189823] (req 26185728->26189823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.814035:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2eff9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26185728->26189823] (req 26185728->26189823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760759.814123:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4131180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512680064:12345-192.168.203.56@tcp:4:dd.0 Request processed in 41891us (42259us total) trans 4294973712 rc 0/0 00000100:00100000:3.0:1789760759.814133:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6423 00000100:00100000:3.0:1789760759.814143:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 739291684165 time 739336515816 state 0x2 ref 0 00000100:00100000:0.0:1789760759.819216:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760759.819365:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512680192 00000100:00100000:2.0:1789760759.819377:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512680192 00000100:00100000:2.0:1789760759.819430:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 739291684165 time 739336515816 state 0x0 ref 1 00000100:00100000:2.0:1789760759.819446:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 739336776232 time 739336515816 state 0x1 ref 1 00000100:00100000:2.0:1789760759.819479:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 739336776232 time 739383778535 state 0x0 ref 1 00000100:00100000:2.0:1789760759.819492:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2fb8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512680192:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760759.819555:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26189824, mtime: 0x6aad94f7, atime: 0x0, ctime: 0x6aad94f7, blocks: 0xc807 00010000:00010000:2.0:1789760759.819564:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2f000 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.819579:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=739s v=(0 0) 00010000:00010000:2.0:1789760759.819584:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760759.819590:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2f000 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26189824->26193919] (req 26189824->26193919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.819603:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2f000 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26189824->26193919] (req 26189824->26193919) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760759.863670:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973713 is committed 00002000:00010000:3.0:1789760759.863760:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26189824 -> 26193920 00010000:00010000:3.0:1789760759.863775:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2f000 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26189824->26193919] (req 26189824->26193919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.863788:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2f000 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26189824->26193919] (req 26189824->26193919) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.863797:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2f000 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26189824->26193919] (req 26189824->26193919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.863808:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760759.863812:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2f000 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26189824->26193919] (req 26189824->26193919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.863820:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2f000 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26189824->26193919] (req 26189824->26193919) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.863837:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2f000 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26189824->26193919] (req 26189824->26193919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.863845:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2f000 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26189824->26193919] (req 26189824->26193919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760759.863936:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2fb8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512680192:12345-192.168.203.56@tcp:4:dd.0 Request processed in 44440us (44731us total) trans 4294973713 rc 0/0 00000100:00100000:3.0:1789760759.863945:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6424 00000100:00100000:3.0:1789760759.863954:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 739336776232 time 739383778535 state 0x2 ref 0 00000100:00100000:1.0:1789760759.867055:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760759.867228:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512680320 00000100:00100000:2.0:1789760759.867238:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512680320 00000100:00100000:2.0:1789760759.867289:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 739336776232 time 739383778535 state 0x0 ref 1 00000100:00100000:2.0:1789760759.867305:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 739384038951 time 739383778535 state 0x1 ref 1 00000100:00100000:2.0:1789760759.867336:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 739384038951 time 739431635134 state 0x0 ref 1 00000100:00100000:2.0:1789760759.867350:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4038a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512680320:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760759.867395:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26193920, mtime: 0x6aad94f7, atime: 0x0, ctime: 0x6aad94f7, blocks: 0xc807 00010000:00010000:2.0:1789760759.867405:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2f007 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.867422:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=739s v=(0 0) 00010000:00010000:2.0:1789760759.867427:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760759.867433:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2f007 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26193920->26198015] (req 26193920->26198015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.867448:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2f007 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26193920->26198015] (req 26193920->26198015) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760759.902942:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973714 is committed 00002000:00010000:3.0:1789760759.904420:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26193920 -> 26198016 00010000:00010000:3.0:1789760759.904437:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2f007 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26193920->26198015] (req 26193920->26198015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.904450:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2f007 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26193920->26198015] (req 26193920->26198015) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.904464:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2f007 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26193920->26198015] (req 26193920->26198015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.904471:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760759.904476:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2f007 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26193920->26198015] (req 26193920->26198015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.904486:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2f007 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26193920->26198015] (req 26193920->26198015) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.904505:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2f007 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26193920->26198015] (req 26193920->26198015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.904512:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2f007 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26193920->26198015] (req 26193920->26198015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760759.904611:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4038a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512680320:12345-192.168.203.56@tcp:4:dd.0 Request processed in 37260us (37559us total) trans 4294973714 rc 0/0 00000100:00100000:3.0:1789760759.904622:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6425 00000100:00100000:3.0:1789760759.904631:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 739384038951 time 739431635134 state 0x2 ref 0 00000100:00100000:2.0:1789760759.909551:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760759.909644:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512680448 00000100:00100000:3.0:1789760759.909655:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512680448 00000100:00100000:3.0:1789760759.909754:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 739384038951 time 739431635134 state 0x0 ref 1 00000100:00100000:3.0:1789760759.909768:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 739431895550 time 739431635134 state 0x1 ref 1 00000100:00100000:3.0:1789760759.909801:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 739431895550 time 739474100950 state 0x0 ref 1 00000100:00100000:3.0:1789760759.909814:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4130a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512680448:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760759.909860:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26198016, mtime: 0x6aad94f7, atime: 0x0, ctime: 0x6aad94f7, blocks: 0xc807 00010000:00010000:3.0:1789760759.909867:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2f00e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.909882:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=739s v=(0 0) 00010000:00010000:3.0:1789760759.909886:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760759.909891:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2f00e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26198016->26202111] (req 26198016->26202111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.909904:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2f00e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26198016->26202111] (req 26198016->26202111) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760759.946968:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973715 is committed 00002000:00010000:1.0:1789760759.947052:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26198016 -> 26202112 00010000:00010000:1.0:1789760759.947067:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2f00e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26198016->26202111] (req 26198016->26202111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.947079:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2f00e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26198016->26202111] (req 26198016->26202111) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.947088:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2f00e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26198016->26202111] (req 26198016->26202111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.947095:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760759.947100:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2f00e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26198016->26202111] (req 26198016->26202111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.947110:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2f00e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26198016->26202111] (req 26198016->26202111) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.947133:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2f00e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26198016->26202111] (req 26198016->26202111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.947140:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2f00e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26198016->26202111] (req 26198016->26202111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760759.947277:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4130a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512680448:12345-192.168.203.56@tcp:4:dd.0 Request processed in 37461us (37731us total) trans 4294973715 rc 0/0 00000100:00100000:1.0:1789760759.947288:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6426 00000100:00100000:1.0:1789760759.947297:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 739431895550 time 739474100950 state 0x2 ref 0 00000100:00100000:0.0:1789760759.950316:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760759.950405:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512680576 00000100:00100000:3.0:1789760759.950416:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512680576 00000100:00100000:3.0:1789760759.950471:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 739431895550 time 739474100950 state 0x0 ref 1 00000100:00100000:3.0:1789760759.950487:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 739474361366 time 739474100950 state 0x1 ref 1 00000100:00100000:3.0:1789760759.950523:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 739474361366 time 739514822400 state 0x0 ref 1 00000100:00100000:3.0:1789760759.950536:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febe7907100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512680576:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760759.950583:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26202112, mtime: 0x6aad94f7, atime: 0x0, ctime: 0x6aad94f7, blocks: 0xc807 00010000:00010000:3.0:1789760759.950599:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2f015 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.950615:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=739s v=(0 0) 00010000:00010000:3.0:1789760759.950620:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760759.950627:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2f015 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26202112->26206207] (req 26202112->26206207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.950642:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2f015 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26202112->26206207] (req 26202112->26206207) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760759.979218:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973716 is committed 00002000:00010000:0.0:1789760759.979330:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26202112 -> 26206208 00010000:00010000:0.0:1789760759.979346:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2f015 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26202112->26206207] (req 26202112->26206207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.979359:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2f015 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26202112->26206207] (req 26202112->26206207) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.979368:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2f015 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26202112->26206207] (req 26202112->26206207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.979375:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760759.979380:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2f015 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26202112->26206207] (req 26202112->26206207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.979390:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2f015 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26202112->26206207] (req 26202112->26206207) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.979409:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2f015 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26202112->26206207] (req 26202112->26206207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.979417:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2f015 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26202112->26206207] (req 26202112->26206207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760759.979507:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febe7907100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512680576:12345-192.168.203.56@tcp:4:dd.0 Request processed in 28968us (29195us total) trans 4294973716 rc 0/0 00000100:00100000:0.0:1789760759.979518:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6427 00000100:00100000:0.0:1789760759.979528:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 739474361366 time 739514822400 state 0x2 ref 0 00000100:00100000:2.0:1789760759.983095:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760759.983210:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512680704 00000100:00100000:3.0:1789760759.983220:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512680704 00000100:00100000:3.0:1789760759.983270:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 739474361366 time 739514822400 state 0x0 ref 1 00000100:00100000:3.0:1789760759.983285:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 739515082816 time 739514822400 state 0x1 ref 1 00000100:00100000:3.0:1789760759.983359:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 739515082816 time 739547657955 state 0x0 ref 1 00000100:00100000:3.0:1789760759.983384:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4132d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512680704:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760759.983430:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26206208, mtime: 0x6aad94f7, atime: 0x0, ctime: 0x6aad94f7, blocks: 0xc807 00010000:00010000:3.0:1789760759.983439:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2f01c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.983454:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=739s v=(0 0) 00010000:00010000:3.0:1789760759.983458:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760759.983464:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2f01c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26206208->26210303] (req 26206208->26210303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.983477:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2f01c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26206208->26210303] (req 26206208->26210303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760760.007652:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973717 is committed 00002000:00010000:3.0:1789760760.007943:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26206208 -> 26210304 00010000:00010000:3.0:1789760760.007959:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2f01c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26206208->26210303] (req 26206208->26210303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.007972:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2f01c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26206208->26210303] (req 26206208->26210303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.007981:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2f01c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26206208->26210303] (req 26206208->26210303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.007989:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760760.007994:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2f01c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26206208->26210303] (req 26206208->26210303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.008003:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2f01c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26206208->26210303] (req 26206208->26210303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.008024:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2f01c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26206208->26210303] (req 26206208->26210303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.008032:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2f01c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26206208->26210303] (req 26206208->26210303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760760.008176:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4132d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512680704:12345-192.168.203.56@tcp:4:dd.0 Request processed in 24799us (25085us total) trans 4294973717 rc 0/0 00000100:00100000:3.0:1789760760.008187:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6428 00000100:00100000:3.0:1789760760.008208:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 739515082816 time 739547657955 state 0x2 ref 0 00000100:00100000:2.0:1789760760.011434:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760760.011544:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512680832 00000100:00100000:3.0:1789760760.011555:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512680832 00000100:00100000:3.0:1789760760.011675:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 739515082816 time 739547657955 state 0x0 ref 1 00000100:00100000:3.0:1789760760.011691:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 739547918371 time 739547657955 state 0x1 ref 1 00000100:00100000:3.0:1789760760.011739:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 739547918371 time 739576038638 state 0x0 ref 1 00000100:00100000:3.0:1789760760.011752:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4132300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512680832:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760760.011801:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26210304, mtime: 0x6aad94f7, atime: 0x0, ctime: 0x6aad94f7, blocks: 0xc807 00010000:00010000:3.0:1789760760.011809:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2f023 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.011825:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=739s v=(0 0) 00010000:00010000:3.0:1789760760.011829:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760760.011837:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2f023 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26210304->26214399] (req 26210304->26214399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.011852:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2f023 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26210304->26214399] (req 26210304->26214399) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760760.038916:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973718 is committed 00002000:00010000:0.0:1789760760.039132:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26210304 -> 26214400 00010000:00010000:0.0:1789760760.039147:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2f023 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26210304->26214399] (req 26210304->26214399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.039159:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2f023 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26210304->26214399] (req 26210304->26214399) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.039168:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2f023 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26210304->26214399] (req 26210304->26214399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.039175:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760760.039180:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2f023 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26210304->26214399] (req 26210304->26214399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.039189:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2f023 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26210304->26214399] (req 26210304->26214399) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.039215:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2f023 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26210304->26214399] (req 26210304->26214399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.039224:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2f023 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26210304->26214399] (req 26210304->26214399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760760.039316:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4132300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512680832:12345-192.168.203.56@tcp:4:dd.0 Request processed in 27561us (27885us total) trans 4294973718 rc 0/0 00000100:00100000:0.0:1789760760.039325:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6429 00000100:00100000:0.0:1789760760.039335:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 739547918371 time 739576038638 state 0x2 ref 0 00000100:00100000:0.0:1789760760.041770:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760760.041922:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512680960 00000100:00100000:3.0:1789760760.041932:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512680960 00000100:00100000:3.0:1789760760.041981:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 739547918371 time 739576038638 state 0x0 ref 1 00000100:00100000:3.0:1789760760.041995:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 739576299054 time 739576038638 state 0x1 ref 1 00000100:00100000:3.0:1789760760.042025:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 739576299054 time 739606325022 state 0x0 ref 1 00000100:00100000:3.0:1789760760.042037:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac437f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512680960:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760760.042082:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26214400, mtime: 0x6aad94f7, atime: 0x0, ctime: 0x6aad94f7, blocks: 0xc807 00010000:00010000:3.0:1789760760.042091:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2f02a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.042106:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=739s v=(0 0) 00010000:00010000:3.0:1789760760.042110:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760760.042117:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2f02a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26214400->26218495] (req 26214400->26218495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.042130:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2f02a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26214400->26218495] (req 26214400->26218495) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760760.067728:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760760.068415:0:5848:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512681216 00000100:00100000:0.0:1789760760.068425:0:5848:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512681216 00000100:00100000:3.0:1789760760.068447:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760760.068495:0:5848:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 489 00000100:00100000:0.0:1789760760.068506:0:5848:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febea89aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+8:2360:x1876699512681216:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:3.0:1789760760.068515:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760760.068534:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512681344 00000100:00100000:3.0:1789760760.068538:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512681344 00000100:00100000:3.0:1789760760.068565:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 490 00000100:00100000:0.0:1789760760.068572:0:5848:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febea89aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+8:2360:x1876699512681216:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 68us (886us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760760.068572:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febea899180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:d2c40150-e97b-40da-952b-cb9f18bf7e96+5:2360:x1876699512681344:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:0.0:1789760760.068579:0:5848:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 489 00000100:00100000:0.0:1789760760.068601:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512681088 00000100:00100000:0.0:1789760760.068604:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512681088 00000100:00100000:3.0:1789760760.068612:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febea899180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:d2c40150-e97b-40da-952b-cb9f18bf7e96+5:2360:x1876699512681344:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 41us (96us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760760.068618:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 490 00000100:00100000:0.0:1789760760.068625:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 404 00000100:00100000:0.0:1789760760.068631:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febe00c5050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:e6cf7267-9b19-462a-a263-14115df15dab+13:2360:x1876699512681088:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:0.0:1789760760.068655:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febe00c5050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:e6cf7267-9b19-462a-a263-14115df15dab+13:2360:x1876699512681088:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 23us (210us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760760.068661:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 404 00000100:00080000:2.0:1789760760.068721:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000001:00080000:0.0:1789760760.070253:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973719 is committed 00002000:00010000:3.0:1789760760.070343:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26214400 -> 26218496 00002000:00010000:3.0:1789760760.070348:0:6708:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb blocks from disk: 51207 -> 53255 00010000:00010000:3.0:1789760760.070361:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2f02a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26214400->26218495] (req 26214400->26218495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.070374:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2f02a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26214400->26218495] (req 26214400->26218495) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.070383:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2f02a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26214400->26218495] (req 26214400->26218495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.070390:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760760.070395:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2f02a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26214400->26218495] (req 26214400->26218495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.070403:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2f02a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26214400->26218495] (req 26214400->26218495) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.070422:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2f02a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26214400->26218495] (req 26214400->26218495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.070435:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2f02a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26214400->26218495] (req 26214400->26218495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760760.070511:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac437f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512680960:12345-192.168.203.56@tcp:4:dd.0 Request processed in 28472us (28747us total) trans 4294973719 rc 0/0 00000100:00100000:3.0:1789760760.070521:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6430 00000100:00100000:3.0:1789760760.070530:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 739576299054 time 739606325022 state 0x2 ref 0 00000100:00100000:2.0:1789760760.073240:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760760.073328:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512681472 00000100:00100000:3.0:1789760760.073338:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512681472 00000100:00100000:3.0:1789760760.073390:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 739576299054 time 739606325022 state 0x0 ref 1 00000100:00100000:3.0:1789760760.073406:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 739606585438 time 739606325022 state 0x1 ref 1 00000100:00100000:3.0:1789760760.073464:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 739606585438 time 739637764274 state 0x0 ref 1 00000100:00100000:3.0:1789760760.073476:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4131f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512681472:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760760.073519:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26218496, mtime: 0x6aad94f7, atime: 0x0, ctime: 0x6aad94f7, blocks: 0xd007 00010000:00010000:3.0:1789760760.073527:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2f031 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.073543:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=739s v=(0 0) 00010000:00010000:3.0:1789760760.073547:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760760.073553:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2f031 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26218496->26222591] (req 26218496->26222591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.073570:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2f031 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26218496->26222591] (req 26218496->26222591) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760760.096590:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973720 is committed 00002000:00010000:0.0:1789760760.096682:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26218496 -> 26222592 00010000:00010000:0.0:1789760760.096726:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2f031 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26218496->26222591] (req 26218496->26222591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.096749:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2f031 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26218496->26222591] (req 26218496->26222591) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.096759:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2f031 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26218496->26222591] (req 26218496->26222591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.096766:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760760.096770:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2f031 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26218496->26222591] (req 26218496->26222591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.096780:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2f031 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26218496->26222591] (req 26218496->26222591) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.096798:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2f031 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26218496->26222591] (req 26218496->26222591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.096806:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2f031 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26218496->26222591] (req 26218496->26222591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760760.096944:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4131f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512681472:12345-192.168.203.56@tcp:4:dd.0 Request processed in 23461us (23705us total) trans 4294973720 rc 0/0 00000100:00100000:0.0:1789760760.096955:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6431 00000100:00100000:0.0:1789760760.096965:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 739606585438 time 739637764274 state 0x2 ref 0 00000100:00100000:0.0:1789760760.099443:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760760.099532:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512681600 00000100:00100000:3.0:1789760760.099541:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512681600 00000100:00100000:3.0:1789760760.099590:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 739606585438 time 739637764274 state 0x0 ref 1 00000100:00100000:3.0:1789760760.099605:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 739638024690 time 739637764274 state 0x1 ref 1 00000100:00100000:3.0:1789760760.099635:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 739638024690 time 739663934897 state 0x0 ref 1 00000100:00100000:3.0:1789760760.099648:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac437fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512681600:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760760.099691:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26222592, mtime: 0x6aad94f7, atime: 0x0, ctime: 0x6aad94f7, blocks: 0xd007 00010000:00010000:3.0:1789760760.099768:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2f038 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.099783:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=739s v=(0 0) 00010000:00010000:3.0:1789760760.099787:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760760.099793:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2f038 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26222592->26226687] (req 26222592->26226687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.099806:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2f038 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26222592->26226687] (req 26222592->26226687) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760760.129139:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973721 is committed 00002000:00010000:3.0:1789760760.129324:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26222592 -> 26226688 00010000:00010000:3.0:1789760760.129340:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2f038 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26222592->26226687] (req 26222592->26226687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.129352:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2f038 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26222592->26226687] (req 26222592->26226687) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.129361:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2f038 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26222592->26226687] (req 26222592->26226687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.129369:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760760.129374:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2f038 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26222592->26226687] (req 26222592->26226687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.129389:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2f038 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26222592->26226687] (req 26222592->26226687) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.129410:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2f038 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26222592->26226687] (req 26222592->26226687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.129417:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2f038 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26222592->26226687] (req 26222592->26226687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760760.129554:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac437fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512681600:12345-192.168.203.56@tcp:4:dd.0 Request processed in 29903us (30115us total) trans 4294973721 rc 0/0 00000100:00100000:3.0:1789760760.129565:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6432 00000100:00100000:3.0:1789760760.129574:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 739638024690 time 739663934897 state 0x2 ref 0 00000100:00100000:2.0:1789760760.134448:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760760.134548:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512681728 00000100:00100000:3.0:1789760760.134558:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512681728 00000100:00100000:3.0:1789760760.134777:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 739638024690 time 739663934897 state 0x0 ref 1 00000100:00100000:3.0:1789760760.134792:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 739664195313 time 739663934897 state 0x1 ref 1 00000100:00100000:3.0:1789760760.134821:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 739664195313 time 739699120132 state 0x0 ref 1 00000100:00100000:3.0:1789760760.134834:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4130380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512681728:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760760.135071:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26226688, mtime: 0x6aad94f7, atime: 0x0, ctime: 0x6aad94f7, blocks: 0xd007 00010000:00010000:2.0:1789760760.135082:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2f03f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.135099:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=739s v=(0 0) 00010000:00010000:2.0:1789760760.135104:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760760.135110:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2f03f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26226688->26230783] (req 26226688->26230783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.135126:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2f03f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26226688->26230783] (req 26226688->26230783) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760760.171012:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26226688 -> 26230784 00010000:00010000:0.0:1789760760.171032:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2f03f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26226688->26230783] (req 26226688->26230783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.171045:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2f03f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26226688->26230783] (req 26226688->26230783) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.171054:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2f03f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26226688->26230783] (req 26226688->26230783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.171061:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760760.171066:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2f03f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26226688->26230783] (req 26226688->26230783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.171075:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2f03f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26226688->26230783] (req 26226688->26230783) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.171093:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2f03f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26226688->26230783] (req 26226688->26230783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.171102:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2f03f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26226688->26230783] (req 26226688->26230783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760760.171180:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973722 is committed 00000100:00100000:0.0:1789760760.171190:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4130380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+6:2360:x1876699512681728:12345-192.168.203.56@tcp:4:dd.0 Request processed in 36354us (36752us total) trans 4294973722 rc 0/0 00000100:00100000:0.0:1789760760.171210:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6433 00000100:00100000:0.0:1789760760.171220:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 739664195313 time 739699120132 state 0x2 ref 0 00000100:00100000:2.0:1789760760.175564:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760760.175659:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512681856 00000100:00100000:0.0:1789760760.175670:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512681856 00000100:00100000:0.0:1789760760.175774:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 739664195313 time 739699120132 state 0x0 ref 1 00000100:00100000:0.0:1789760760.175791:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 739699380548 time 739699120132 state 0x1 ref 1 00000100:00100000:0.0:1789760760.175816:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 739699380548 time 739740115425 state 0x0 ref 1 00000100:00100000:0.0:1789760760.175828:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4133b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512681856:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760760.175881:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26230784, mtime: 0x6aad94f7, atime: 0x0, ctime: 0x6aad94f7, blocks: 0xd007 00010000:00010000:0.0:1789760760.175889:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2f046 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.175905:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=739s v=(0 0) 00010000:00010000:0.0:1789760760.175910:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760760.175916:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2f046 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26230784->26234879] (req 26230784->26234879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.175929:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2f046 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26230784->26234879] (req 26230784->26234879) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.179063:0:5836:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:0.0:1789760760.216773:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973723 is committed 00002000:00010000:1.0:1789760760.216883:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26230784 -> 26234880 00010000:00010000:1.0:1789760760.216898:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2f046 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26230784->26234879] (req 26230784->26234879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.216911:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2f046 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26230784->26234879] (req 26230784->26234879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.216920:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2f046 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26230784->26234879] (req 26230784->26234879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.216931:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760760.216936:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2f046 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26230784->26234879] (req 26230784->26234879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.216943:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2f046 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26230784->26234879] (req 26230784->26234879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.216961:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2f046 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26230784->26234879] (req 26230784->26234879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.216969:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2f046 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26230784->26234879] (req 26230784->26234879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760760.217312:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4133b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512681856:12345-192.168.203.56@tcp:4:dd.0 Request processed in 41479us (41753us total) trans 4294973723 rc 0/0 00000100:00100000:1.0:1789760760.217323:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6434 00000100:00100000:1.0:1789760760.217333:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 739699380548 time 739740115425 state 0x2 ref 0 00000100:00100000:3.0:1789760760.229612:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760760.229784:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512681984 00000100:00100000:0.0:1789760760.229794:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512681984 00000100:00100000:0.0:1789760760.229849:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 739699380548 time 739740115425 state 0x0 ref 1 00000100:00100000:0.0:1789760760.229864:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 739740375841 time 739740115425 state 0x1 ref 1 00000100:00100000:0.0:1789760760.229961:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 739740375841 time 739794259751 state 0x0 ref 1 00000100:00100000:0.0:1789760760.229974:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febffb9f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512681984:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760760.230019:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26234880, mtime: 0x6aad94f7, atime: 0x0, ctime: 0x6aad94f7, blocks: 0xd007 00010000:00010000:0.0:1789760760.230033:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2f04d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.230047:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=739s v=(0 0) 00010000:00010000:0.0:1789760760.230052:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760760.230058:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2f04d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26234880->26238975] (req 26234880->26238975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.230072:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2f04d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26234880->26238975] (req 26234880->26238975) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760760.269518:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26234880 -> 26238976 00010000:00010000:1.0:1789760760.269544:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2f04d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26234880->26238975] (req 26234880->26238975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.269558:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2f04d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26234880->26238975] (req 26234880->26238975) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760760.269565:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973724 is committed 00010000:00010000:1.0:1789760760.269579:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2f04d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26234880->26238975] (req 26234880->26238975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.269595:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760760.269600:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2f04d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26234880->26238975] (req 26234880->26238975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.269609:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2f04d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26234880->26238975] (req 26234880->26238975) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.269628:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2f04d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26234880->26238975] (req 26234880->26238975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.269636:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2f04d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26234880->26238975] (req 26234880->26238975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760760.269767:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febffb9f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512681984:12345-192.168.203.56@tcp:4:dd.0 Request processed in 39789us (40158us total) trans 4294973724 rc 0/0 00000100:00100000:1.0:1789760760.269778:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6435 00000100:00100000:1.0:1789760760.269788:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 739740375841 time 739794259751 state 0x2 ref 0 00000100:00100000:0.0:1789760760.275127:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760760.275204:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512682112 00000100:00100000:2.0:1789760760.275214:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512682112 00000100:00100000:2.0:1789760760.275265:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 739740375841 time 739794259751 state 0x0 ref 1 00000100:00100000:2.0:1789760760.275352:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 739794520167 time 739794259751 state 0x1 ref 1 00000100:00100000:2.0:1789760760.275385:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 739794520167 time 739839685150 state 0x0 ref 1 00000100:00100000:2.0:1789760760.275398:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac437d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512682112:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760760.275444:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26238976, mtime: 0x6aad94f7, atime: 0x0, ctime: 0x6aad94f7, blocks: 0xd007 00010000:00010000:2.0:1789760760.275452:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2f054 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.275644:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=739s v=(0 0) 00010000:00010000:2.0:1789760760.275648:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760760.275655:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2f054 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26238976->26243071] (req 26238976->26243071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.275669:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2f054 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26238976->26243071] (req 26238976->26243071) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760760.323020:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973725 is committed 00002000:00010000:1.0:1789760760.323367:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26238976 -> 26243072 00010000:00010000:1.0:1789760760.323383:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2f054 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26238976->26243071] (req 26238976->26243071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.323401:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2f054 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26238976->26243071] (req 26238976->26243071) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.323409:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2f054 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26238976->26243071] (req 26238976->26243071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.323539:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760760.323544:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2f054 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26238976->26243071] (req 26238976->26243071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.323553:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2f054 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26238976->26243071] (req 26238976->26243071) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.323573:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2f054 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26238976->26243071] (req 26238976->26243071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.323580:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2f054 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26238976->26243071] (req 26238976->26243071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760760.323667:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac437d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512682112:12345-192.168.203.56@tcp:4:dd.0 Request processed in 48267us (48543us total) trans 4294973725 rc 0/0 00000100:00100000:1.0:1789760760.323678:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6436 00000100:00100000:1.0:1789760760.323688:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 739794520167 time 739839685150 state 0x2 ref 0 00000100:00100000:2.0:1789760760.331161:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760760.331320:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512682240 00000100:00100000:1.0:1789760760.331331:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512682240 00000100:00100000:1.0:1789760760.331437:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 739794520167 time 739839685150 state 0x0 ref 1 00000100:00100000:1.0:1789760760.331453:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 739839945566 time 739839685150 state 0x1 ref 1 00000100:00100000:1.0:1789760760.331528:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 739839945566 time 739895827283 state 0x0 ref 1 00000100:00100000:1.0:1789760760.331543:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4133800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512682240:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760760.331592:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26243072, mtime: 0x6aad94f7, atime: 0x0, ctime: 0x6aad94f7, blocks: 0xd007 00010000:00010000:1.0:1789760760.331601:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2f05b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.331642:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=739s v=(0 0) 00010000:00010000:1.0:1789760760.331647:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760760.331653:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2f05b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26243072->26247167] (req 26243072->26247167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.331666:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2f05b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26243072->26247167] (req 26243072->26247167) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760760.364820:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973726 is committed 00002000:00010000:0.0:1789760760.365045:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26243072 -> 26247168 00010000:00010000:0.0:1789760760.365061:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2f05b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26243072->26247167] (req 26243072->26247167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.365075:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2f05b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26243072->26247167] (req 26243072->26247167) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.365084:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2f05b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26243072->26247167] (req 26243072->26247167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.365096:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760760.366364:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2f05b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26243072->26247167] (req 26243072->26247167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.366386:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2f05b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26243072->26247167] (req 26243072->26247167) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.366407:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2f05b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26243072->26247167] (req 26243072->26247167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.366415:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2f05b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26243072->26247167] (req 26243072->26247167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760760.366515:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4133800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512682240:12345-192.168.203.56@tcp:4:dd.0 Request processed in 34970us (35358us total) trans 4294973726 rc 0/0 00000100:00100000:0.0:1789760760.366525:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6437 00000100:00100000:0.0:1789760760.366535:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 739839945566 time 739895827283 state 0x2 ref 0 00000100:00100000:3.0:1789760760.374313:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760760.374418:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512682368 00000100:00100000:1.0:1789760760.374431:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512682368 00000100:00100000:1.0:1789760760.374493:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 739839945566 time 739895827283 state 0x0 ref 1 00000100:00100000:1.0:1789760760.374512:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 739896087699 time 739895827283 state 0x1 ref 1 00000100:00100000:1.0:1789760760.374544:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 739896087699 time 739938844049 state 0x0 ref 1 00000100:00100000:1.0:1789760760.374561:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febffb9ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512682368:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760760.374618:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26247168, mtime: 0x6aad94f7, atime: 0x0, ctime: 0x6aad94f7, blocks: 0xd007 00010000:00010000:1.0:1789760760.374627:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2f062 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.374642:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=739s v=(0 0) 00010000:00010000:1.0:1789760760.374647:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760760.374654:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2f062 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26247168->26251263] (req 26247168->26251263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.374669:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2f062 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26247168->26251263] (req 26247168->26251263) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760760.404930:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26247168 -> 26251264 00010000:00010000:1.0:1789760760.404949:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2f062 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26247168->26251263] (req 26247168->26251263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.404961:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2f062 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26247168->26251263] (req 26247168->26251263) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.404968:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2f062 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26247168->26251263] (req 26247168->26251263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.404975:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760760.404979:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2f062 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26247168->26251263] (req 26247168->26251263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.404988:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2f062 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26247168->26251263] (req 26247168->26251263) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.405005:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2f062 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26247168->26251263] (req 26247168->26251263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.405011:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2f062 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26247168->26251263] (req 26247168->26251263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760760.405102:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febffb9ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2358:x1876699512682368:12345-192.168.203.56@tcp:4:dd.0 Request processed in 30537us (30793us total) trans 4294973727 rc 0/0 00000100:00100000:1.0:1789760760.405112:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6438 00000100:00100000:1.0:1789760760.405122:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 739896087699 time 739938844049 state 0x2 ref 0 00000001:00080000:2.0:1789760760.405860:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973727 is committed 00000100:00100000:2.0:1789760760.409146:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760760.409255:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512682496 00000100:00100000:1.0:1789760760.409265:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512682496 00000100:00100000:1.0:1789760760.409314:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 739896087699 time 739938844049 state 0x0 ref 1 00000100:00100000:1.0:1789760760.409327:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 739939104465 time 739938844049 state 0x1 ref 1 00000100:00100000:1.0:1789760760.409350:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 739939104465 time 739973650450 state 0x0 ref 1 00000100:00100000:1.0:1789760760.409361:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac28a3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512682496:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760760.409411:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26251264, mtime: 0x6aad94f7, atime: 0x0, ctime: 0x6aad94f7, blocks: 0xd007 00010000:00010000:1.0:1789760760.409421:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2f069 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.409437:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=739s v=(0 0) 00010000:00010000:1.0:1789760760.409442:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760760.409449:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2f069 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26251264->26255359] (req 26251264->26255359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.409463:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2f069 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26251264->26255359] (req 26251264->26255359) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760760.439302:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26251264 -> 26255360 00010000:00010000:0.0:1789760760.439323:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2f069 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26251264->26255359] (req 26251264->26255359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.439337:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2f069 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26251264->26255359] (req 26251264->26255359) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.439352:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2f069 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26251264->26255359] (req 26251264->26255359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.439360:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760760.439365:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2f069 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26251264->26255359] (req 26251264->26255359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.439375:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2f069 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26251264->26255359] (req 26251264->26255359) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.439397:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2f069 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26251264->26255359] (req 26251264->26255359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.439405:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2f069 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26251264->26255359] (req 26251264->26255359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760760.439550:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973728 is committed 00000100:00100000:0.0:1789760760.439610:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac28a3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512682496:12345-192.168.203.56@tcp:4:dd.0 Request processed in 30244us (30467us total) trans 4294973728 rc 0/0 00000100:00100000:0.0:1789760760.439621:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6439 00000100:00100000:0.0:1789760760.439630:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 739939104465 time 739973650450 state 0x2 ref 0 00000100:00100000:0.0:1789760760.443142:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760760.443420:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512682624 00000100:00100000:2.0:1789760760.443430:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512682624 00000100:00100000:2.0:1789760760.443483:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 739939104465 time 739973650450 state 0x0 ref 1 00000100:00100000:2.0:1789760760.443497:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 739973910866 time 739973650450 state 0x1 ref 1 00000100:00100000:2.0:1789760760.443525:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 739973910866 time 740007825059 state 0x0 ref 1 00000100:00100000:2.0:1789760760.443537:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac437df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512682624:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760760.443583:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26255360, mtime: 0x6aad94f7, atime: 0x0, ctime: 0x6aad94f7, blocks: 0xd007 00010000:00010000:2.0:1789760760.443593:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2f070 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.443609:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=740s v=(0 0) 00010000:00010000:2.0:1789760760.443614:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760760.443621:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2f070 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26255360->26259455] (req 26255360->26259455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.443636:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2f070 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26255360->26259455] (req 26255360->26259455) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760760.485635:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973729 is committed 00002000:00010000:0.0:1789760760.485753:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26255360 -> 26259456 00010000:00010000:0.0:1789760760.485769:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2f070 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26255360->26259455] (req 26255360->26259455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.485788:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2f070 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26255360->26259455] (req 26255360->26259455) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.485797:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2f070 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26255360->26259455] (req 26255360->26259455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.485804:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760760.485809:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2f070 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26255360->26259455] (req 26255360->26259455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.485819:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2f070 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26255360->26259455] (req 26255360->26259455) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.485839:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2f070 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26255360->26259455] (req 26255360->26259455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.485848:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2f070 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26255360->26259455] (req 26255360->26259455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760760.485957:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac437df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512682624:12345-192.168.203.56@tcp:4:dd.0 Request processed in 42417us (42819us total) trans 4294973729 rc 0/0 00000100:00100000:0.0:1789760760.485967:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6440 00000100:00100000:0.0:1789760760.485977:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 739973910866 time 740007825059 state 0x2 ref 0 00000100:00100000:3.0:1789760760.489211:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760760.489307:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512682752 00000100:00100000:1.0:1789760760.489316:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512682752 00000100:00100000:1.0:1789760760.489377:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 739973910866 time 740007825059 state 0x0 ref 1 00000100:00100000:1.0:1789760760.489393:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 740008085475 time 740007825059 state 0x1 ref 1 00000100:00100000:1.0:1789760760.489422:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 740008085475 time 740053721119 state 0x0 ref 1 00000100:00100000:1.0:1789760760.489435:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febffb9c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512682752:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760760.489486:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26259456, mtime: 0x6aad94f7, atime: 0x0, ctime: 0x6aad94f7, blocks: 0xd007 00010000:00010000:1.0:1789760760.489495:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2f077 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.489512:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=740s v=(0 0) 00010000:00010000:1.0:1789760760.489517:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760760.489524:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2f077 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26259456->26263551] (req 26259456->26263551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.489539:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2f077 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26259456->26263551] (req 26259456->26263551) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760760.525790:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26259456 -> 26263552 00002000:00010000:1.0:1789760760.525799:0:13063:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760759 -> 1789760760 00002000:00010000:1.0:1789760760.525802:0:13063:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760759 -> 1789760760 00010000:00010000:1.0:1789760760.525817:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2f077 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26259456->26263551] (req 26259456->26263551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.525832:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2f077 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26259456->26263551] (req 26259456->26263551) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.525843:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2f077 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26259456->26263551] (req 26259456->26263551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.525851:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760760.525856:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2f077 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26259456->26263551] (req 26259456->26263551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.525930:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2f077 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26259456->26263551] (req 26259456->26263551) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.525951:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2f077 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26259456->26263551] (req 26259456->26263551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.525965:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2f077 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26259456->26263551] (req 26259456->26263551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760760.526058:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febffb9c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2359:x1876699512682752:12345-192.168.203.56@tcp:4:dd.0 Request processed in 36620us (36863us total) trans 4294973730 rc 0/0 00000100:00100000:1.0:1789760760.526070:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6441 00000100:00100000:1.0:1789760760.526079:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 740008085475 time 740053721119 state 0x2 ref 0 00000001:00080000:0.0:1789760760.527370:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973730 is committed 00000100:00100000:1.0:1789760760.531361:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760760.531566:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512682880 00000100:00100000:0.0:1789760760.531576:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512682880 00000100:00100000:0.0:1789760760.531723:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 740008085475 time 740053721119 state 0x0 ref 1 00000100:00100000:0.0:1789760760.531742:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 740053981535 time 740053721119 state 0x1 ref 1 00000100:00100000:0.0:1789760760.531817:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 740053981535 time 740096115941 state 0x0 ref 1 00000100:00100000:0.0:1789760760.531831:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac403ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512682880:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760760.531890:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26263552, mtime: 0x6aad94f8, atime: 0x0, ctime: 0x6aad94f8, blocks: 0xd007 00010000:00010000:0.0:1789760760.531900:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2f07e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.531915:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=740s v=(0 0) 00010000:00010000:0.0:1789760760.531919:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760760.531926:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2f07e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26263552->26267647] (req 26263552->26267647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.531940:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2f07e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26263552->26267647] (req 26263552->26267647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760760.578544:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973731 is committed 00002000:00010000:0.0:1789760760.578657:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26263552 -> 26267648 00010000:00010000:0.0:1789760760.578673:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2f07e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26263552->26267647] (req 26263552->26267647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.578685:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2f07e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26263552->26267647] (req 26263552->26267647) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.578719:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2f07e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26263552->26267647] (req 26263552->26267647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.578727:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760760.578733:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2f07e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26263552->26267647] (req 26263552->26267647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.578742:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2f07e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26263552->26267647] (req 26263552->26267647) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.578765:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2f07e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26263552->26267647] (req 26263552->26267647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.578776:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2f07e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26263552->26267647] (req 26263552->26267647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760760.578877:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac403ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512682880:12345-192.168.203.56@tcp:4:dd.0 Request processed in 47036us (47513us total) trans 4294973731 rc 0/0 00000100:00100000:0.0:1789760760.578887:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6442 00000100:00100000:0.0:1789760760.578896:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 740053981535 time 740096115941 state 0x2 ref 0 00000100:00100000:2.0:1789760760.586293:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760760.586479:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512683008 00000100:00100000:0.0:1789760760.586489:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512683008 00000100:00100000:0.0:1789760760.586542:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 740053981535 time 740096115941 state 0x0 ref 1 00000100:00100000:0.0:1789760760.586557:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 740096376357 time 740096115941 state 0x1 ref 1 00000100:00100000:0.0:1789760760.586586:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 740096376357 time 740150885832 state 0x0 ref 1 00000100:00100000:0.0:1789760760.586598:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5305f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512683008:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760760.586640:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26267648, mtime: 0x6aad94f8, atime: 0x0, ctime: 0x6aad94f8, blocks: 0xd007 00010000:00010000:0.0:1789760760.586648:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2f085 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.586662:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=740s v=(0 0) 00010000:00010000:0.0:1789760760.586666:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760760.586672:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2f085 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26267648->26271743] (req 26267648->26271743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.586685:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2f085 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26267648->26271743] (req 26267648->26271743) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760760.620542:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973732 is committed 00002000:00010000:2.0:1789760760.620642:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26267648 -> 26271744 00010000:00010000:2.0:1789760760.620658:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2f085 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26267648->26271743] (req 26267648->26271743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.620670:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2f085 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26267648->26271743] (req 26267648->26271743) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.620679:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2f085 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26267648->26271743] (req 26267648->26271743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.620686:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760760.620691:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2f085 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26267648->26271743] (req 26267648->26271743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.620726:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2f085 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26267648->26271743] (req 26267648->26271743) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.620756:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2f085 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26267648->26271743] (req 26267648->26271743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.620765:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2f085 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26267648->26271743] (req 26267648->26271743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760760.620849:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5305f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512683008:12345-192.168.203.56@tcp:4:dd.0 Request processed in 34248us (34559us total) trans 4294973732 rc 0/0 00000100:00100000:2.0:1789760760.620860:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6443 00000100:00100000:2.0:1789760760.620987:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 740096376357 time 740150885832 state 0x2 ref 0 00000100:00100000:3.0:1789760760.625083:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760760.625252:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512683136 00000100:00100000:1.0:1789760760.625263:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512683136 00000100:00100000:1.0:1789760760.625367:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 740096376357 time 740150885832 state 0x0 ref 1 00000100:00100000:1.0:1789760760.625385:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 740151146248 time 740150885832 state 0x1 ref 1 00000100:00100000:1.0:1789760760.625412:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 740151146248 time 740189712229 state 0x0 ref 1 00000100:00100000:1.0:1789760760.625437:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febffb9c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512683136:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760760.625504:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26271744, mtime: 0x6aad94f8, atime: 0x0, ctime: 0x6aad94f8, blocks: 0xd007 00010000:00010000:1.0:1789760760.625514:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2f08c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.625532:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=740s v=(0 0) 00010000:00010000:1.0:1789760760.625538:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760760.625544:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2f08c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26271744->26275839] (req 26271744->26275839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.625558:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2f08c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26271744->26275839] (req 26271744->26275839) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760760.681021:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973733 is committed 00002000:00010000:1.0:1789760760.681153:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26271744 -> 26275840 00010000:00010000:1.0:1789760760.681168:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2f08c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26271744->26275839] (req 26271744->26275839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.681181:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2f08c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26271744->26275839] (req 26271744->26275839) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.681200:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2f08c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26271744->26275839] (req 26271744->26275839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.681208:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760760.681213:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2f08c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26271744->26275839] (req 26271744->26275839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.681223:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2f08c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26271744->26275839] (req 26271744->26275839) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.681241:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2f08c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26271744->26275839] (req 26271744->26275839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.681249:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2f08c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26271744->26275839] (req 26271744->26275839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760760.681560:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febffb9c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512683136:12345-192.168.203.56@tcp:4:dd.0 Request processed in 56132us (56481us total) trans 4294973733 rc 0/0 00000100:00100000:1.0:1789760760.681571:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6444 00000100:00100000:1.0:1789760760.681580:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 740151146248 time 740189712229 state 0x2 ref 0 00000100:00100000:1.0:1789760760.688202:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760760.688292:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512683264 00000100:00100000:0.0:1789760760.688302:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512683264 00000100:00100000:0.0:1789760760.688358:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 740151146248 time 740189712229 state 0x0 ref 1 00000100:00100000:0.0:1789760760.688373:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 740189972645 time 740189712229 state 0x1 ref 1 00000100:00100000:0.0:1789760760.688402:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 740189972645 time 740252702095 state 0x0 ref 1 00000100:00100000:0.0:1789760760.688416:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac403a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512683264:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760760.688470:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26275840, mtime: 0x6aad94f8, atime: 0x0, ctime: 0x6aad94f8, blocks: 0xd007 00010000:00010000:0.0:1789760760.688479:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2f093 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.688496:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=740s v=(0 0) 00010000:00010000:0.0:1789760760.688500:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760760.688518:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2f093 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26275840->26279935] (req 26275840->26279935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.688534:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2f093 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26275840->26279935] (req 26275840->26279935) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760760.751002:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973734 is committed 00002000:00010000:1.0:1789760760.751184:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26275840 -> 26279936 00010000:00010000:1.0:1789760760.751210:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2f093 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26275840->26279935] (req 26275840->26279935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.751224:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2f093 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26275840->26279935] (req 26275840->26279935) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.751234:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2f093 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26275840->26279935] (req 26275840->26279935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.751242:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760760.751247:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2f093 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26275840->26279935] (req 26275840->26279935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.751257:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2f093 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26275840->26279935] (req 26275840->26279935) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.751274:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2f093 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26275840->26279935] (req 26275840->26279935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.751361:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2f093 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26275840->26279935] (req 26275840->26279935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760760.751484:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac403a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512683264:12345-192.168.203.56@tcp:4:dd.0 Request processed in 63065us (63296us total) trans 4294973734 rc 0/0 00000100:00100000:1.0:1789760760.751494:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6445 00000100:00100000:1.0:1789760760.751504:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 740189972645 time 740252702095 state 0x2 ref 0 00000100:00100000:0.0:1789760760.755732:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760760.755799:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512683392 00000100:00100000:2.0:1789760760.755810:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512683392 00000100:00100000:2.0:1789760760.755878:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 740189972645 time 740252702095 state 0x0 ref 1 00000100:00100000:2.0:1789760760.755892:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 740252962511 time 740252702095 state 0x1 ref 1 00000100:00100000:2.0:1789760760.755922:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 740252962511 time 740320222423 state 0x0 ref 1 00000100:00100000:2.0:1789760760.755935:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac437e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512683392:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760760.755984:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26279936, mtime: 0x6aad94f8, atime: 0x0, ctime: 0x6aad94f8, blocks: 0xd007 00010000:00010000:2.0:1789760760.755993:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2f09a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.756008:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=740s v=(0 0) 00010000:00010000:2.0:1789760760.756013:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760760.756018:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2f09a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26279936->26284031] (req 26279936->26284031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.756031:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2f09a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26279936->26284031] (req 26279936->26284031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760760.795885:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973735 is committed 00002000:00010000:3.0:1789760760.796027:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26279936 -> 26284032 00010000:00010000:3.0:1789760760.796041:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2f09a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26279936->26284031] (req 26279936->26284031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.796053:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2f09a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26279936->26284031] (req 26279936->26284031) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.796062:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2f09a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26279936->26284031] (req 26279936->26284031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.796076:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760760.796081:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2f09a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26279936->26284031] (req 26279936->26284031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.796090:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2f09a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26279936->26284031] (req 26279936->26284031) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.796109:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2f09a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26279936->26284031] (req 26279936->26284031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.796117:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2f09a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26279936->26284031] (req 26279936->26284031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760760.796209:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac437e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512683392:12345-192.168.203.56@tcp:4:dd.0 Request processed in 40272us (40482us total) trans 4294973735 rc 0/0 00000100:00100000:3.0:1789760760.796218:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6446 00000100:00100000:3.0:1789760760.796227:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 740252962511 time 740320222423 state 0x2 ref 0 00000100:00100000:3.0:1789760760.801126:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760760.801326:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512683520 00000100:00100000:1.0:1789760760.801336:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512683520 00000100:00100000:1.0:1789760760.801390:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 740252962511 time 740320222423 state 0x0 ref 1 00000100:00100000:1.0:1789760760.801405:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 740320482839 time 740320222423 state 0x1 ref 1 00000100:00100000:1.0:1789760760.801434:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 740320482839 time 740365734002 state 0x0 ref 1 00000100:00100000:1.0:1789760760.801446:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febffb9f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512683520:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760760.801491:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26284032, mtime: 0x6aad94f8, atime: 0x0, ctime: 0x6aad94f8, blocks: 0xd007 00010000:00010000:1.0:1789760760.801500:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2f0a1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.801515:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=740s v=(0 0) 00010000:00010000:1.0:1789760760.801519:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760760.801525:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2f0a1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26284032->26288127] (req 26284032->26288127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.801545:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2f0a1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26284032->26288127] (req 26284032->26288127) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760760.827449:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973736 is committed 00002000:00010000:2.0:1789760760.827514:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26284032 -> 26288128 00010000:00010000:2.0:1789760760.827530:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2f0a1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26284032->26288127] (req 26284032->26288127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.827545:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2f0a1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26284032->26288127] (req 26284032->26288127) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.827556:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2f0a1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26284032->26288127] (req 26284032->26288127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.827564:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760760.827569:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2f0a1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26284032->26288127] (req 26284032->26288127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.827579:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2f0a1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26284032->26288127] (req 26284032->26288127) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.827598:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2f0a1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26284032->26288127] (req 26284032->26288127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.827606:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2f0a1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26284032->26288127] (req 26284032->26288127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760760.827860:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febffb9f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512683520:12345-192.168.203.56@tcp:4:dd.0 Request processed in 26410us (26739us total) trans 4294973736 rc 0/0 00000100:00100000:2.0:1789760760.827879:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6447 00000100:00100000:2.0:1789760760.827889:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 740320482839 time 740365734002 state 0x2 ref 0 00000100:00100000:1.0:1789760760.845927:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760760.849422:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512683648 00000100:00100000:0.0:1789760760.849440:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512683648 00000100:00100000:0.0:1789760760.849501:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 740320482839 time 740365734002 state 0x0 ref 1 00000100:00100000:0.0:1789760760.849517:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 740365994418 time 740365734002 state 0x1 ref 1 00000100:00100000:0.0:1789760760.849620:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 740365994418 time 740413918914 state 0x0 ref 1 00000100:00100000:0.0:1789760760.849635:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4039c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512683648:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760760.849686:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26288128, mtime: 0x6aad94f8, atime: 0x0, ctime: 0x6aad94f8, blocks: 0xd007 00010000:00010000:0.0:1789760760.849723:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2f0a8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.849746:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=740s v=(0 0) 00010000:00010000:0.0:1789760760.849751:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760760.849757:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2f0a8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26288128->26292223] (req 26288128->26292223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.849771:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2f0a8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26288128->26292223] (req 26288128->26292223) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760760.907027:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973737 is committed 00002000:00010000:0.0:1789760760.907320:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26288128 -> 26292224 00010000:00010000:0.0:1789760760.907335:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2f0a8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26288128->26292223] (req 26288128->26292223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.907348:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2f0a8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26288128->26292223] (req 26288128->26292223) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.907357:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2f0a8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26288128->26292223] (req 26288128->26292223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.907372:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760760.907378:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2f0a8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26288128->26292223] (req 26288128->26292223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.907387:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2f0a8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26288128->26292223] (req 26288128->26292223) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.907408:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2f0a8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26288128->26292223] (req 26288128->26292223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.907416:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2f0a8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26288128->26292223] (req 26288128->26292223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760760.907504:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4039c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512683648:12345-192.168.203.56@tcp:4:dd.0 Request processed in 57868us (61582us total) trans 4294973737 rc 0/0 00000100:00100000:0.0:1789760760.907515:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6448 00000100:00100000:0.0:1789760760.907524:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 740365994418 time 740413918914 state 0x2 ref 0 00000100:00100000:0.0:1789760760.916166:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760760.916236:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512683776 00000100:00100000:3.0:1789760760.916246:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512683776 00000100:00100000:3.0:1789760760.916324:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 740365994418 time 740413918914 state 0x0 ref 1 00000100:00100000:3.0:1789760760.916340:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 740414179330 time 740413918914 state 0x1 ref 1 00000100:00100000:3.0:1789760760.916374:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 740414179330 time 740480673637 state 0x0 ref 1 00000100:00100000:3.0:1789760760.916387:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febff932a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512683776:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760760.916433:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26292224, mtime: 0x6aad94f8, atime: 0x0, ctime: 0x6aad94f8, blocks: 0xd007 00010000:00010000:3.0:1789760760.916443:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2f0af lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.916459:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=740s v=(0 0) 00010000:00010000:3.0:1789760760.916464:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760760.916470:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2f0af lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26292224->26296319] (req 26292224->26296319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.916483:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2f0af lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26292224->26296319] (req 26292224->26296319) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760760.973689:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973738 is committed 00002000:00010000:2.0:1789760760.973807:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26292224 -> 26296320 00010000:00010000:2.0:1789760760.973822:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2f0af lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26292224->26296319] (req 26292224->26296319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.973835:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2f0af lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26292224->26296319] (req 26292224->26296319) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.973844:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2f0af lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26292224->26296319] (req 26292224->26296319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.973850:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760760.973854:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2f0af lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26292224->26296319] (req 26292224->26296319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.973871:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2f0af lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26292224->26296319] (req 26292224->26296319) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.973889:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2f0af lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26292224->26296319] (req 26292224->26296319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.973901:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2f0af lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26292224->26296319] (req 26292224->26296319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760760.973998:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febff932a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512683776:12345-192.168.203.56@tcp:4:dd.0 Request processed in 57608us (57833us total) trans 4294973738 rc 0/0 00000100:00100000:2.0:1789760760.974007:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6449 00000100:00100000:2.0:1789760760.974017:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 740414179330 time 740480673637 state 0x2 ref 0 00000100:00100000:3.0:1789760760.977287:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760760.979023:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512683904 00000100:00100000:2.0:1789760760.979036:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512683904 00000100:00100000:3.0:1789760760.979267:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 740414179330 time 740480673637 state 0x0 ref 1 00000100:00100000:3.0:1789760760.979288:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 740480934053 time 740480673637 state 0x1 ref 1 00000100:00100000:3.0:1789760760.979321:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 740480934053 time 740543620911 state 0x0 ref 1 00000100:00100000:3.0:1789760760.979334:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febea651500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512683904:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760760.979382:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26296320, mtime: 0x6aad94f8, atime: 0x0, ctime: 0x6aad94f8, blocks: 0xd007 00010000:00010000:3.0:1789760760.979392:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2f0b6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.979413:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=740s v=(0 0) 00010000:00010000:3.0:1789760760.979427:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760760.979436:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2f0b6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26296320->26300415] (req 26296320->26300415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.979450:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2f0b6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26296320->26300415] (req 26296320->26300415) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760761.033942:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973739 is committed 00002000:00010000:0.0:1789760761.034109:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26296320 -> 26300416 00010000:00010000:0.0:1789760761.034125:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2f0b6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26296320->26300415] (req 26296320->26300415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760761.034137:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2f0b6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26296320->26300415] (req 26296320->26300415) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760761.034146:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2f0b6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26296320->26300415] (req 26296320->26300415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760761.034153:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760761.034158:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2f0b6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26296320->26300415] (req 26296320->26300415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760761.034167:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2f0b6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26296320->26300415] (req 26296320->26300415) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760761.034200:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2f0b6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26296320->26300415] (req 26296320->26300415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760761.034208:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2f0b6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26296320->26300415] (req 26296320->26300415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760761.034307:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febea651500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512683904:12345-192.168.203.56@tcp:4:dd.0 Request processed in 54970us (57022us total) trans 4294973739 rc 0/0 00000100:00100000:0.0:1789760761.034318:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6450 00000100:00100000:0.0:1789760761.034327:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 740480934053 time 740543620911 state 0x2 ref 0 00000100:00100000:0.0:1789760761.039238:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760761.039333:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512684032 00000100:00100000:2.0:1789760761.039344:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512684032 00000100:00100000:2.0:1789760761.039413:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 740480934053 time 740543620911 state 0x0 ref 1 00000100:00100000:2.0:1789760761.039428:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 740543881327 time 740543620911 state 0x1 ref 1 00000100:00100000:2.0:1789760761.039457:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 740543881327 time 740603756458 state 0x0 ref 1 00000100:00100000:2.0:1789760761.039469:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febff930e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512684032:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760761.039513:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26300416, mtime: 0x6aad94f8, atime: 0x0, ctime: 0x6aad94f8, blocks: 0xd007 00010000:00010000:2.0:1789760761.039523:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2f0bd lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.039537:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=740s v=(0 0) 00010000:00010000:2.0:1789760761.039542:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760761.039549:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2f0bd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26300416->26304511] (req 26300416->26304511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.039562:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2f0bd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26300416->26304511] (req 26300416->26304511) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1789760761.040171:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00010000:2.0:1789760761.071672:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26300416 -> 26304512 00010000:00010000:2.0:1789760761.071692:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2f0bd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26300416->26304511] (req 26300416->26304511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.071727:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2f0bd lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26300416->26304511] (req 26300416->26304511) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.071736:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2f0bd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26300416->26304511] (req 26300416->26304511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.071743:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760761.071748:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2f0bd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26300416->26304511] (req 26300416->26304511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.071757:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2f0bd lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26300416->26304511] (req 26300416->26304511) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.071776:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2f0bd lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26300416->26304511] (req 26300416->26304511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.071784:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2f0bd lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26300416->26304511] (req 26300416->26304511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760761.071878:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febff930e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2359:x1876699512684032:12345-192.168.203.56@tcp:4:dd.0 Request processed in 32406us (32643us total) trans 4294973740 rc 0/0 00000100:00100000:2.0:1789760761.071888:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6451 00000100:00100000:2.0:1789760761.071897:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 740543881327 time 740603756458 state 0x2 ref 0 00000001:00080000:0.0:1789760761.072031:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973740 is committed 00000100:00100000:1.0:1789760761.078058:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760761.078220:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512684160 00000100:00100000:2.0:1789760761.078230:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512684160 00000100:00100000:2.0:1789760761.078281:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 740543881327 time 740603756458 state 0x0 ref 1 00000100:00100000:2.0:1789760761.078296:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 740604016874 time 740603756458 state 0x1 ref 1 00000100:00100000:2.0:1789760761.078323:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 740604016874 time 740642623039 state 0x0 ref 1 00000100:00100000:2.0:1789760761.078335:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4038e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512684160:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760761.078380:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26304512, mtime: 0x6aad94f8, atime: 0x0, ctime: 0x6aad94f8, blocks: 0xd007 00010000:00010000:2.0:1789760761.078388:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2f0c4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.078404:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=740s v=(0 0) 00010000:00010000:2.0:1789760761.078408:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760761.078414:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2f0c4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26304512->26308607] (req 26304512->26308607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.078428:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2f0c4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26304512->26308607] (req 26304512->26308607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760761.111350:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26304512 -> 26308608 00010000:00010000:3.0:1789760761.111371:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2f0c4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26304512->26308607] (req 26304512->26308607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.111383:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2f0c4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26304512->26308607] (req 26304512->26308607) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.111392:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2f0c4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26304512->26308607] (req 26304512->26308607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.111399:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760761.111403:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2f0c4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26304512->26308607] (req 26304512->26308607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.111412:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2f0c4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26304512->26308607] (req 26304512->26308607) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.111430:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2f0c4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26304512->26308607] (req 26304512->26308607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.111438:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2f0c4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26304512->26308607] (req 26304512->26308607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760761.111530:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4038e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2358:x1876699512684160:12345-192.168.203.56@tcp:4:dd.0 Request processed in 33191us (33475us total) trans 4294973741 rc 0/0 00000100:00100000:3.0:1789760761.111541:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6452 00000100:00100000:3.0:1789760761.111551:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 740604016874 time 740642623039 state 0x2 ref 0 00000001:00080000:0.0:1789760761.114231:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973741 is committed 00000100:00100000:1.0:1789760761.114461:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760761.114554:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512684288 00000100:00100000:0.0:1789760761.114564:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512684288 00000100:00100000:0.0:1789760761.114685:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 740604016874 time 740642623039 state 0x0 ref 1 00000100:00100000:0.0:1789760761.114762:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 740642883455 time 740642623039 state 0x1 ref 1 00000100:00100000:0.0:1789760761.114824:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 740642883455 time 740679123255 state 0x0 ref 1 00000100:00100000:0.0:1789760761.114838:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac403b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512684288:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760761.114890:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26308608, mtime: 0x6aad94f8, atime: 0x0, ctime: 0x6aad94f8, blocks: 0xd007 00010000:00010000:0.0:1789760761.114900:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2f0cb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760761.114915:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=740s v=(0 0) 00010000:00010000:0.0:1789760761.114919:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760761.114925:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2f0cb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26308608->26312703] (req 26308608->26312703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760761.114937:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2f0cb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26308608->26312703] (req 26308608->26312703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760761.190528:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973742 is committed 00002000:00010000:3.0:1789760761.190635:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26308608 -> 26312704 00010000:00010000:3.0:1789760761.190650:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2f0cb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26308608->26312703] (req 26308608->26312703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.190663:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2f0cb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26308608->26312703] (req 26308608->26312703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.190672:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2f0cb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26308608->26312703] (req 26308608->26312703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.190679:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760761.190683:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2f0cb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26308608->26312703] (req 26308608->26312703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.190691:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2f0cb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26308608->26312703] (req 26308608->26312703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.190745:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2f0cb lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26308608->26312703] (req 26308608->26312703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.190753:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2f0cb lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26308608->26312703] (req 26308608->26312703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760761.190833:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac403b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512684288:12345-192.168.203.56@tcp:4:dd.0 Request processed in 75992us (76371us total) trans 4294973742 rc 0/0 00000100:00100000:3.0:1789760761.190843:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6453 00000100:00100000:3.0:1789760761.190853:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 740642883455 time 740679123255 state 0x2 ref 0 00000100:00100000:3.0:1789760761.193640:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760761.193911:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512684416 00000100:00100000:2.0:1789760761.193920:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512684416 00000100:00100000:2.0:1789760761.193970:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 740642883455 time 740679123255 state 0x0 ref 1 00000100:00100000:2.0:1789760761.193985:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 740679383671 time 740679123255 state 0x1 ref 1 00000100:00100000:2.0:1789760761.194017:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 740679383671 time 740758316089 state 0x0 ref 1 00000100:00100000:2.0:1789760761.194028:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febea650380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512684416:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760761.194073:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26312704, mtime: 0x6aad94f8, atime: 0x0, ctime: 0x6aad94f8, blocks: 0xd007 00010000:00010000:2.0:1789760761.194081:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2f0d2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.194097:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=740s v=(0 0) 00010000:00010000:2.0:1789760761.194102:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760761.194108:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2f0d2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26312704->26316799] (req 26312704->26316799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.194122:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2f0d2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26312704->26316799] (req 26312704->26316799) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.203010:0:5835:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:2.0:1789760761.217305:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973743 is committed 00002000:00010000:2.0:1789760761.217363:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26312704 -> 26316800 00010000:00010000:2.0:1789760761.217378:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2f0d2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26312704->26316799] (req 26312704->26316799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.217392:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2f0d2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26312704->26316799] (req 26312704->26316799) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.217401:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2f0d2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26312704->26316799] (req 26312704->26316799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.217408:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760761.217413:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2f0d2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26312704->26316799] (req 26312704->26316799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.217422:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2f0d2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26312704->26316799] (req 26312704->26316799) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.217441:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2f0d2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26312704->26316799] (req 26312704->26316799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.217449:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2f0d2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26312704->26316799] (req 26312704->26316799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760761.217535:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febea650380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512684416:12345-192.168.203.56@tcp:4:dd.0 Request processed in 23504us (23899us total) trans 4294973743 rc 0/0 00000100:00100000:2.0:1789760761.217545:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6454 00000100:00100000:2.0:1789760761.217555:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 740679383671 time 740758316089 state 0x2 ref 0 00000100:00100000:2.0:1789760761.220805:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760761.221055:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512684544 00000100:00100000:3.0:1789760761.221066:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512684544 00000100:00100000:3.0:1789760761.221117:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 740679383671 time 740758316089 state 0x0 ref 1 00000100:00100000:3.0:1789760761.221131:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 740758576505 time 740758316089 state 0x1 ref 1 00000100:00100000:3.0:1789760761.221159:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 740758576505 time 740785459722 state 0x0 ref 1 00000100:00100000:3.0:1789760761.221171:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5307480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512684544:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760761.221228:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26316800, mtime: 0x6aad94f8, atime: 0x0, ctime: 0x6aad94f8, blocks: 0xd007 00010000:00010000:3.0:1789760761.221236:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2f0d9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.221255:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=740s v=(0 0) 00010000:00010000:3.0:1789760761.221260:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760761.221266:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2f0d9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26316800->26320895] (req 26316800->26320895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.221280:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2f0d9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26316800->26320895] (req 26316800->26320895) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760761.255832:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26316800 -> 26320896 00010000:00010000:1.0:1789760761.255853:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2f0d9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26316800->26320895] (req 26316800->26320895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.255875:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2f0d9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26316800->26320895] (req 26316800->26320895) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.255885:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2f0d9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26316800->26320895] (req 26316800->26320895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.255892:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760761.255898:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2f0d9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26316800->26320895] (req 26316800->26320895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.255907:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2f0d9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26316800->26320895] (req 26316800->26320895) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.255932:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2f0d9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26316800->26320895] (req 26316800->26320895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.255940:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2f0d9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26316800->26320895] (req 26316800->26320895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760761.256028:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5307480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2361:x1876699512684544:12345-192.168.203.56@tcp:4:dd.0 Request processed in 34854us (35227us total) trans 4294973744 rc 0/0 00000100:00100000:1.0:1789760761.256046:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6455 00000100:00100000:1.0:1789760761.256057:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 740758576505 time 740785459722 state 0x2 ref 0 00000001:00080000:3.0:1789760761.258929:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973744 is committed 00000100:00100000:1.0:1789760761.259312:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760761.259381:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512684672 00000100:00100000:3.0:1789760761.259391:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512684672 00000100:00100000:3.0:1789760761.259444:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 740758576505 time 740785459722 state 0x0 ref 1 00000100:00100000:3.0:1789760761.259459:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 740785720138 time 740785459722 state 0x1 ref 1 00000100:00100000:3.0:1789760761.259486:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 740785720138 time 740823786134 state 0x0 ref 1 00000100:00100000:3.0:1789760761.259497:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4039f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512684672:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760761.259551:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26320896, mtime: 0x6aad94f8, atime: 0x0, ctime: 0x6aad94f8, blocks: 0xd007 00010000:00010000:3.0:1789760761.259560:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2f0e0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.259575:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=740s v=(0 0) 00010000:00010000:3.0:1789760761.259587:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760761.259594:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2f0e0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26320896->26324991] (req 26320896->26324991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.259608:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2f0e0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26320896->26324991] (req 26320896->26324991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760761.302500:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973745 is committed 00002000:00010000:0.0:1789760761.302738:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26320896 -> 26324992 00010000:00010000:0.0:1789760761.302753:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2f0e0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26320896->26324991] (req 26320896->26324991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760761.302767:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2f0e0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26320896->26324991] (req 26320896->26324991) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760761.302927:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2f0e0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26320896->26324991] (req 26320896->26324991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760761.302936:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760761.302940:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2f0e0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26320896->26324991] (req 26320896->26324991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760761.302949:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2f0e0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26320896->26324991] (req 26320896->26324991) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760761.302973:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2f0e0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26320896->26324991] (req 26320896->26324991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760761.302981:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2f0e0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26320896->26324991] (req 26320896->26324991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760761.303113:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4039f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512684672:12345-192.168.203.56@tcp:4:dd.0 Request processed in 43613us (43800us total) trans 4294973745 rc 0/0 00000100:00100000:0.0:1789760761.303123:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6456 00000100:00100000:0.0:1789760761.303132:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 740785720138 time 740823786134 state 0x2 ref 0 00000100:00100000:3.0:1789760761.306464:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760761.306558:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512684800 00000100:00100000:2.0:1789760761.306568:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512684800 00000100:00100000:2.0:1789760761.306684:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 740785720138 time 740823786134 state 0x0 ref 1 00000100:00100000:2.0:1789760761.306861:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 740824046550 time 740823786134 state 0x1 ref 1 00000100:00100000:2.0:1789760761.306953:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 740824046550 time 740871252227 state 0x0 ref 1 00000100:00100000:2.0:1789760761.306967:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2747b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512684800:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760761.307012:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26324992, mtime: 0x6aad94f8, atime: 0x0, ctime: 0x6aad94f8, blocks: 0xd007 00010000:00010000:2.0:1789760761.307023:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2f0e7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.307038:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=740s v=(0 0) 00010000:00010000:2.0:1789760761.307044:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760761.307051:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2f0e7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26324992->26329087] (req 26324992->26329087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.307064:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2f0e7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26324992->26329087] (req 26324992->26329087) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760761.336924:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973746 is committed 00002000:00010000:0.0:1789760761.337011:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26324992 -> 26329088 00010000:00010000:0.0:1789760761.337027:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2f0e7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26324992->26329087] (req 26324992->26329087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760761.337040:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2f0e7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26324992->26329087] (req 26324992->26329087) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760761.337048:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2f0e7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26324992->26329087] (req 26324992->26329087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760761.337056:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760761.337061:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2f0e7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26324992->26329087] (req 26324992->26329087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760761.337078:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2f0e7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26324992->26329087] (req 26324992->26329087) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760761.337098:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2f0e7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26324992->26329087] (req 26324992->26329087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760761.337106:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2f0e7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26324992->26329087] (req 26324992->26329087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760761.337242:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2747b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512684800:12345-192.168.203.56@tcp:4:dd.0 Request processed in 30272us (30781us total) trans 4294973746 rc 0/0 00000100:00100000:0.0:1789760761.337252:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6457 00000100:00100000:0.0:1789760761.337262:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 740824046550 time 740871252227 state 0x2 ref 0 00000100:00100000:0.0:1789760761.344179:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760761.344267:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512684928 00000100:00100000:1.0:1789760761.344276:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512684928 00000100:00100000:1.0:1789760761.344328:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 740824046550 time 740871252227 state 0x0 ref 1 00000100:00100000:1.0:1789760761.344343:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 740871512643 time 740871252227 state 0x1 ref 1 00000100:00100000:1.0:1789760761.344372:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 740871512643 time 740908672302 state 0x0 ref 1 00000100:00100000:1.0:1789760761.344386:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac38b6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512684928:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760761.344431:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26329088, mtime: 0x6aad94f8, atime: 0x0, ctime: 0x6aad94f8, blocks: 0xd007 00010000:00010000:1.0:1789760761.344442:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2f0ee lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.344457:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=740s v=(0 0) 00010000:00010000:1.0:1789760761.344461:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760761.344467:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2f0ee lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26329088->26333183] (req 26329088->26333183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.344481:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2f0ee lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26329088->26333183] (req 26329088->26333183) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760761.384256:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26329088 -> 26333184 00010000:00010000:3.0:1789760761.384275:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2f0ee lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26329088->26333183] (req 26329088->26333183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.384287:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2f0ee lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26329088->26333183] (req 26329088->26333183) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.384297:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2f0ee lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26329088->26333183] (req 26329088->26333183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.384304:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760761.384309:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2f0ee lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26329088->26333183] (req 26329088->26333183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.384318:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2f0ee lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26329088->26333183] (req 26329088->26333183) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.384338:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2f0ee lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26329088->26333183] (req 26329088->26333183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.384347:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2f0ee lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26329088->26333183] (req 26329088->26333183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760761.384447:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac38b6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2359:x1876699512684928:12345-192.168.203.56@tcp:4:dd.0 Request processed in 40051us (40265us total) trans 4294973747 rc 0/0 00000100:00100000:3.0:1789760761.384458:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6458 00000100:00100000:3.0:1789760761.384468:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 740871512643 time 740908672302 state 0x2 ref 0 00000001:00080000:0.0:1789760761.385018:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973747 is committed 00000100:00100000:1.0:1789760761.388564:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760761.389149:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512685056 00000100:00100000:0.0:1789760761.389160:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512685056 00000100:00100000:2.0:1789760761.389557:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 740871512643 time 740908672302 state 0x0 ref 1 00000100:00100000:2.0:1789760761.389573:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 740908932718 time 740908672302 state 0x1 ref 1 00000100:00100000:2.0:1789760761.389635:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 740908932718 time 740953934161 state 0x0 ref 1 00000100:00100000:2.0:1789760761.389651:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4038380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512685056:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760761.389723:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26333184, mtime: 0x6aad94f8, atime: 0x0, ctime: 0x6aad94f8, blocks: 0xd007 00010000:00010000:2.0:1789760761.389731:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2f0f5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.389746:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=740s v=(0 0) 00010000:00010000:2.0:1789760761.389751:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760761.389757:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2f0f5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26333184->26337279] (req 26333184->26337279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.389770:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2f0f5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26333184->26337279] (req 26333184->26337279) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760761.428980:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26333184 -> 26337280 00010000:00010000:2.0:1789760761.429001:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2f0f5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26333184->26337279] (req 26333184->26337279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.429013:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2f0f5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26333184->26337279] (req 26333184->26337279) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.429021:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2f0f5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26333184->26337279] (req 26333184->26337279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.429028:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760761.429033:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2f0f5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26333184->26337279] (req 26333184->26337279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.429043:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2f0f5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26333184->26337279] (req 26333184->26337279) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.429063:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2f0f5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26333184->26337279] (req 26333184->26337279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.429072:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2f0f5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26333184->26337279] (req 26333184->26337279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760761.429074:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973748 is committed 00000100:00100000:2.0:1789760761.429172:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4038380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512685056:12345-192.168.203.56@tcp:4:dd.0 Request processed in 39518us (40611us total) trans 4294973748 rc 0/0 00000100:00100000:2.0:1789760761.429182:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6459 00000100:00100000:2.0:1789760761.429206:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 740908932718 time 740953934161 state 0x2 ref 0 00000100:00100000:2.0:1789760761.431405:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760761.431490:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512685184 00000100:00100000:0.0:1789760761.431499:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512685184 00000100:00100000:0.0:1789760761.431555:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 740908932718 time 740953934161 state 0x0 ref 1 00000100:00100000:0.0:1789760761.431569:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 740954194577 time 740953934161 state 0x1 ref 1 00000100:00100000:0.0:1789760761.431597:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 740954194577 time 740995896447 state 0x0 ref 1 00000100:00100000:0.0:1789760761.431616:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5304700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512685184:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760761.431660:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26337280, mtime: 0x6aad94f8, atime: 0x0, ctime: 0x6aad94f8, blocks: 0xd007 00010000:00010000:0.0:1789760761.431668:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2f0fc lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760761.431687:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=740s v=(0 0) 00010000:00010000:0.0:1789760761.431692:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760761.431742:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2f0fc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26337280->26341375] (req 26337280->26341375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760761.431756:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2f0fc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26337280->26341375] (req 26337280->26341375) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760761.516157:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973749 is committed 00002000:00010000:1.0:1789760761.516265:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26337280 -> 26341376 00010000:00010000:1.0:1789760761.516280:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2f0fc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26337280->26341375] (req 26337280->26341375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.516295:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2f0fc lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26337280->26341375] (req 26337280->26341375) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.516303:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2f0fc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26337280->26341375] (req 26337280->26341375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.516311:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760761.516316:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2f0fc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26337280->26341375] (req 26337280->26341375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.516324:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2f0fc lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26337280->26341375] (req 26337280->26341375) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.516352:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2f0fc lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26337280->26341375] (req 26337280->26341375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.516359:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2f0fc lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26337280->26341375] (req 26337280->26341375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760761.516455:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5304700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512685184:12345-192.168.203.56@tcp:4:dd.0 Request processed in 84839us (85053us total) trans 4294973749 rc 0/0 00000100:00100000:1.0:1789760761.516464:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6460 00000100:00100000:1.0:1789760761.516474:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 740954194577 time 740995896447 state 0x2 ref 0 00000100:00100000:0.0:1789760761.520768:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760761.520937:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512685312 00000100:00100000:1.0:1789760761.520947:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512685312 00000100:00100000:1.0:1789760761.520999:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 740954194577 time 740995896447 state 0x0 ref 1 00000100:00100000:1.0:1789760761.521014:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 740996156863 time 740995896447 state 0x1 ref 1 00000100:00100000:1.0:1789760761.521041:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 740996156863 time 741085341369 state 0x0 ref 1 00000100:00100000:1.0:1789760761.521056:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac28f0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512685312:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760761.521105:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26341376, mtime: 0x6aad94f8, atime: 0x0, ctime: 0x6aad94f8, blocks: 0xd007 00010000:00010000:1.0:1789760761.521114:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2f103 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.521131:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=741s v=(0 0) 00010000:00010000:1.0:1789760761.521136:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760761.521143:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2f103 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26341376->26345471] (req 26341376->26345471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.521158:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2f103 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26341376->26345471] (req 26341376->26345471) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760761.565731:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973750 is committed 00002000:00010000:2.0:1789760761.566039:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26341376 -> 26345472 00002000:00010000:2.0:1789760761.566044:0:6710:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760760 -> 1789760761 00002000:00010000:2.0:1789760761.566046:0:6710:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760760 -> 1789760761 00010000:00010000:2.0:1789760761.566060:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2f103 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26341376->26345471] (req 26341376->26345471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.566072:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2f103 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26341376->26345471] (req 26341376->26345471) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.566081:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2f103 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26341376->26345471] (req 26341376->26345471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.566088:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760761.566092:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2f103 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26341376->26345471] (req 26341376->26345471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.566101:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2f103 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26341376->26345471] (req 26341376->26345471) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.566120:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2f103 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26341376->26345471] (req 26341376->26345471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.566126:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2f103 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26341376->26345471] (req 26341376->26345471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760761.566260:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac28f0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512685312:12345-192.168.203.56@tcp:4:dd.0 Request processed in 45202us (45498us total) trans 4294973750 rc 0/0 00000100:00100000:2.0:1789760761.566269:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6461 00000100:00100000:2.0:1789760761.566277:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 740996156863 time 741085341369 state 0x2 ref 0 00000100:00100000:2.0:1789760761.573129:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760761.573221:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512685440 00000100:00100000:3.0:1789760761.573232:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512685440 00000100:00100000:3.0:1789760761.573286:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 740996156863 time 741085341369 state 0x0 ref 1 00000100:00100000:3.0:1789760761.573301:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 741085601785 time 741085341369 state 0x1 ref 1 00000100:00100000:3.0:1789760761.573335:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 741085601785 time 741137634308 state 0x0 ref 1 00000100:00100000:3.0:1789760761.573348:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5304380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512685440:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760761.573393:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26345472, mtime: 0x6aad94f9, atime: 0x0, ctime: 0x6aad94f9, blocks: 0xd007 00010000:00010000:3.0:1789760761.573404:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2f10a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.573419:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=741s v=(0 0) 00010000:00010000:3.0:1789760761.573423:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760761.573429:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2f10a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26345472->26349567] (req 26345472->26349567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.573443:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2f10a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26345472->26349567] (req 26345472->26349567) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760761.621441:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973751 is committed 00002000:00010000:3.0:1789760761.621732:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26345472 -> 26349568 00010000:00010000:3.0:1789760761.621760:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2f10a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26345472->26349567] (req 26345472->26349567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.621775:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2f10a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26345472->26349567] (req 26345472->26349567) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.621784:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2f10a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26345472->26349567] (req 26345472->26349567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.621792:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760761.621798:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2f10a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26345472->26349567] (req 26345472->26349567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.621808:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2f10a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26345472->26349567] (req 26345472->26349567) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.621827:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2f10a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26345472->26349567] (req 26345472->26349567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.621835:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2f10a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26345472->26349567] (req 26345472->26349567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760761.621937:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5304380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512685440:12345-192.168.203.56@tcp:4:dd.0 Request processed in 48586us (48812us total) trans 4294973751 rc 0/0 00000100:00100000:3.0:1789760761.621952:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6462 00000100:00100000:3.0:1789760761.621962:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 741085601785 time 741137634308 state 0x2 ref 0 00000100:00100000:3.0:1789760761.626741:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760761.626850:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512685568 00000100:00100000:1.0:1789760761.626868:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512685568 00000100:00100000:1.0:1789760761.626923:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 741085601785 time 741137634308 state 0x0 ref 1 00000100:00100000:1.0:1789760761.626939:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 741137894724 time 741137634308 state 0x1 ref 1 00000100:00100000:1.0:1789760761.626971:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 741137894724 time 741191271122 state 0x0 ref 1 00000100:00100000:1.0:1789760761.626984:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2747800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512685568:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760761.627035:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26349568, mtime: 0x6aad94f9, atime: 0x0, ctime: 0x6aad94f9, blocks: 0xd007 00010000:00010000:1.0:1789760761.627044:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2f111 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.627062:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=741s v=(0 0) 00010000:00010000:1.0:1789760761.627067:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760761.627073:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2f111 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26349568->26353663] (req 26349568->26353663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.627087:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2f111 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26349568->26353663] (req 26349568->26353663) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760761.652282:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973752 is committed 00002000:00010000:3.0:1789760761.652362:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26349568 -> 26353664 00010000:00010000:3.0:1789760761.652377:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2f111 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26349568->26353663] (req 26349568->26353663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.652390:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2f111 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26349568->26353663] (req 26349568->26353663) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.652398:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2f111 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26349568->26353663] (req 26349568->26353663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.652405:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760761.652409:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2f111 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26349568->26353663] (req 26349568->26353663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.652419:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2f111 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26349568->26353663] (req 26349568->26353663) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.652435:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2f111 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26349568->26353663] (req 26349568->26353663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.652442:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2f111 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26349568->26353663] (req 26349568->26353663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760761.652578:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2747800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512685568:12345-192.168.203.56@tcp:4:dd.0 Request processed in 25591us (25884us total) trans 4294973752 rc 0/0 00000100:00100000:3.0:1789760761.652588:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6463 00000100:00100000:3.0:1789760761.652597:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 741137894724 time 741191271122 state 0x2 ref 0 00000100:00100000:2.0:1789760761.655872:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760761.656066:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512685696 00000100:00100000:1.0:1789760761.656077:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512685696 00000100:00100000:1.0:1789760761.656130:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 741137894724 time 741191271122 state 0x0 ref 1 00000100:00100000:1.0:1789760761.656145:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 741191531538 time 741191271122 state 0x1 ref 1 00000100:00100000:1.0:1789760761.656175:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 741191531538 time 741220474363 state 0x0 ref 1 00000100:00100000:1.0:1789760761.656196:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5306680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512685696:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760761.656245:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26353664, mtime: 0x6aad94f9, atime: 0x0, ctime: 0x6aad94f9, blocks: 0xd007 00010000:00010000:1.0:1789760761.656254:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2f118 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.656270:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=741s v=(0 0) 00010000:00010000:1.0:1789760761.656274:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760761.656281:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2f118 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26353664->26357759] (req 26353664->26357759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.656294:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2f118 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26353664->26357759] (req 26353664->26357759) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760761.690318:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973753 is committed 00002000:00010000:0.0:1789760761.690525:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26353664 -> 26357760 00010000:00010000:0.0:1789760761.690541:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2f118 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26353664->26357759] (req 26353664->26357759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760761.690553:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2f118 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26353664->26357759] (req 26353664->26357759) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760761.690563:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2f118 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26353664->26357759] (req 26353664->26357759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760761.690577:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760761.690582:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2f118 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26353664->26357759] (req 26353664->26357759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760761.690591:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2f118 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26353664->26357759] (req 26353664->26357759) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760761.690609:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2f118 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26353664->26357759] (req 26353664->26357759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760761.690617:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2f118 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26353664->26357759] (req 26353664->26357759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760761.690775:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5306680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512685696:12345-192.168.203.56@tcp:4:dd.0 Request processed in 34585us (34907us total) trans 4294973753 rc 0/0 00000100:00100000:0.0:1789760761.690786:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6464 00000100:00100000:0.0:1789760761.690795:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 741191531538 time 741220474363 state 0x2 ref 0 00000100:00100000:2.0:1789760761.693976:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760761.694062:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512685824 00000100:00100000:3.0:1789760761.694158:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512685824 00000100:00100000:3.0:1789760761.694216:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 741191531538 time 741220474363 state 0x0 ref 1 00000100:00100000:3.0:1789760761.694229:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 741220734779 time 741220474363 state 0x1 ref 1 00000100:00100000:3.0:1789760761.694299:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 741220734779 time 741258598465 state 0x0 ref 1 00000100:00100000:3.0:1789760761.694311:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5307100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512685824:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760761.694353:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26357760, mtime: 0x6aad94f9, atime: 0x0, ctime: 0x6aad94f9, blocks: 0xd007 00010000:00010000:3.0:1789760761.694361:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2f11f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.694377:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=741s v=(0 0) 00010000:00010000:3.0:1789760761.694381:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760761.694387:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2f11f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26357760->26361855] (req 26357760->26361855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.694401:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2f11f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26357760->26361855] (req 26357760->26361855) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760761.742110:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973754 is committed 00002000:00010000:2.0:1789760761.742206:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26357760 -> 26361856 00010000:00010000:2.0:1789760761.742221:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2f11f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26357760->26361855] (req 26357760->26361855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.742234:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2f11f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26357760->26361855] (req 26357760->26361855) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.742243:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2f11f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26357760->26361855] (req 26357760->26361855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.742250:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760761.742255:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2f11f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26357760->26361855] (req 26357760->26361855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.742264:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2f11f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26357760->26361855] (req 26357760->26361855) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.742283:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2f11f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26357760->26361855] (req 26357760->26361855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.742291:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2f11f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26357760->26361855] (req 26357760->26361855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760761.742517:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5307100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512685824:12345-192.168.203.56@tcp:4:dd.0 Request processed in 48202us (48545us total) trans 4294973754 rc 0/0 00000100:00100000:2.0:1789760761.742528:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6465 00000100:00100000:2.0:1789760761.742539:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 741220734779 time 741258598465 state 0x2 ref 0 00000100:00100000:3.0:1789760761.745160:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760761.745272:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512685952 00000100:00100000:1.0:1789760761.745282:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512685952 00000100:00100000:1.0:1789760761.745332:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 741220734779 time 741258598465 state 0x0 ref 1 00000100:00100000:1.0:1789760761.745347:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 741258858881 time 741258598465 state 0x1 ref 1 00000100:00100000:1.0:1789760761.745484:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 741258858881 time 741309712940 state 0x0 ref 1 00000100:00100000:1.0:1789760761.745501:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2745500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512685952:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760761.745548:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26361856, mtime: 0x6aad94f9, atime: 0x0, ctime: 0x6aad94f9, blocks: 0xd007 00010000:00010000:1.0:1789760761.745557:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2f126 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.745572:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=741s v=(0 0) 00010000:00010000:1.0:1789760761.745587:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760761.745594:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2f126 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26361856->26365951] (req 26361856->26365951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.745608:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2f126 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26361856->26365951] (req 26361856->26365951) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760761.779405:0:3312:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8febd02e0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3312:1876699540227456:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1789760761.779474:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760761.779604:0:6707:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540227456 00000100:00100000:0.0:1789760761.779612:0:6707:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540227456 00000100:00100000:0.0:1789760761.779675:0:6707:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 204 00000100:00100000:0.0:1789760761.779684:0:6707:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac403a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3312:x1876699540227456:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1789760761.779725:0:6707:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 527630336 pending 1703936 free 3831496704 avail 3802136576 00000100:00100000:0.0:1789760761.779769:0:6707:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac403a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3312:x1876699540227456:12345-0@lo:13:osp-pre-0-0.0 Request processed in 86us (294us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760761.779777:0:6707:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 204 00000100:00080000:1.0:1789760761.779837:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1789760761.779873:0:3312:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8febd02e0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3312:1876699540227456:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789760761.784151:0:3310:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8feac5307800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3310:1876699540227712:0@lo:400:kworker.0 00000100:00100000:2.0:1789760761.784232:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760761.784286:0:3310:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:2.0:1789760761.784293:0:3310:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8feac5304a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:6df98d54-eb52-4c5c-8fc9-ee9d74f90f98:3310:1876699540227584:0@lo:400:kworker.0 00000100:00100000:2.0:1789760761.784324:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760761.784347:0:3310:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8feac5306a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3310:1876699540228096:0@lo:400:kworker.0 00000100:00100000:2.0:1789760761.784371:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760761.784400:0:3311:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8feac5305880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3311:1876699540227840:0@lo:400:kworker.0 00000100:00100000:2.0:1789760761.784434:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760761.784452:0:3311:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1789760761.784457:0:3311:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8feac5306300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3311:1876699540227968:0@lo:400:kworker.0 00000100:00100000:2.0:1789760761.784476:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760761.784505:0:5848:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540227712 00000100:00100000:2.0:1789760761.784513:0:5848:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540227712 00000100:00100000:2.0:1789760761.784554:0:5848:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540227840 00000100:00100000:2.0:1789760761.784557:0:5848:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540227840 00000100:00100000:2.0:1789760761.784580:0:5848:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540227968 00000100:00100000:2.0:1789760761.784583:0:5848:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540227968 00000100:00100000:2.0:1789760761.784603:0:5848:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 491 00000100:00100000:2.0:1789760761.784610:0:5848:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2877b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3310:x1876699540227712:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760761.784661:0:5848:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2877b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3310:x1876699540227712:12345-0@lo:400:kworker.0 Request processed in 51us (429us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760761.784668:0:5848:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 491 00000100:00100000:2.0:1789760761.784684:0:5848:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 492 00000100:00100000:2.0:1789760761.784690:0:5848:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2875880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3311:x1876699540227840:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760761.784739:0:5848:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2875880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3311:x1876699540227840:12345-0@lo:400:kworker.0 Request processed in 49us (304us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760761.784744:0:5848:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 492 00000100:00100000:2.0:1789760761.784756:0:5848:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 493 00000100:00100000:2.0:1789760761.784762:0:5848:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2876a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3311:x1876699540227968:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760761.784789:0:5848:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2876a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3311:x1876699540227968:12345-0@lo:400:kworker.0 Request processed in 27us (312us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760761.784795:0:5848:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 493 00000100:00100000:2.0:1789760761.784813:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540228096 00000100:00100000:2.0:1789760761.784817:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540228096 00000100:00100000:2.0:1789760761.784846:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 187 00000100:00100000:2.0:1789760761.784853:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2874700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3310:x1876699540228096:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760761.784881:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2874700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3310:x1876699540228096:12345-0@lo:400:kworker.0 Request processed in 29us (510us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760761.784888:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 187 00000100:00100000:2.0:1789760761.784925:0:3310:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8feac5306a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3310:1876699540228096:0@lo:400:kworker.0 00000100:00100000:2.0:1789760761.784937:0:3310:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8feac5307800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3310:1876699540227712:0@lo:400:kworker.0 00000100:00100000:2.0:1789760761.784963:0:3311:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8feac5306300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3311:1876699540227968:0@lo:400:kworker.0 00000100:00100000:2.0:1789760761.784971:0:3311:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8feac5305880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3311:1876699540227840:0@lo:400:kworker.0 00000100:00080000:1.0:1789760761.785897:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:1.0:1789760761.785905:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789760761.786798:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540227584 00000100:00100000:3.0:1789760761.786809:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540227584 00000100:00100000:3.0:1789760761.786905:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 405 00000100:00100000:3.0:1789760761.786915:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febe00c4050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:6df98d54-eb52-4c5c-8fc9-ee9d74f90f98+19:3310:x1876699540227584:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760761.787108:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febe00c4050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:6df98d54-eb52-4c5c-8fc9-ee9d74f90f98+19:3310:x1876699540227584:12345-0@lo:400:kworker.0 Request processed in 194us (2782us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760761.787116:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 405 00000100:00100000:3.0:1789760761.787175:0:3310:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8feac5304a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:6df98d54-eb52-4c5c-8fc9-ee9d74f90f98:3310:1876699540227584:0@lo:400:kworker.0 00000001:00080000:2.0:1789760761.814765:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973755 is committed 00002000:00010000:0.0:1789760761.815106:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26361856 -> 26365952 00010000:00010000:0.0:1789760761.815124:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2f126 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26361856->26365951] (req 26361856->26365951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760761.815138:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2f126 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26361856->26365951] (req 26361856->26365951) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760761.815147:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2f126 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26361856->26365951] (req 26361856->26365951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760761.815155:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760761.815160:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2f126 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26361856->26365951] (req 26361856->26365951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760761.815169:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2f126 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26361856->26365951] (req 26361856->26365951) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760761.815202:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2f126 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26361856->26365951] (req 26361856->26365951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760761.815211:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2f126 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26361856->26365951] (req 26361856->26365951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760761.815300:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2745500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512685952:12345-192.168.203.56@tcp:4:dd.0 Request processed in 69798us (70146us total) trans 4294973755 rc 0/0 00000100:00100000:0.0:1789760761.815313:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6466 00000100:00100000:0.0:1789760761.815323:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 741258858881 time 741309712940 state 0x2 ref 0 00000100:00100000:0.0:1789760761.819984:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760761.820207:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512686080 00000100:00100000:1.0:1789760761.820218:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512686080 00000100:00100000:1.0:1789760761.820273:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 741258858881 time 741309712940 state 0x0 ref 1 00000100:00100000:1.0:1789760761.820288:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 741309973356 time 741309712940 state 0x1 ref 1 00000100:00100000:1.0:1789760761.820359:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 741309973356 time 741384658310 state 0x0 ref 1 00000100:00100000:1.0:1789760761.820372:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febea4b6050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512686080:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760761.820419:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26365952, mtime: 0x6aad94f9, atime: 0x0, ctime: 0x6aad94f9, blocks: 0xd007 00010000:00010000:1.0:1789760761.820429:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2f12d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.820444:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=741s v=(0 0) 00010000:00010000:1.0:1789760761.820448:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760761.820455:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2f12d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26365952->26370047] (req 26365952->26370047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.820469:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2f12d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26365952->26370047] (req 26365952->26370047) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760761.855337:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973756 is committed 00002000:00010000:2.0:1789760761.855463:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26365952 -> 26370048 00010000:00010000:2.0:1789760761.855480:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2f12d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26365952->26370047] (req 26365952->26370047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.855494:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2f12d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26365952->26370047] (req 26365952->26370047) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.855504:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2f12d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26365952->26370047] (req 26365952->26370047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.855512:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760761.855517:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2f12d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26365952->26370047] (req 26365952->26370047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.855526:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2f12d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26365952->26370047] (req 26365952->26370047) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.855547:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2f12d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26365952->26370047] (req 26365952->26370047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.855555:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2f12d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26365952->26370047] (req 26365952->26370047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760761.855660:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febea4b6050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512686080:12345-192.168.203.56@tcp:4:dd.0 Request processed in 35284us (35681us total) trans 4294973756 rc 0/0 00000100:00100000:2.0:1789760761.855670:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6467 00000100:00100000:2.0:1789760761.855680:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 741309973356 time 741384658310 state 0x2 ref 0 00000100:00100000:2.0:1789760761.860903:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760761.861588:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512686208 00000100:00100000:3.0:1789760761.861599:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512686208 00000100:00100000:3.0:1789760761.861656:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 741309973356 time 741384658310 state 0x0 ref 1 00000100:00100000:3.0:1789760761.861671:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 741384918726 time 741384658310 state 0x1 ref 1 00000100:00100000:3.0:1789760761.861728:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 741384918726 time 741426027093 state 0x0 ref 1 00000100:00100000:3.0:1789760761.861742:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2875c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512686208:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760761.861787:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26370048, mtime: 0x6aad94f9, atime: 0x0, ctime: 0x6aad94f9, blocks: 0xd007 00010000:00010000:3.0:1789760761.861797:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2f134 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.861813:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=741s v=(0 0) 00010000:00010000:3.0:1789760761.861817:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760761.861823:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2f134 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26370048->26374143] (req 26370048->26374143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.861837:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2f134 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26370048->26374143] (req 26370048->26374143) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760761.887962:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973757 is committed 00002000:00010000:3.0:1789760761.888022:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26370048 -> 26374144 00010000:00010000:3.0:1789760761.888037:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2f134 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26370048->26374143] (req 26370048->26374143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.888050:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2f134 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26370048->26374143] (req 26370048->26374143) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.888060:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2f134 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26370048->26374143] (req 26370048->26374143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.888068:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760761.888074:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2f134 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26370048->26374143] (req 26370048->26374143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.888083:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2f134 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26370048->26374143] (req 26370048->26374143) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.888101:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2f134 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26370048->26374143] (req 26370048->26374143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.888109:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2f134 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26370048->26374143] (req 26370048->26374143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760761.888206:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2875c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512686208:12345-192.168.203.56@tcp:4:dd.0 Request processed in 26463us (27309us total) trans 4294973757 rc 0/0 00000100:00100000:3.0:1789760761.888216:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6468 00000100:00100000:3.0:1789760761.888225:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 741384918726 time 741426027093 state 0x2 ref 0 00000100:00100000:0.0:1789760761.892472:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760761.892572:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512686336 00000100:00100000:1.0:1789760761.892581:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512686336 00000100:00100000:1.0:1789760761.892631:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 741384918726 time 741426027093 state 0x0 ref 1 00000100:00100000:1.0:1789760761.892646:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 741426287509 time 741426027093 state 0x1 ref 1 00000100:00100000:1.0:1789760761.892675:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 741426287509 time 741456974738 state 0x0 ref 1 00000100:00100000:1.0:1789760761.892687:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febd02e0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512686336:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760761.892769:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26374144, mtime: 0x6aad94f9, atime: 0x0, ctime: 0x6aad94f9, blocks: 0xd007 00010000:00010000:1.0:1789760761.892778:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2f13b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.892794:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=741s v=(0 0) 00010000:00010000:1.0:1789760761.892798:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760761.892804:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2f13b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26374144->26378239] (req 26374144->26378239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.892818:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2f13b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26374144->26378239] (req 26374144->26378239) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760761.916769:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26374144 -> 26378240 00010000:00010000:2.0:1789760761.916790:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2f13b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26374144->26378239] (req 26374144->26378239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.916803:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2f13b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26374144->26378239] (req 26374144->26378239) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.916818:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2f13b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26374144->26378239] (req 26374144->26378239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.916826:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760761.916831:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2f13b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26374144->26378239] (req 26374144->26378239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.916841:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2f13b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26374144->26378239] (req 26374144->26378239) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.916952:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2f13b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26374144->26378239] (req 26374144->26378239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.916963:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2f13b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26374144->26378239] (req 26374144->26378239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760761.917055:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febd02e0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2359:x1876699512686336:12345-192.168.203.56@tcp:4:dd.0 Request processed in 24366us (24588us total) trans 4294973758 rc 0/0 00000100:00100000:2.0:1789760761.917065:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6469 00000100:00100000:2.0:1789760761.917075:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 741426287509 time 741456974738 state 0x2 ref 0 00000001:00080000:3.0:1789760761.917465:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973758 is committed 00000100:00100000:3.0:1789760761.919639:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760761.919759:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512686464 00000100:00100000:2.0:1789760761.919768:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512686464 00000100:00100000:2.0:1789760761.919816:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 741426287509 time 741456974738 state 0x0 ref 1 00000100:00100000:2.0:1789760761.919832:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 741457235154 time 741456974738 state 0x1 ref 1 00000100:00100000:2.0:1789760761.919869:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 741457235154 time 741484169130 state 0x0 ref 1 00000100:00100000:2.0:1789760761.919882:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2746a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512686464:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760761.919929:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26378240, mtime: 0x6aad94f9, atime: 0x0, ctime: 0x6aad94f9, blocks: 0xd007 00010000:00010000:2.0:1789760761.919939:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2f142 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.919954:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=741s v=(0 0) 00010000:00010000:2.0:1789760761.919958:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760761.919964:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2f142 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26378240->26382335] (req 26378240->26382335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.919978:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2f142 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26378240->26382335] (req 26378240->26382335) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760761.960081:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26378240 -> 26382336 00010000:00010000:3.0:1789760761.960100:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2f142 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26378240->26382335] (req 26378240->26382335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.960113:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2f142 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26378240->26382335] (req 26378240->26382335) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.960121:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2f142 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26378240->26382335] (req 26378240->26382335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.960128:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760761.960133:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2f142 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26378240->26382335] (req 26378240->26382335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.960142:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2f142 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26378240->26382335] (req 26378240->26382335) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.960160:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2f142 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26378240->26382335] (req 26378240->26382335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.960167:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2f142 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26378240->26382335] (req 26378240->26382335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760761.960265:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2746a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2358:x1876699512686464:12345-192.168.203.56@tcp:4:dd.0 Request processed in 40380us (40629us total) trans 4294973759 rc 0/0 00000100:00100000:3.0:1789760761.960276:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6470 00000100:00100000:3.0:1789760761.960286:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 741457235154 time 741484169130 state 0x2 ref 0 00000001:00080000:3.0:1789760761.960322:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973759 is committed 00000100:00100000:0.0:1789760761.963143:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760761.963231:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512686592 00000100:00100000:3.0:1789760761.963241:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512686592 00000100:00100000:3.0:1789760761.963298:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 741457235154 time 741484169130 state 0x0 ref 1 00000100:00100000:3.0:1789760761.963313:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 741484429546 time 741484169130 state 0x1 ref 1 00000100:00100000:3.0:1789760761.963352:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 741484429546 time 741527650963 state 0x0 ref 1 00000100:00100000:3.0:1789760761.963366:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac3302680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512686592:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760761.963416:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26382336, mtime: 0x6aad94f9, atime: 0x0, ctime: 0x6aad94f9, blocks: 0xd007 00010000:00010000:3.0:1789760761.963425:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2f149 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.963442:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=741s v=(0 0) 00010000:00010000:3.0:1789760761.963446:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760761.963452:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2f149 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26382336->26386431] (req 26382336->26386431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.963466:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2f149 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26382336->26386431] (req 26382336->26386431) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760761.992397:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973760 is committed 00002000:00010000:0.0:1789760761.992484:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26382336 -> 26386432 00010000:00010000:0.0:1789760761.992499:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2f149 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26382336->26386431] (req 26382336->26386431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760761.992511:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2f149 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26382336->26386431] (req 26382336->26386431) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760761.992520:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2f149 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26382336->26386431] (req 26382336->26386431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760761.992526:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760761.992530:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2f149 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26382336->26386431] (req 26382336->26386431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760761.992538:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2f149 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26382336->26386431] (req 26382336->26386431) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760761.992557:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2f149 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26382336->26386431] (req 26382336->26386431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760761.992568:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2f149 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26382336->26386431] (req 26382336->26386431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760761.992807:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac3302680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512686592:12345-192.168.203.56@tcp:4:dd.0 Request processed in 29438us (29669us total) trans 4294973760 rc 0/0 00000100:00100000:0.0:1789760761.992818:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6471 00000100:00100000:0.0:1789760761.992828:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 741484429546 time 741527650963 state 0x2 ref 0 00000100:00100000:3.0:1789760761.995713:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760761.995803:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512686720 00000100:00100000:2.0:1789760761.995814:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512686720 00000100:00100000:2.0:1789760761.995880:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 741484429546 time 741527650963 state 0x0 ref 1 00000100:00100000:2.0:1789760761.995896:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 741527911379 time 741527650963 state 0x1 ref 1 00000100:00100000:2.0:1789760761.995926:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 741527911379 time 741560225699 state 0x0 ref 1 00000100:00100000:2.0:1789760761.995938:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2745880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512686720:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760761.995993:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26386432, mtime: 0x6aad94f9, atime: 0x0, ctime: 0x6aad94f9, blocks: 0xd007 00010000:00010000:2.0:1789760761.996004:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2f150 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.996021:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=741s v=(0 0) 00010000:00010000:2.0:1789760761.996025:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760761.996032:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2f150 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26386432->26390527] (req 26386432->26390527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.996046:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2f150 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26386432->26390527] (req 26386432->26390527) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760762.029298:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973761 is committed 00002000:00010000:1.0:1789760762.029402:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26386432 -> 26390528 00010000:00010000:1.0:1789760762.029417:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2f150 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26386432->26390527] (req 26386432->26390527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.029430:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2f150 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26386432->26390527] (req 26386432->26390527) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.029440:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2f150 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26386432->26390527] (req 26386432->26390527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.029448:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760762.029453:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2f150 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26386432->26390527] (req 26386432->26390527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.029466:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2f150 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26386432->26390527] (req 26386432->26390527) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.029491:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2f150 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26386432->26390527] (req 26386432->26390527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.029499:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2f150 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26386432->26390527] (req 26386432->26390527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760762.029601:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2745880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512686720:12345-192.168.203.56@tcp:4:dd.0 Request processed in 33659us (33906us total) trans 4294973761 rc 0/0 00000100:00100000:1.0:1789760762.029611:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6472 00000100:00100000:1.0:1789760762.029620:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 741527911379 time 741560225699 state 0x2 ref 0 00000100:00100000:3.0:1789760762.033005:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760762.033143:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512686848 00000100:00100000:2.0:1789760762.033152:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512686848 00000100:00100000:2.0:1789760762.033213:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 741527911379 time 741560225699 state 0x0 ref 1 00000100:00100000:2.0:1789760762.033229:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 741560486115 time 741560225699 state 0x1 ref 1 00000100:00100000:2.0:1789760762.033258:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 741560486115 time 741597557502 state 0x0 ref 1 00000100:00100000:2.0:1789760762.033270:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2745180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512686848:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760762.033315:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26390528, mtime: 0x6aad94f9, atime: 0x0, ctime: 0x6aad94f9, blocks: 0xd007 00010000:00010000:2.0:1789760762.033324:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2f157 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.033345:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=741s v=(0 0) 00010000:00010000:2.0:1789760762.033349:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760762.033356:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2f157 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26390528->26394623] (req 26390528->26394623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.033370:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2f157 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26390528->26394623] (req 26390528->26394623) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760762.062157:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973762 is committed 00002000:00010000:2.0:1789760762.062267:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26390528 -> 26394624 00010000:00010000:2.0:1789760762.062282:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2f157 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26390528->26394623] (req 26390528->26394623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.062295:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2f157 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26390528->26394623] (req 26390528->26394623) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.062304:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2f157 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26390528->26394623] (req 26390528->26394623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.062311:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760762.062317:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2f157 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26390528->26394623] (req 26390528->26394623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.062327:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2f157 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26390528->26394623] (req 26390528->26394623) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.062345:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2f157 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26390528->26394623] (req 26390528->26394623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.062354:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2f157 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26390528->26394623] (req 26390528->26394623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760762.062453:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2745180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512686848:12345-192.168.203.56@tcp:4:dd.0 Request processed in 29179us (29453us total) trans 4294973762 rc 0/0 00000100:00100000:2.0:1789760762.062463:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6473 00000100:00100000:2.0:1789760762.062473:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 741560486115 time 741597557502 state 0x2 ref 0 00000100:00100000:2.0:1789760762.065960:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760762.066050:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512686976 00000100:00100000:0.0:1789760762.066060:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512686976 00000100:00100000:0.0:1789760762.066110:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 741560486115 time 741597557502 state 0x0 ref 1 00000100:00100000:0.0:1789760762.066125:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 741597817918 time 741597557502 state 0x1 ref 1 00000100:00100000:0.0:1789760762.066201:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 741597817918 time 741630499520 state 0x0 ref 1 00000100:00100000:0.0:1789760762.066214:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2877100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512686976:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760762.066264:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26394624, mtime: 0x6aad94f9, atime: 0x0, ctime: 0x6aad94f9, blocks: 0xd007 00010000:00010000:0.0:1789760762.066274:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f15e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.066291:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=741s v=(0 0) 00010000:00010000:0.0:1789760762.066295:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760762.066301:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f15e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26394624->26398719] (req 26394624->26398719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.066320:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f15e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26394624->26398719] (req 26394624->26398719) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760762.091772:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973763 is committed 00002000:00010000:0.0:1789760762.091922:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26394624 -> 26398720 00010000:00010000:0.0:1789760762.091936:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f15e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26394624->26398719] (req 26394624->26398719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.091947:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f15e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26394624->26398719] (req 26394624->26398719) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.091955:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f15e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26394624->26398719] (req 26394624->26398719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.091961:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760762.091966:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f15e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26394624->26398719] (req 26394624->26398719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.091973:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f15e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26394624->26398719] (req 26394624->26398719) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.091991:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f15e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26394624->26398719] (req 26394624->26398719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.091999:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f15e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26394624->26398719] (req 26394624->26398719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760762.092091:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2877100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512686976:12345-192.168.203.56@tcp:4:dd.0 Request processed in 25875us (26134us total) trans 4294973763 rc 0/0 00000100:00100000:0.0:1789760762.092102:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6474 00000100:00100000:0.0:1789760762.092112:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 741597817918 time 741630499520 state 0x2 ref 0 00000100:00100000:1.0:1789760762.095266:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760762.095915:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512687104 00000100:00100000:2.0:1789760762.095926:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512687104 00000100:00100000:2.0:1789760762.095980:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 741597817918 time 741630499520 state 0x0 ref 1 00000100:00100000:2.0:1789760762.095996:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 741630759936 time 741630499520 state 0x1 ref 1 00000100:00100000:2.0:1789760762.096067:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 741630759936 time 741660365913 state 0x0 ref 1 00000100:00100000:2.0:1789760762.096081:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac403bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512687104:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760762.096129:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26398720, mtime: 0x6aad94f9, atime: 0x0, ctime: 0x6aad94f9, blocks: 0xd007 00010000:00010000:2.0:1789760762.096138:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2f165 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.096154:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=741s v=(0 0) 00010000:00010000:2.0:1789760762.096159:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760762.096165:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2f165 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26398720->26402815] (req 26398720->26402815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.096189:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2f165 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26398720->26402815] (req 26398720->26402815) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760762.125630:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26398720 -> 26402816 00010000:00010000:2.0:1789760762.125653:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2f165 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26398720->26402815] (req 26398720->26402815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760762.125661:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973764 is committed 00010000:00010000:2.0:1789760762.125666:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2f165 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26398720->26402815] (req 26398720->26402815) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.125676:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2f165 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26398720->26402815] (req 26398720->26402815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.125683:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760762.125689:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2f165 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26398720->26402815] (req 26398720->26402815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.125727:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2f165 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26398720->26402815] (req 26398720->26402815) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.125748:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2f165 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26398720->26402815] (req 26398720->26402815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.125756:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2f165 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26398720->26402815] (req 26398720->26402815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760762.125859:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac403bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512687104:12345-192.168.203.56@tcp:4:dd.0 Request processed in 29767us (30587us total) trans 4294973764 rc 0/0 00000100:00100000:2.0:1789760762.125870:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6475 00000100:00100000:2.0:1789760762.125880:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 741630759936 time 741660365913 state 0x2 ref 0 00000100:00100000:3.0:1789760762.132272:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760762.132329:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512687232 00000100:00100000:2.0:1789760762.132339:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512687232 00000100:00100000:2.0:1789760762.132394:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 741630759936 time 741660365913 state 0x0 ref 1 00000100:00100000:2.0:1789760762.132409:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 741660626329 time 741660365913 state 0x1 ref 1 00000100:00100000:2.0:1789760762.132489:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 741660626329 time 741696788418 state 0x0 ref 1 00000100:00100000:2.0:1789760762.132503:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2744a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512687232:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760762.132566:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26402816, mtime: 0x6aad94f9, atime: 0x0, ctime: 0x6aad94f9, blocks: 0xd007 00010000:00010000:2.0:1789760762.132583:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2f16c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.132600:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=741s v=(0 0) 00010000:00010000:2.0:1789760762.132604:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760762.132611:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2f16c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26402816->26406911] (req 26402816->26406911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.132625:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2f16c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26402816->26406911] (req 26402816->26406911) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760762.175234:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973765 is committed 00002000:00010000:2.0:1789760762.175403:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26402816 -> 26406912 00010000:00010000:2.0:1789760762.175419:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2f16c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26402816->26406911] (req 26402816->26406911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.175432:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2f16c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26402816->26406911] (req 26402816->26406911) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.175440:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2f16c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26402816->26406911] (req 26402816->26406911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.175447:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760762.175452:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2f16c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26402816->26406911] (req 26402816->26406911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.175461:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2f16c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26402816->26406911] (req 26402816->26406911) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.175479:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2f16c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26402816->26406911] (req 26402816->26406911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.175486:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2f16c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26402816->26406911] (req 26402816->26406911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760762.176147:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2744a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512687232:12345-192.168.203.56@tcp:4:dd.0 Request processed in 43070us (43308us total) trans 4294973765 rc 0/0 00000100:00100000:2.0:1789760762.176162:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6476 00000100:00100000:2.0:1789760762.176172:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 741660626329 time 741696788418 state 0x2 ref 0 00000100:00100000:0.0:1789760762.179309:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760762.179418:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512687360 00000100:00100000:2.0:1789760762.179429:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512687360 00000100:00100000:2.0:1789760762.179482:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 741660626329 time 741696788418 state 0x0 ref 1 00000100:00100000:2.0:1789760762.179497:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 741697048834 time 741696788418 state 0x1 ref 1 00000100:00100000:2.0:1789760762.179528:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 741697048834 time 741743827656 state 0x0 ref 1 00000100:00100000:2.0:1789760762.179541:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac3300e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512687360:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760762.179591:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26406912, mtime: 0x6aad94f9, atime: 0x0, ctime: 0x6aad94f9, blocks: 0xd007 00010000:00010000:2.0:1789760762.179600:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2f173 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.179616:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=741s v=(0 0) 00010000:00010000:2.0:1789760762.179621:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760762.179627:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2f173 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26406912->26411007] (req 26406912->26411007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.179641:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2f173 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26406912->26411007] (req 26406912->26411007) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760762.210555:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973766 is committed 00002000:00010000:2.0:1789760762.210726:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26406912 -> 26411008 00010000:00010000:2.0:1789760762.210741:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2f173 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26406912->26411007] (req 26406912->26411007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.210754:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2f173 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26406912->26411007] (req 26406912->26411007) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.210762:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2f173 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26406912->26411007] (req 26406912->26411007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.210769:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760762.210774:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2f173 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26406912->26411007] (req 26406912->26411007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.210784:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2f173 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26406912->26411007] (req 26406912->26411007) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.210804:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2f173 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26406912->26411007] (req 26406912->26411007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.210812:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2f173 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26406912->26411007] (req 26406912->26411007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760762.210903:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac3300e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512687360:12345-192.168.203.56@tcp:4:dd.0 Request processed in 31360us (31598us total) trans 4294973766 rc 0/0 00000100:00100000:2.0:1789760762.210914:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6477 00000100:00100000:2.0:1789760762.210923:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 741697048834 time 741743827656 state 0x2 ref 0 00000100:00100000:3.0:1789760762.215343:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760762.215408:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512687488 00000100:00100000:2.0:1789760762.215419:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512687488 00000100:00100000:2.0:1789760762.215472:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 741697048834 time 741743827656 state 0x0 ref 1 00000100:00100000:2.0:1789760762.215487:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 741744088072 time 741743827656 state 0x1 ref 1 00000100:00100000:2.0:1789760762.215537:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 741744088072 time 741779826617 state 0x0 ref 1 00000100:00100000:2.0:1789760762.215550:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2746300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512687488:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760762.215607:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26411008, mtime: 0x6aad94f9, atime: 0x0, ctime: 0x6aad94f9, blocks: 0xd007 00010000:00010000:2.0:1789760762.215616:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2f17a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.215632:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=741s v=(0 0) 00010000:00010000:2.0:1789760762.215637:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760762.215643:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2f17a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26411008->26415103] (req 26411008->26415103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.215656:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2f17a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26411008->26415103] (req 26411008->26415103) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.226816:0:7647:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:2.0:1789760762.240902:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26411008 -> 26415104 00010000:00010000:2.0:1789760762.240921:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2f17a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26411008->26415103] (req 26411008->26415103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.240935:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2f17a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26411008->26415103] (req 26411008->26415103) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.240949:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2f17a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26411008->26415103] (req 26411008->26415103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.240956:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760762.240961:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2f17a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26411008->26415103] (req 26411008->26415103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.240970:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2f17a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26411008->26415103] (req 26411008->26415103) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.240988:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2f17a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26411008->26415103] (req 26411008->26415103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.240997:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2f17a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26411008->26415103] (req 26411008->26415103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760762.241080:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2746300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2359:x1876699512687488:12345-192.168.203.56@tcp:4:dd.0 Request processed in 25528us (25743us total) trans 4294973767 rc 0/0 00000100:00100000:2.0:1789760762.241088:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6478 00000001:00080000:3.0:1789760762.241228:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973767 is committed 00000100:00100000:2.0:1789760762.241351:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 741744088072 time 741779826617 state 0x2 ref 0 00000100:00100000:1.0:1789760762.243495:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760762.243581:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512687616 00000100:00100000:2.0:1789760762.243591:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512687616 00000100:00100000:2.0:1789760762.243642:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 741744088072 time 741779826617 state 0x0 ref 1 00000100:00100000:2.0:1789760762.243656:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 741780087033 time 741779826617 state 0x1 ref 1 00000100:00100000:2.0:1789760762.243748:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 741780087033 time 741808046958 state 0x0 ref 1 00000100:00100000:2.0:1789760762.243766:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac403b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512687616:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760762.243809:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26415104, mtime: 0x6aad94f9, atime: 0x0, ctime: 0x6aad94f9, blocks: 0xd007 00010000:00010000:2.0:1789760762.243817:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2f181 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.243832:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=741s v=(0 0) 00010000:00010000:2.0:1789760762.243836:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760762.243842:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2f181 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26415104->26419199] (req 26415104->26419199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.243855:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2f181 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26415104->26419199] (req 26415104->26419199) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760762.282636:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973768 is committed 00002000:00010000:2.0:1789760762.282779:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26415104 -> 26419200 00010000:00010000:2.0:1789760762.282794:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2f181 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26415104->26419199] (req 26415104->26419199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.282945:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2f181 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26415104->26419199] (req 26415104->26419199) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.282956:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2f181 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26415104->26419199] (req 26415104->26419199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.283042:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760762.283047:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2f181 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26415104->26419199] (req 26415104->26419199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.283056:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2f181 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26415104->26419199] (req 26415104->26419199) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.283076:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2f181 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26415104->26419199] (req 26415104->26419199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.283083:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2f181 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26415104->26419199] (req 26415104->26419199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760762.283223:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac403b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512687616:12345-192.168.203.56@tcp:4:dd.0 Request processed in 39457us (39731us total) trans 4294973768 rc 0/0 00000100:00100000:2.0:1789760762.283233:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6479 00000100:00100000:2.0:1789760762.283242:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 741780087033 time 741808046958 state 0x2 ref 0 00000100:00100000:0.0:1789760762.286216:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760762.286306:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512687744 00000100:00100000:3.0:1789760762.286315:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512687744 00000100:00100000:3.0:1789760762.286367:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 741780087033 time 741808046958 state 0x0 ref 1 00000100:00100000:3.0:1789760762.286382:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 741808307374 time 741808046958 state 0x1 ref 1 00000100:00100000:3.0:1789760762.286412:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 741808307374 time 741850711755 state 0x0 ref 1 00000100:00100000:3.0:1789760762.286426:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4059180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512687744:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760762.286473:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26419200, mtime: 0x6aad94f9, atime: 0x0, ctime: 0x6aad94f9, blocks: 0xd007 00010000:00010000:3.0:1789760762.286482:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2f188 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.286498:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=741s v=(0 0) 00010000:00010000:3.0:1789760762.286503:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760762.286508:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2f188 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26419200->26423295] (req 26419200->26423295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.286522:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2f188 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26419200->26423295] (req 26419200->26423295) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760762.292597:0:3310:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8feac4059500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3310:1876699540228224:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1789760762.292665:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760762.292789:0:6707:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540228224 00000100:00100000:0.0:1789760762.292797:0:6707:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540228224 00000100:00100000:0.0:1789760762.292848:0:6707:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 205 00000100:00100000:0.0:1789760762.292867:0:6707:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4038000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3310:x1876699540228224:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1789760762.292903:0:6707:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3859808256 avail 3857711104 00000100:00100000:0.0:1789760762.292951:0:6707:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4038000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3310:x1876699540228224:12345-0@lo:13:osp-pre-1-0.0 Request processed in 86us (285us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760762.292970:0:6707:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 205 00000100:00100000:0.0:1789760762.293041:0:3310:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8feac4059500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3310:1876699540228224:0@lo:13:osp-pre-1-0.0 00000001:00080000:3.0:1789760762.316375:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973769 is committed 00002000:00010000:2.0:1789760762.316488:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26419200 -> 26423296 00010000:00010000:2.0:1789760762.316503:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2f188 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26419200->26423295] (req 26419200->26423295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.316516:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2f188 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26419200->26423295] (req 26419200->26423295) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.316525:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2f188 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26419200->26423295] (req 26419200->26423295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.316539:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760762.316544:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2f188 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26419200->26423295] (req 26419200->26423295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.316554:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2f188 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26419200->26423295] (req 26419200->26423295) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.316639:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2f188 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26419200->26423295] (req 26419200->26423295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.316649:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2f188 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26419200->26423295] (req 26419200->26423295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760762.316985:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4059180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512687744:12345-192.168.203.56@tcp:4:dd.0 Request processed in 30556us (30772us total) trans 4294973769 rc 0/0 00000100:00100000:1.0:1789760762.316996:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6480 00000100:00100000:1.0:1789760762.317005:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 741808307374 time 741850711755 state 0x2 ref 0 00000100:00100000:1.0:1789760762.321224:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760762.321315:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512687872 00000100:00100000:2.0:1789760762.321326:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512687872 00000100:00100000:2.0:1789760762.321378:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 741808307374 time 741850711755 state 0x0 ref 1 00000100:00100000:2.0:1789760762.321392:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 741850972171 time 741850711755 state 0x1 ref 1 00000100:00100000:2.0:1789760762.321456:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 741850972171 time 741885755277 state 0x0 ref 1 00000100:00100000:2.0:1789760762.321468:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4038700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512687872:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760762.321515:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26423296, mtime: 0x6aad94f9, atime: 0x0, ctime: 0x6aad94f9, blocks: 0xd007 00010000:00010000:2.0:1789760762.321524:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2f18f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.321543:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=741s v=(0 0) 00010000:00010000:2.0:1789760762.321548:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760762.321553:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2f18f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26423296->26427391] (req 26423296->26427391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.321567:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2f18f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26423296->26427391] (req 26423296->26427391) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760762.358447:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973770 is committed 00002000:00010000:2.0:1789760762.358644:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26423296 -> 26427392 00010000:00010000:2.0:1789760762.358660:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2f18f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26423296->26427391] (req 26423296->26427391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.358673:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2f18f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26423296->26427391] (req 26423296->26427391) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.358682:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2f18f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26423296->26427391] (req 26423296->26427391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.358690:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760762.358723:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2f18f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26423296->26427391] (req 26423296->26427391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.358735:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2f18f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26423296->26427391] (req 26423296->26427391) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.358754:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2f18f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26423296->26427391] (req 26423296->26427391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.358763:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2f18f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26423296->26427391] (req 26423296->26427391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760762.358859:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4038700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512687872:12345-192.168.203.56@tcp:4:dd.0 Request processed in 37377us (37628us total) trans 4294973770 rc 0/0 00000100:00100000:2.0:1789760762.358870:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6481 00000100:00100000:2.0:1789760762.358880:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 741850972171 time 741885755277 state 0x2 ref 0 00000100:00100000:0.0:1789760762.363082:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760762.363152:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512688000 00000100:00100000:2.0:1789760762.363172:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512688000 00000100:00100000:2.0:1789760762.363233:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 741850972171 time 741885755277 state 0x0 ref 1 00000100:00100000:2.0:1789760762.363249:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 741886015693 time 741885755277 state 0x1 ref 1 00000100:00100000:2.0:1789760762.363280:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 741886015693 time 741927579467 state 0x0 ref 1 00000100:00100000:2.0:1789760762.363293:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4059500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512688000:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760762.363343:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26427392, mtime: 0x6aad94f9, atime: 0x0, ctime: 0x6aad94f9, blocks: 0xd007 00010000:00010000:2.0:1789760762.363351:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2f196 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.363367:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=741s v=(0 0) 00010000:00010000:2.0:1789760762.363371:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760762.363377:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2f196 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26427392->26431487] (req 26427392->26431487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.363395:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2f196 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26427392->26431487] (req 26427392->26431487) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760762.406330:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973771 is committed 00002000:00010000:2.0:1789760762.406443:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26427392 -> 26431488 00010000:00010000:2.0:1789760762.406462:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2f196 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26427392->26431487] (req 26427392->26431487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.406475:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2f196 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26427392->26431487] (req 26427392->26431487) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.406485:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2f196 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26427392->26431487] (req 26427392->26431487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.406494:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760762.406499:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2f196 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26427392->26431487] (req 26427392->26431487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.406509:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2f196 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26427392->26431487] (req 26427392->26431487) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.406528:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2f196 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26427392->26431487] (req 26427392->26431487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.406537:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2f196 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26427392->26431487] (req 26427392->26431487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760762.407939:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4059500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512688000:12345-192.168.203.56@tcp:4:dd.0 Request processed in 43756us (43974us total) trans 4294973771 rc 0/0 00000100:00100000:2.0:1789760762.407955:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6482 00000100:00100000:2.0:1789760762.407965:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 741886015693 time 741927579467 state 0x2 ref 0 00000100:00100000:0.0:1789760762.410254:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760762.410350:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512688128 00000100:00100000:3.0:1789760762.410362:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512688128 00000100:00100000:3.0:1789760762.410422:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 741886015693 time 741927579467 state 0x0 ref 1 00000100:00100000:3.0:1789760762.410439:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 741927839883 time 741927579467 state 0x1 ref 1 00000100:00100000:3.0:1789760762.410475:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 741927839883 time 741974774336 state 0x0 ref 1 00000100:00100000:3.0:1789760762.410487:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac405ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512688128:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760762.410546:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26431488, mtime: 0x6aad94f9, atime: 0x0, ctime: 0x6aad94f9, blocks: 0xd007 00010000:00010000:3.0:1789760762.410555:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2f19d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.410682:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=741s v=(0 0) 00010000:00010000:3.0:1789760762.410688:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760762.410733:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2f19d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26431488->26435583] (req 26431488->26435583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.410749:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2f19d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26431488->26435583] (req 26431488->26435583) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760762.443844:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973772 is committed 00002000:00010000:0.0:1789760762.444007:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26431488 -> 26435584 00010000:00010000:0.0:1789760762.444023:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2f19d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26431488->26435583] (req 26431488->26435583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.444037:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2f19d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26431488->26435583] (req 26431488->26435583) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.444048:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2f19d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26431488->26435583] (req 26431488->26435583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.444139:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760762.444144:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2f19d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26431488->26435583] (req 26431488->26435583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.444154:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2f19d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26431488->26435583] (req 26431488->26435583) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.444188:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2f19d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26431488->26435583] (req 26431488->26435583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.444197:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2f19d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26431488->26435583] (req 26431488->26435583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760762.444295:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac405ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512688128:12345-192.168.203.56@tcp:4:dd.0 Request processed in 33805us (34047us total) trans 4294973772 rc 0/0 00000100:00100000:0.0:1789760762.444305:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6483 00000100:00100000:0.0:1789760762.444315:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 741927839883 time 741974774336 state 0x2 ref 0 00000100:00100000:0.0:1789760762.447618:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760762.447803:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512688256 00000100:00100000:2.0:1789760762.447821:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512688256 00000100:00100000:2.0:1789760762.447880:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 741927839883 time 741974774336 state 0x0 ref 1 00000100:00100000:2.0:1789760762.447895:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 741975034752 time 741974774336 state 0x1 ref 1 00000100:00100000:2.0:1789760762.447925:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 741975034752 time 742012224965 state 0x0 ref 1 00000100:00100000:2.0:1789760762.447938:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4058e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512688256:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760762.447985:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26435584, mtime: 0x6aad94f9, atime: 0x0, ctime: 0x6aad94f9, blocks: 0xd007 00010000:00010000:2.0:1789760762.447994:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2f1a4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.448010:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=742s v=(0 0) 00010000:00010000:2.0:1789760762.448014:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760762.448021:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2f1a4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26435584->26439679] (req 26435584->26439679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.448036:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2f1a4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26435584->26439679] (req 26435584->26439679) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760762.479364:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973773 is committed 00002000:00010000:1.0:1789760762.479643:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26435584 -> 26439680 00010000:00010000:1.0:1789760762.479660:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2f1a4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26435584->26439679] (req 26435584->26439679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.479674:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2f1a4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26435584->26439679] (req 26435584->26439679) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.479683:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2f1a4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26435584->26439679] (req 26435584->26439679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.479691:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760762.479775:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2f1a4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26435584->26439679] (req 26435584->26439679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.479786:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2f1a4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26435584->26439679] (req 26435584->26439679) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.479806:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2f1a4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26435584->26439679] (req 26435584->26439679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.479814:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2f1a4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26435584->26439679] (req 26435584->26439679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760762.480142:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4058e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512688256:12345-192.168.203.56@tcp:4:dd.0 Request processed in 32200us (32526us total) trans 4294973773 rc 0/0 00000100:00100000:2.0:1789760762.480153:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6484 00000100:00100000:2.0:1789760762.480163:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 741975034752 time 742012224965 state 0x2 ref 0 00000100:00100000:2.0:1789760762.483439:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760762.483519:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512688384 00000100:00100000:0.0:1789760762.483528:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512688384 00000100:00100000:0.0:1789760762.483642:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 741975034752 time 742012224965 state 0x0 ref 1 00000100:00100000:0.0:1789760762.483661:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 742012485381 time 742012224965 state 0x1 ref 1 00000100:00100000:0.0:1789760762.483765:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 742012485381 time 742048063207 state 0x0 ref 1 00000100:00100000:0.0:1789760762.483778:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febea89b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512688384:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760762.483826:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26439680, mtime: 0x6aad94f9, atime: 0x0, ctime: 0x6aad94f9, blocks: 0xd007 00010000:00010000:0.0:1789760762.483835:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2f1ab lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.483861:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=742s v=(0 0) 00010000:00010000:0.0:1789760762.483866:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760762.483873:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2f1ab lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26439680->26443775] (req 26439680->26443775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.483886:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2f1ab lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26439680->26443775] (req 26439680->26443775) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760762.520747:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973774 is committed 00002000:00010000:2.0:1789760762.521554:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26439680 -> 26443776 00010000:00010000:2.0:1789760762.521569:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2f1ab lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26439680->26443775] (req 26439680->26443775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.521582:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2f1ab lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26439680->26443775] (req 26439680->26443775) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.521591:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2f1ab lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26439680->26443775] (req 26439680->26443775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.521599:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760762.521604:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2f1ab lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26439680->26443775] (req 26439680->26443775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.521615:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2f1ab lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26439680->26443775] (req 26439680->26443775) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.521634:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2f1ab lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26439680->26443775] (req 26439680->26443775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.521648:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2f1ab lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26439680->26443775] (req 26439680->26443775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760762.521824:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febea89b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512688384:12345-192.168.203.56@tcp:4:dd.0 Request processed in 38042us (38389us total) trans 4294973774 rc 0/0 00000100:00100000:2.0:1789760762.521835:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6485 00000100:00100000:2.0:1789760762.521855:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 742012485381 time 742048063207 state 0x2 ref 0 00000100:00100000:0.0:1789760762.525435:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760762.525787:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512688512 00000100:00100000:3.0:1789760762.525798:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512688512 00000100:00100000:3.0:1789760762.525917:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 742012485381 time 742048063207 state 0x0 ref 1 00000100:00100000:3.0:1789760762.525933:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 742048323623 time 742048063207 state 0x1 ref 1 00000100:00100000:3.0:1789760762.525990:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 742048323623 time 742090288494 state 0x0 ref 1 00000100:00100000:3.0:1789760762.526005:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac38b2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512688512:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760762.526139:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26443776, mtime: 0x6aad94f9, atime: 0x0, ctime: 0x6aad94f9, blocks: 0xd007 00010000:00010000:3.0:1789760762.526149:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2f1b2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.526166:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=742s v=(0 0) 00010000:00010000:3.0:1789760762.526171:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760762.526186:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2f1b2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26443776->26447871] (req 26443776->26447871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.526200:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2f1b2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26443776->26447871] (req 26443776->26447871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760762.559648:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26443776 -> 26447872 00002000:00010000:3.0:1789760762.559657:0:13063:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760761 -> 1789760762 00002000:00010000:3.0:1789760762.559661:0:13063:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760761 -> 1789760762 00010000:00010000:3.0:1789760762.559675:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2f1b2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26443776->26447871] (req 26443776->26447871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.559688:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2f1b2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26443776->26447871] (req 26443776->26447871) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.559728:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2f1b2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26443776->26447871] (req 26443776->26447871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.559737:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760762.559741:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2f1b2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26443776->26447871] (req 26443776->26447871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.559750:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2f1b2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26443776->26447871] (req 26443776->26447871) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.559770:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2f1b2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26443776->26447871] (req 26443776->26447871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.559779:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2f1b2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26443776->26447871] (req 26443776->26447871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760762.559987:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac38b2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2360:x1876699512688512:12345-192.168.203.56@tcp:4:dd.0 Request processed in 33980us (34557us total) trans 4294973775 rc 0/0 00000100:00100000:3.0:1789760762.559999:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6486 00000100:00100000:3.0:1789760762.560010:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 742048323623 time 742090288494 state 0x2 ref 0 00000001:00080000:1.0:1789760762.560193:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973775 is committed 00000100:00100000:2.0:1789760762.564011:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760762.564211:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512688640 00000100:00100000:0.0:1789760762.564223:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512688640 00000100:00100000:0.0:1789760762.564278:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 742048323623 time 742090288494 state 0x0 ref 1 00000100:00100000:0.0:1789760762.564293:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 742090548910 time 742090288494 state 0x1 ref 1 00000100:00100000:0.0:1789760762.564354:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 742090548910 time 742128653876 state 0x0 ref 1 00000100:00100000:0.0:1789760762.564370:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2fba680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512688640:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760762.564423:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26447872, mtime: 0x6aad94fa, atime: 0x0, ctime: 0x6aad94fa, blocks: 0xd007 00010000:00010000:0.0:1789760762.564433:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2f1b9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.564450:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=742s v=(0 0) 00010000:00010000:0.0:1789760762.564455:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760762.564461:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2f1b9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26447872->26451967] (req 26447872->26451967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.564474:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2f1b9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26447872->26451967] (req 26447872->26451967) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760762.592317:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26447872 -> 26451968 00010000:00010000:0.0:1789760762.592339:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2f1b9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26447872->26451967] (req 26447872->26451967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.592354:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2f1b9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26447872->26451967] (req 26447872->26451967) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.592363:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2f1b9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26447872->26451967] (req 26447872->26451967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.592371:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760762.592376:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2f1b9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26447872->26451967] (req 26447872->26451967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.592386:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2f1b9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26447872->26451967] (req 26447872->26451967) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.592404:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2f1b9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26447872->26451967] (req 26447872->26451967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.592412:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2f1b9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26447872->26451967] (req 26447872->26451967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760762.592501:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2fba680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2361:x1876699512688640:12345-192.168.203.56@tcp:4:dd.0 Request processed in 28128us (28495us total) trans 4294973776 rc 0/0 00000100:00100000:0.0:1789760762.592511:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6487 00000100:00100000:0.0:1789760762.592521:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 742090548910 time 742128653876 state 0x2 ref 0 00000001:00080000:0.0:1789760762.592560:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973776 is committed 00000100:00100000:2.0:1789760762.595471:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760762.595545:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512688768 00000100:00100000:0.0:1789760762.595555:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512688768 00000100:00100000:0.0:1789760762.595616:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 742090548910 time 742128653876 state 0x0 ref 1 00000100:00100000:0.0:1789760762.595633:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 742128914292 time 742128653876 state 0x1 ref 1 00000100:00100000:0.0:1789760762.595725:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 742128914292 time 742160023697 state 0x0 ref 1 00000100:00100000:0.0:1789760762.595738:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2fb8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512688768:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760762.595788:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26451968, mtime: 0x6aad94fa, atime: 0x0, ctime: 0x6aad94fa, blocks: 0xd007 00010000:00010000:0.0:1789760762.595798:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2f1c0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.595814:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=742s v=(0 0) 00010000:00010000:0.0:1789760762.595818:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760762.595824:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2f1c0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26451968->26456063] (req 26451968->26456063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.595838:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2f1c0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26451968->26456063] (req 26451968->26456063) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760762.628999:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26451968 -> 26456064 00010000:00010000:0.0:1789760762.629019:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2f1c0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26451968->26456063] (req 26451968->26456063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.629032:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2f1c0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26451968->26456063] (req 26451968->26456063) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.629042:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2f1c0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26451968->26456063] (req 26451968->26456063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.629061:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760762.629066:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2f1c0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26451968->26456063] (req 26451968->26456063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.629075:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2f1c0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26451968->26456063] (req 26451968->26456063) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.629100:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2f1c0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26451968->26456063] (req 26451968->26456063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.629108:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2f1c0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26451968->26456063] (req 26451968->26456063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760762.629208:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2fb8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2361:x1876699512688768:12345-192.168.203.56@tcp:4:dd.0 Request processed in 33467us (33742us total) trans 4294973777 rc 0/0 00000100:00100000:0.0:1789760762.629219:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6488 00000100:00100000:0.0:1789760762.629228:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 742128914292 time 742160023697 state 0x2 ref 0 00000001:00080000:0.0:1789760762.629265:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973777 is committed 00000100:00100000:1.0:1789760762.638329:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760762.638434:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512688896 00000100:00100000:3.0:1789760762.638445:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512688896 00000100:00100000:3.0:1789760762.638503:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 742128914292 time 742160023697 state 0x0 ref 1 00000100:00100000:3.0:1789760762.638519:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 742160284113 time 742160023697 state 0x1 ref 1 00000100:00100000:3.0:1789760762.638659:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 742160284113 time 742202957251 state 0x0 ref 1 00000100:00100000:3.0:1789760762.638673:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac403aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512688896:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760762.638749:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26456064, mtime: 0x6aad94fa, atime: 0x0, ctime: 0x6aad94fa, blocks: 0xd007 00010000:00010000:3.0:1789760762.638759:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2f1c7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.638776:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=742s v=(0 0) 00010000:00010000:3.0:1789760762.638781:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760762.638788:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2f1c7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26456064->26460159] (req 26456064->26460159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.638801:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2f1c7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26456064->26460159] (req 26456064->26460159) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760762.686347:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973778 is committed 00002000:00010000:1.0:1789760762.686602:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26456064 -> 26460160 00010000:00010000:1.0:1789760762.686618:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2f1c7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26456064->26460159] (req 26456064->26460159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.686630:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2f1c7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26456064->26460159] (req 26456064->26460159) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.686641:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2f1c7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26456064->26460159] (req 26456064->26460159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.686654:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760762.686659:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2f1c7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26456064->26460159] (req 26456064->26460159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.686668:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2f1c7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26456064->26460159] (req 26456064->26460159) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.686687:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2f1c7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26456064->26460159] (req 26456064->26460159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.686711:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2f1c7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26456064->26460159] (req 26456064->26460159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760762.686805:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac403aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512688896:12345-192.168.203.56@tcp:4:dd.0 Request processed in 48131us (48482us total) trans 4294973778 rc 0/0 00000100:00100000:1.0:1789760762.686817:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6489 00000100:00100000:1.0:1789760762.686828:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 742160284113 time 742202957251 state 0x2 ref 0 00000100:00100000:3.0:1789760762.689491:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760762.692775:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512689024 00000100:00100000:0.0:1789760762.692791:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512689024 00000100:00100000:0.0:1789760762.692860:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 742160284113 time 742202957251 state 0x0 ref 1 00000100:00100000:0.0:1789760762.692879:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 742203217667 time 742202957251 state 0x1 ref 1 00000100:00100000:0.0:1789760762.692910:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 742203217667 time 742257210401 state 0x0 ref 1 00000100:00100000:0.0:1789760762.692925:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2747b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512689024:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760762.692972:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26460160, mtime: 0x6aad94fa, atime: 0x0, ctime: 0x6aad94fa, blocks: 0xd007 00010000:00010000:0.0:1789760762.692982:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2f1ce lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.693002:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=742s v=(0 0) 00010000:00010000:0.0:1789760762.693007:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760762.693013:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2f1ce lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26460160->26464255] (req 26460160->26464255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.693027:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2f1ce lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26460160->26464255] (req 26460160->26464255) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760762.731291:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973779 is committed 00002000:00010000:1.0:1789760762.731394:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26460160 -> 26464256 00010000:00010000:1.0:1789760762.731410:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2f1ce lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26460160->26464255] (req 26460160->26464255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.731428:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2f1ce lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26460160->26464255] (req 26460160->26464255) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.731439:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2f1ce lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26460160->26464255] (req 26460160->26464255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.731447:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760762.731452:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2f1ce lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26460160->26464255] (req 26460160->26464255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.731461:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2f1ce lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26460160->26464255] (req 26460160->26464255) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.731480:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2f1ce lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26460160->26464255] (req 26460160->26464255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.731487:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2f1ce lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26460160->26464255] (req 26460160->26464255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760762.731665:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2747b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512689024:12345-192.168.203.56@tcp:4:dd.0 Request processed in 38739us (42178us total) trans 4294973779 rc 0/0 00000100:00100000:1.0:1789760762.731676:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6490 00000100:00100000:1.0:1789760762.731686:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 742203217667 time 742257210401 state 0x2 ref 0 00000100:00100000:0.0:1789760762.734278:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760762.734429:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512689152 00000100:00100000:2.0:1789760762.734440:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512689152 00000100:00100000:2.0:1789760762.734495:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 742203217667 time 742257210401 state 0x0 ref 1 00000100:00100000:2.0:1789760762.734510:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 742257470817 time 742257210401 state 0x1 ref 1 00000100:00100000:2.0:1789760762.734646:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 742257470817 time 742298944031 state 0x0 ref 1 00000100:00100000:2.0:1789760762.734660:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febfb374380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512689152:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760762.734736:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26464256, mtime: 0x6aad94fa, atime: 0x0, ctime: 0x6aad94fa, blocks: 0xd007 00010000:00010000:2.0:1789760762.734746:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2f1d5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.734763:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=742s v=(0 0) 00010000:00010000:2.0:1789760762.734767:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760762.734775:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2f1d5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26464256->26468351] (req 26464256->26468351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.734789:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2f1d5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26464256->26468351] (req 26464256->26468351) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760762.768460:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26464256 -> 26468352 00010000:00010000:0.0:1789760762.768480:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2f1d5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26464256->26468351] (req 26464256->26468351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.768493:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2f1d5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26464256->26468351] (req 26464256->26468351) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.768502:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2f1d5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26464256->26468351] (req 26464256->26468351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.768510:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760762.768516:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2f1d5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26464256->26468351] (req 26464256->26468351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.768526:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2f1d5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26464256->26468351] (req 26464256->26468351) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.768552:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2f1d5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26464256->26468351] (req 26464256->26468351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.768559:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2f1d5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26464256->26468351] (req 26464256->26468351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760762.768763:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febfb374380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2361:x1876699512689152:12345-192.168.203.56@tcp:4:dd.0 Request processed in 34100us (34490us total) trans 4294973780 rc 0/0 00000100:00100000:0.0:1789760762.768774:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6491 00000100:00100000:0.0:1789760762.768784:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 742257470817 time 742298944031 state 0x2 ref 0 00000001:00080000:3.0:1789760762.769008:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973780 is committed 00000100:00100000:2.0:1789760762.772219:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760762.772336:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512689280 00000100:00100000:1.0:1789760762.772346:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512689280 00000100:00100000:1.0:1789760762.772399:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 742257470817 time 742298944031 state 0x0 ref 1 00000100:00100000:1.0:1789760762.772414:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 742299204447 time 742298944031 state 0x1 ref 1 00000100:00100000:1.0:1789760762.772484:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 742299204447 time 742336782775 state 0x0 ref 1 00000100:00100000:1.0:1789760762.772497:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2fb8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512689280:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760762.772546:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26468352, mtime: 0x6aad94fa, atime: 0x0, ctime: 0x6aad94fa, blocks: 0xd007 00010000:00010000:1.0:1789760762.772555:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2f1dc lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.772637:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=742s v=(0 0) 00010000:00010000:1.0:1789760762.772643:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760762.772649:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2f1dc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26468352->26472447] (req 26468352->26472447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.772663:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2f1dc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26468352->26472447] (req 26468352->26472447) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760762.802555:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26468352 -> 26472448 00010000:00010000:3.0:1789760762.802588:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2f1dc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26468352->26472447] (req 26468352->26472447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.802602:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2f1dc lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26468352->26472447] (req 26468352->26472447) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.802611:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2f1dc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26468352->26472447] (req 26468352->26472447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.802618:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760762.802623:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2f1dc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26468352->26472447] (req 26468352->26472447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.802632:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2f1dc lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26468352->26472447] (req 26468352->26472447) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.802649:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2f1dc lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26468352->26472447] (req 26468352->26472447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.802656:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2f1dc lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26468352->26472447] (req 26468352->26472447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760762.802778:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2fb8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2361:x1876699512689280:12345-192.168.203.56@tcp:4:dd.0 Request processed in 30280us (30563us total) trans 4294973781 rc 0/0 00000100:00100000:3.0:1789760762.802788:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6492 00000100:00100000:3.0:1789760762.802797:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 742299204447 time 742336782775 state 0x2 ref 0 00000001:00080000:0.0:1789760762.803992:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973781 is committed 00000100:00100000:2.0:1789760762.807299:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760762.807564:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512689408 00000100:00100000:1.0:1789760762.807585:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512689408 00000100:00100000:1.0:1789760762.807639:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 742299204447 time 742336782775 state 0x0 ref 1 00000100:00100000:1.0:1789760762.807653:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 742337043191 time 742336782775 state 0x1 ref 1 00000100:00100000:1.0:1789760762.807687:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 742337043191 time 742371986692 state 0x0 ref 1 00000100:00100000:1.0:1789760762.807734:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febe7907100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512689408:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760762.807781:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26472448, mtime: 0x6aad94fa, atime: 0x0, ctime: 0x6aad94fa, blocks: 0xd007 00010000:00010000:1.0:1789760762.807789:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2f1e3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.807806:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=742s v=(0 0) 00010000:00010000:1.0:1789760762.807811:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760762.807817:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2f1e3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26472448->26476543] (req 26472448->26476543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.807832:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2f1e3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26472448->26476543] (req 26472448->26476543) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760762.835648:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973782 is committed 00002000:00010000:3.0:1789760762.835757:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26472448 -> 26476544 00010000:00010000:3.0:1789760762.835773:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2f1e3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26472448->26476543] (req 26472448->26476543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.835786:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2f1e3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26472448->26476543] (req 26472448->26476543) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.835796:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2f1e3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26472448->26476543] (req 26472448->26476543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.835803:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760762.835808:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2f1e3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26472448->26476543] (req 26472448->26476543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.835818:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2f1e3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26472448->26476543] (req 26472448->26476543) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.835835:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2f1e3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26472448->26476543] (req 26472448->26476543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.835842:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2f1e3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26472448->26476543] (req 26472448->26476543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760762.835935:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febe7907100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512689408:12345-192.168.203.56@tcp:4:dd.0 Request processed in 28199us (28640us total) trans 4294973782 rc 0/0 00000100:00100000:3.0:1789760762.835944:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6493 00000100:00100000:3.0:1789760762.835954:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 742337043191 time 742371986692 state 0x2 ref 0 00000100:00100000:2.0:1789760762.839239:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760762.839321:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512689536 00000100:00100000:1.0:1789760762.839331:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512689536 00000100:00100000:1.0:1789760762.839379:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 742337043191 time 742371986692 state 0x0 ref 1 00000100:00100000:1.0:1789760762.839394:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 742372247108 time 742371986692 state 0x1 ref 1 00000100:00100000:1.0:1789760762.839421:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 742372247108 time 742403721507 state 0x0 ref 1 00000100:00100000:1.0:1789760762.839433:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4130000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512689536:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760762.839476:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26476544, mtime: 0x6aad94fa, atime: 0x0, ctime: 0x6aad94fa, blocks: 0xd007 00010000:00010000:1.0:1789760762.839485:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2f1ea lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.839501:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=742s v=(0 0) 00010000:00010000:1.0:1789760762.839505:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760762.839511:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2f1ea lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26476544->26480639] (req 26476544->26480639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.839524:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2f1ea lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26476544->26480639] (req 26476544->26480639) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760762.869820:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26476544 -> 26480640 00010000:00010000:0.0:1789760762.869838:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2f1ea lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26476544->26480639] (req 26476544->26480639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.869863:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2f1ea lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26476544->26480639] (req 26476544->26480639) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.869872:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2f1ea lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26476544->26480639] (req 26476544->26480639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.869879:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760762.869884:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2f1ea lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26476544->26480639] (req 26476544->26480639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.869899:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2f1ea lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26476544->26480639] (req 26476544->26480639) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.869917:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2f1ea lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26476544->26480639] (req 26476544->26480639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.869926:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2f1ea lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26476544->26480639] (req 26476544->26480639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760762.870020:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4130000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2361:x1876699512689536:12345-192.168.203.56@tcp:4:dd.0 Request processed in 30583us (30784us total) trans 4294973783 rc 0/0 00000100:00100000:0.0:1789760762.870031:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6494 00000100:00100000:0.0:1789760762.870041:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 742372247108 time 742403721507 state 0x2 ref 0 00000001:00080000:2.0:1789760762.870613:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973783 is committed 00000100:00100000:1.0:1789760762.874033:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760762.874115:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512689664 00000100:00100000:3.0:1789760762.874125:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512689664 00000100:00100000:3.0:1789760762.874192:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 742372247108 time 742403721507 state 0x0 ref 1 00000100:00100000:3.0:1789760762.874207:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 742403981923 time 742403721507 state 0x1 ref 1 00000100:00100000:3.0:1789760762.874238:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 742403981923 time 742438537719 state 0x0 ref 1 00000100:00100000:3.0:1789760762.874250:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4039880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512689664:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760762.874297:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26480640, mtime: 0x6aad94fa, atime: 0x0, ctime: 0x6aad94fa, blocks: 0xd007 00010000:00010000:3.0:1789760762.874306:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2f1f1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.874321:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=742s v=(0 0) 00010000:00010000:3.0:1789760762.874326:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760762.874333:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2f1f1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26480640->26484735] (req 26480640->26484735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.874347:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2f1f1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26480640->26484735] (req 26480640->26484735) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760762.896166:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26480640 -> 26484736 00010000:00010000:0.0:1789760762.896197:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2f1f1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26480640->26484735] (req 26480640->26484735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.896211:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2f1f1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26480640->26484735] (req 26480640->26484735) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.896221:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2f1f1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26480640->26484735] (req 26480640->26484735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.896228:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760762.896233:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2f1f1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26480640->26484735] (req 26480640->26484735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.896246:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2f1f1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26480640->26484735] (req 26480640->26484735) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.896264:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2f1f1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26480640->26484735] (req 26480640->26484735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.896271:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2f1f1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26480640->26484735] (req 26480640->26484735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760762.896370:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4039880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2360:x1876699512689664:12345-192.168.203.56@tcp:4:dd.0 Request processed in 22117us (22340us total) trans 4294973784 rc 0/0 00000100:00100000:0.0:1789760762.896381:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6495 00000100:00100000:0.0:1789760762.896391:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 742403981923 time 742438537719 state 0x2 ref 0 00000001:00080000:2.0:1789760762.896912:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973784 is committed 00000100:00100000:3.0:1789760762.900528:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760762.900836:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512689792 00000100:00100000:1.0:1789760762.900856:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512689792 00000100:00100000:1.0:1789760762.900908:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 742403981923 time 742438537719 state 0x0 ref 1 00000100:00100000:1.0:1789760762.900924:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 742438798135 time 742438537719 state 0x1 ref 1 00000100:00100000:1.0:1789760762.900957:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 742438798135 time 742465256678 state 0x0 ref 1 00000100:00100000:1.0:1789760762.900969:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2747800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512689792:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760762.901015:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26484736, mtime: 0x6aad94fa, atime: 0x0, ctime: 0x6aad94fa, blocks: 0xd007 00010000:00010000:1.0:1789760762.901024:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2f1f8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.901040:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=742s v=(0 0) 00010000:00010000:1.0:1789760762.901045:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760762.901051:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2f1f8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26484736->26488831] (req 26484736->26488831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.901065:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2f1f8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26484736->26488831] (req 26484736->26488831) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760762.972692:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973785 is committed 00002000:00010000:0.0:1789760762.972988:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26484736 -> 26488832 00010000:00010000:0.0:1789760762.973006:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2f1f8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26484736->26488831] (req 26484736->26488831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.973020:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2f1f8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26484736->26488831] (req 26484736->26488831) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.973028:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2f1f8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26484736->26488831] (req 26484736->26488831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.973041:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760762.973047:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2f1f8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26484736->26488831] (req 26484736->26488831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.973056:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2f1f8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26484736->26488831] (req 26484736->26488831) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.973078:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2f1f8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26484736->26488831] (req 26484736->26488831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.973086:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2f1f8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26484736->26488831] (req 26484736->26488831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760762.973191:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2747800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512689792:12345-192.168.203.56@tcp:4:dd.0 Request processed in 72219us (72668us total) trans 4294973785 rc 0/0 00000100:00100000:0.0:1789760762.973201:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6496 00000100:00100000:0.0:1789760762.973211:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 742438798135 time 742465256678 state 0x2 ref 0 00000100:00100000:3.0:1789760762.978222:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760762.979824:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512689920 00000100:00100000:0.0:1789760762.979836:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512689920 00000100:00100000:1.0:1789760762.980031:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 742438798135 time 742465256678 state 0x0 ref 1 00000100:00100000:1.0:1789760762.980047:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 742465517094 time 742465256678 state 0x1 ref 1 00000100:00100000:1.0:1789760762.980126:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 742465517094 time 742544425036 state 0x0 ref 1 00000100:00100000:1.0:1789760762.980140:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2745500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512689920:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760762.980196:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26488832, mtime: 0x6aad94fa, atime: 0x0, ctime: 0x6aad94fa, blocks: 0xd007 00010000:00010000:1.0:1789760762.980206:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2f1ff lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.980223:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=742s v=(0 0) 00010000:00010000:1.0:1789760762.980228:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760762.980234:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2f1ff lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26488832->26492927] (req 26488832->26492927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.980248:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2f1ff lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26488832->26492927] (req 26488832->26492927) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760763.007544:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26488832 -> 26492928 00010000:00010000:1.0:1789760763.007565:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2f1ff lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26488832->26492927] (req 26488832->26492927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.007578:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2f1ff lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26488832->26492927] (req 26488832->26492927) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.007587:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2f1ff lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26488832->26492927] (req 26488832->26492927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.007594:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760763.007600:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2f1ff lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26488832->26492927] (req 26488832->26492927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.007609:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2f1ff lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26488832->26492927] (req 26488832->26492927) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.007629:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2f1ff lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26488832->26492927] (req 26488832->26492927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.007637:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2f1ff lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26488832->26492927] (req 26488832->26492927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760763.007787:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973786 is committed 00000100:00100000:1.0:1789760763.008021:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2745500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512689920:12345-192.168.203.56@tcp:4:dd.0 Request processed in 27879us (29806us total) trans 4294973786 rc 0/0 00000100:00100000:1.0:1789760763.008033:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6497 00000100:00100000:1.0:1789760763.008044:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 742465517094 time 742544425036 state 0x2 ref 0 00000100:00100000:1.0:1789760763.012916:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760763.013061:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512690048 00000100:00100000:0.0:1789760763.013071:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512690048 00000100:00100000:0.0:1789760763.013135:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 742465517094 time 742544425036 state 0x0 ref 1 00000100:00100000:0.0:1789760763.013151:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 742544685452 time 742544425036 state 0x1 ref 1 00000100:00100000:0.0:1789760763.013188:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 742544685452 time 742577488038 state 0x0 ref 1 00000100:00100000:0.0:1789760763.013202:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4039180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512690048:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760763.013249:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26492928, mtime: 0x6aad94fa, atime: 0x0, ctime: 0x6aad94fa, blocks: 0xd007 00010000:00010000:0.0:1789760763.013259:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2f206 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.013274:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=742s v=(0 0) 00010000:00010000:0.0:1789760763.013278:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760763.013284:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2f206 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26492928->26497023] (req 26492928->26497023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.013297:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2f206 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26492928->26497023] (req 26492928->26497023) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:1.0:1789760763.013735:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00010000:0.0:1789760763.042098:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26492928 -> 26497024 00010000:00010000:0.0:1789760763.042118:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2f206 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26492928->26497023] (req 26492928->26497023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.042132:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2f206 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26492928->26497023] (req 26492928->26497023) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.042141:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2f206 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26492928->26497023] (req 26492928->26497023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.042148:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760763.042154:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2f206 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26492928->26497023] (req 26492928->26497023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.042163:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2f206 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26492928->26497023] (req 26492928->26497023) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.042190:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2f206 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26492928->26497023] (req 26492928->26497023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.042198:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2f206 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26492928->26497023] (req 26492928->26497023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760763.042283:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4039180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2359:x1876699512690048:12345-192.168.203.56@tcp:4:dd.0 Request processed in 29079us (29372us total) trans 4294973787 rc 0/0 00000100:00100000:0.0:1789760763.042293:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6498 00000100:00100000:0.0:1789760763.042303:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 742544685452 time 742577488038 state 0x2 ref 0 00000001:00080000:1.0:1789760763.042838:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973787 is committed 00000100:00100000:3.0:1789760763.045102:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760763.045182:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512690176 00000100:00100000:0.0:1789760763.045191:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512690176 00000100:00100000:0.0:1789760763.045240:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 742544685452 time 742577488038 state 0x0 ref 1 00000100:00100000:0.0:1789760763.045256:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 742577748454 time 742577488038 state 0x1 ref 1 00000100:00100000:0.0:1789760763.045289:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 742577748454 time 742609588928 state 0x0 ref 1 00000100:00100000:0.0:1789760763.045301:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac28a3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512690176:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760763.045349:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26497024, mtime: 0x6aad94fa, atime: 0x0, ctime: 0x6aad94fa, blocks: 0xd007 00010000:00010000:0.0:1789760763.045358:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2f20d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.045373:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=742s v=(0 0) 00010000:00010000:0.0:1789760763.045377:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760763.045383:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2f20d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26497024->26501119] (req 26497024->26501119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.045397:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2f20d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26497024->26501119] (req 26497024->26501119) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760763.108792:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973788 is committed 00002000:00010000:3.0:1789760763.108992:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26497024 -> 26501120 00010000:00010000:3.0:1789760763.109008:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2f20d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26497024->26501119] (req 26497024->26501119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.109021:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2f20d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26497024->26501119] (req 26497024->26501119) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.109030:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2f20d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26497024->26501119] (req 26497024->26501119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.109038:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760763.109043:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2f20d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26497024->26501119] (req 26497024->26501119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.109051:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2f20d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26497024->26501119] (req 26497024->26501119) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.109070:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2f20d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26497024->26501119] (req 26497024->26501119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.109078:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2f20d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26497024->26501119] (req 26497024->26501119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760763.109167:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac28a3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512690176:12345-192.168.203.56@tcp:4:dd.0 Request processed in 63863us (64068us total) trans 4294973788 rc 0/0 00000100:00100000:3.0:1789760763.109186:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6499 00000100:00100000:3.0:1789760763.109196:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 742577748454 time 742609588928 state 0x2 ref 0 00000100:00100000:3.0:1789760763.115731:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760763.115925:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512690304 00000100:00100000:1.0:1789760763.115935:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512690304 00000100:00100000:1.0:1789760763.115989:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 742577748454 time 742609588928 state 0x0 ref 1 00000100:00100000:1.0:1789760763.116004:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 742609849344 time 742609588928 state 0x1 ref 1 00000100:00100000:1.0:1789760763.116034:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 742609849344 time 742680333842 state 0x0 ref 1 00000100:00100000:1.0:1789760763.116046:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febffb9f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512690304:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760763.116098:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26501120, mtime: 0x6aad94fa, atime: 0x0, ctime: 0x6aad94fa, blocks: 0xd007 00010000:00010000:1.0:1789760763.116108:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2f214 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.116125:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=742s v=(0 0) 00010000:00010000:1.0:1789760763.116129:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760763.116136:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2f214 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26501120->26505215] (req 26501120->26505215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.116151:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2f214 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26501120->26505215] (req 26501120->26505215) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760763.173692:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973789 is committed 00002000:00010000:3.0:1789760763.173811:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26501120 -> 26505216 00010000:00010000:3.0:1789760763.173826:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2f214 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26501120->26505215] (req 26501120->26505215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.173840:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2f214 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26501120->26505215] (req 26501120->26505215) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.173849:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2f214 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26501120->26505215] (req 26501120->26505215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.173861:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760763.173866:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2f214 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26501120->26505215] (req 26501120->26505215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.173875:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2f214 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26501120->26505215] (req 26501120->26505215) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.173893:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2f214 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26501120->26505215] (req 26501120->26505215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.173900:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2f214 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26501120->26505215] (req 26501120->26505215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760763.173995:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febffb9f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512690304:12345-192.168.203.56@tcp:4:dd.0 Request processed in 57945us (58300us total) trans 4294973789 rc 0/0 00000100:00100000:3.0:1789760763.174009:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6500 00000100:00100000:3.0:1789760763.174020:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 742609849344 time 742680333842 state 0x2 ref 0 00000100:00100000:1.0:1789760763.176654:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760763.176810:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512690432 00000100:00100000:0.0:1789760763.176819:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512690432 00000100:00100000:0.0:1789760763.176880:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 742609849344 time 742680333842 state 0x0 ref 1 00000100:00100000:0.0:1789760763.176972:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 742680594258 time 742680333842 state 0x1 ref 1 00000100:00100000:0.0:1789760763.177004:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 742680594258 time 742741303256 state 0x0 ref 1 00000100:00100000:0.0:1789760763.177017:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4039500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512690432:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760763.177062:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26505216, mtime: 0x6aad94fa, atime: 0x0, ctime: 0x6aad94fa, blocks: 0xd007 00010000:00010000:0.0:1789760763.177070:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2f21b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.177086:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=742s v=(0 0) 00010000:00010000:0.0:1789760763.177090:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760763.177097:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2f21b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26505216->26509311] (req 26505216->26509311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.177110:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2f21b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26505216->26509311] (req 26505216->26509311) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.251009:0:5836:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:2.0:1789760763.255818:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973790 is committed 00002000:00010000:3.0:1789760763.256027:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26505216 -> 26509312 00010000:00010000:3.0:1789760763.256042:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2f21b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26505216->26509311] (req 26505216->26509311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.256054:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2f21b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26505216->26509311] (req 26505216->26509311) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.256064:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2f21b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26505216->26509311] (req 26505216->26509311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.256072:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760763.256077:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2f21b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26505216->26509311] (req 26505216->26509311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.256086:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2f21b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26505216->26509311] (req 26505216->26509311) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.256104:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2f21b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26505216->26509311] (req 26505216->26509311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.256112:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2f21b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26505216->26509311] (req 26505216->26509311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760763.256247:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4039500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512690432:12345-192.168.203.56@tcp:4:dd.0 Request processed in 79229us (79597us total) trans 4294973790 rc 0/0 00000100:00100000:3.0:1789760763.256258:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6501 00000100:00100000:3.0:1789760763.256266:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 742680594258 time 742741303256 state 0x2 ref 0 00000100:00100000:3.0:1789760763.263068:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760763.263189:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512690560 00000100:00100000:0.0:1789760763.263199:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512690560 00000100:00100000:0.0:1789760763.263256:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 742680594258 time 742741303256 state 0x0 ref 1 00000100:00100000:0.0:1789760763.263271:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 742741563672 time 742741303256 state 0x1 ref 1 00000100:00100000:0.0:1789760763.263344:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 742741563672 time 742827642479 state 0x0 ref 1 00000100:00100000:0.0:1789760763.263359:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febffb9ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512690560:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760763.263421:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26509312, mtime: 0x6aad94fa, atime: 0x0, ctime: 0x6aad94fa, blocks: 0xd007 00010000:00010000:0.0:1789760763.263430:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2f222 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.263445:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=742s v=(0 0) 00010000:00010000:0.0:1789760763.263448:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760763.263454:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2f222 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26509312->26513407] (req 26509312->26513407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.263465:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2f222 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26509312->26513407] (req 26509312->26513407) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760763.301143:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973791 is committed 00002000:00010000:3.0:1789760763.301316:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26509312 -> 26513408 00010000:00010000:3.0:1789760763.301332:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2f222 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26509312->26513407] (req 26509312->26513407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.301346:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2f222 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26509312->26513407] (req 26509312->26513407) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.301356:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2f222 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26509312->26513407] (req 26509312->26513407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.301364:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760763.301369:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2f222 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26509312->26513407] (req 26509312->26513407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.301377:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2f222 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26509312->26513407] (req 26509312->26513407) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.301396:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2f222 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26509312->26513407] (req 26509312->26513407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.301410:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2f222 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26509312->26513407] (req 26509312->26513407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760763.301503:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febffb9ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512690560:12345-192.168.203.56@tcp:4:dd.0 Request processed in 38144us (38446us total) trans 4294973791 rc 0/0 00000100:00100000:3.0:1789760763.301513:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6502 00000100:00100000:3.0:1789760763.301523:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 742741563672 time 742827642479 state 0x2 ref 0 00000100:00100000:3.0:1789760763.305011:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760763.305093:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512690688 00000100:00100000:1.0:1789760763.305103:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512690688 00000100:00100000:1.0:1789760763.305160:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 742741563672 time 742827642479 state 0x0 ref 1 00000100:00100000:1.0:1789760763.305193:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 742827902895 time 742827642479 state 0x1 ref 1 00000100:00100000:1.0:1789760763.305226:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 742827902895 time 742869525141 state 0x0 ref 1 00000100:00100000:1.0:1789760763.305244:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febffb9c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512690688:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760763.305291:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26513408, mtime: 0x6aad94fa, atime: 0x0, ctime: 0x6aad94fa, blocks: 0xd007 00010000:00010000:1.0:1789760763.305300:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2f229 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.305317:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=742s v=(0 0) 00010000:00010000:1.0:1789760763.305321:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760763.305327:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2f229 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26513408->26517503] (req 26513408->26517503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.305341:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2f229 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26513408->26517503] (req 26513408->26517503) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760763.331490:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973792 is committed 00002000:00010000:3.0:1789760763.331564:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26513408 -> 26517504 00010000:00010000:3.0:1789760763.331579:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2f229 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26513408->26517503] (req 26513408->26517503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.331592:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2f229 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26513408->26517503] (req 26513408->26517503) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.331601:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2f229 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26513408->26517503] (req 26513408->26517503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.331609:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760763.331614:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2f229 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26513408->26517503] (req 26513408->26517503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.331624:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2f229 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26513408->26517503] (req 26513408->26517503) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.331648:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2f229 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26513408->26517503] (req 26513408->26517503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.331657:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2f229 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26513408->26517503] (req 26513408->26517503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760763.331802:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febffb9c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512690688:12345-192.168.203.56@tcp:4:dd.0 Request processed in 26561us (26796us total) trans 4294973792 rc 0/0 00000100:00100000:3.0:1789760763.331812:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6503 00000100:00100000:3.0:1789760763.331821:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 742827902895 time 742869525141 state 0x2 ref 0 00000100:00100000:1.0:1789760763.335802:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760763.335899:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512690816 00000100:00100000:3.0:1789760763.335909:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512690816 00000100:00100000:3.0:1789760763.335964:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 742827902895 time 742869525141 state 0x0 ref 1 00000100:00100000:3.0:1789760763.335979:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 742869785557 time 742869525141 state 0x1 ref 1 00000100:00100000:3.0:1789760763.336011:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 742869785557 time 742900310762 state 0x0 ref 1 00000100:00100000:3.0:1789760763.336024:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4038a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512690816:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760763.336070:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26517504, mtime: 0x6aad94fa, atime: 0x0, ctime: 0x6aad94fa, blocks: 0xd007 00010000:00010000:3.0:1789760763.336079:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2f230 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.336249:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=742s v=(0 0) 00010000:00010000:3.0:1789760763.336254:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760763.336261:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2f230 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26517504->26521599] (req 26517504->26521599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.336275:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2f230 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26517504->26521599] (req 26517504->26521599) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760763.391153:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973793 is committed 00002000:00010000:1.0:1789760763.391309:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26517504 -> 26521600 00010000:00010000:1.0:1789760763.391323:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2f230 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26517504->26521599] (req 26517504->26521599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.391337:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2f230 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26517504->26521599] (req 26517504->26521599) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.391347:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2f230 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26517504->26521599] (req 26517504->26521599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.391356:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760763.391361:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2f230 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26517504->26521599] (req 26517504->26521599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.391370:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2f230 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26517504->26521599] (req 26517504->26521599) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.391390:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2f230 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26517504->26521599] (req 26517504->26521599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.391397:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2f230 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26517504->26521599] (req 26517504->26521599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760763.391489:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4038a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512690816:12345-192.168.203.56@tcp:4:dd.0 Request processed in 55462us (55690us total) trans 4294973793 rc 0/0 00000100:00100000:1.0:1789760763.391499:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6504 00000100:00100000:1.0:1789760763.391509:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 742869785557 time 742900310762 state 0x2 ref 0 00000100:00100000:0.0:1789760763.394508:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760763.394665:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512690944 00000100:00100000:3.0:1789760763.394675:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512690944 00000100:00100000:0.0:1789760763.394975:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 742869785557 time 742900310762 state 0x0 ref 1 00000100:00100000:0.0:1789760763.394990:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 742900571178 time 742900310762 state 0x1 ref 1 00000100:00100000:0.0:1789760763.395073:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 742900571178 time 742959371769 state 0x0 ref 1 00000100:00100000:0.0:1789760763.395088:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac287a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512690944:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760763.395133:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26521600, mtime: 0x6aad94fa, atime: 0x0, ctime: 0x6aad94fa, blocks: 0xd007 00010000:00010000:0.0:1789760763.395141:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2f237 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.395157:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=742s v=(0 0) 00010000:00010000:0.0:1789760763.395161:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760763.395167:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2f237 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26521600->26525695] (req 26521600->26525695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.395195:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2f237 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26521600->26525695] (req 26521600->26525695) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760763.458830:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26521600 -> 26525696 00010000:00010000:1.0:1789760763.458858:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2f237 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26521600->26525695] (req 26521600->26525695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.458872:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2f237 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26521600->26525695] (req 26521600->26525695) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.458887:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2f237 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26521600->26525695] (req 26521600->26525695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.458893:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760763.458898:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2f237 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26521600->26525695] (req 26521600->26525695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.458907:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2f237 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26521600->26525695] (req 26521600->26525695) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.458927:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2f237 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26521600->26525695] (req 26521600->26525695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.458945:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2f237 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26521600->26525695] (req 26521600->26525695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760763.459322:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac287a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2360:x1876699512690944:12345-192.168.203.56@tcp:4:dd.0 Request processed in 64233us (64819us total) trans 4294973794 rc 0/0 00000100:00100000:1.0:1789760763.459334:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6505 00000100:00100000:1.0:1789760763.459344:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 742900571178 time 742959371769 state 0x2 ref 0 00000001:00080000:3.0:1789760763.459767:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973794 is committed 00000100:00100000:2.0:1789760763.464591:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760763.464666:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512691072 00000100:00100000:3.0:1789760763.464676:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512691072 00000100:00100000:3.0:1789760763.464756:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 742900571178 time 742959371769 state 0x0 ref 1 00000100:00100000:3.0:1789760763.464771:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 742959632185 time 742959371769 state 0x1 ref 1 00000100:00100000:3.0:1789760763.464798:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 742959632185 time 743029097326 state 0x0 ref 1 00000100:00100000:3.0:1789760763.464810:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4133480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512691072:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760763.464855:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26525696, mtime: 0x6aad94fa, atime: 0x0, ctime: 0x6aad94fa, blocks: 0xd007 00010000:00010000:3.0:1789760763.464863:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2f23e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.464878:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=743s v=(0 0) 00010000:00010000:3.0:1789760763.464882:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760763.464889:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2f23e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26525696->26529791] (req 26525696->26529791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.464909:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2f23e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26525696->26529791] (req 26525696->26529791) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760763.521594:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26525696 -> 26529792 00010000:00010000:3.0:1789760763.521614:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2f23e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26525696->26529791] (req 26525696->26529791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.521627:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2f23e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26525696->26529791] (req 26525696->26529791) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.521636:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2f23e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26525696->26529791] (req 26525696->26529791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.521643:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760763.521648:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2f23e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26525696->26529791] (req 26525696->26529791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.521657:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2f23e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26525696->26529791] (req 26525696->26529791) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.521674:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2f23e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26525696->26529791] (req 26525696->26529791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.521682:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2f23e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26525696->26529791] (req 26525696->26529791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760763.521800:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4133480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2358:x1876699512691072:12345-192.168.203.56@tcp:4:dd.0 Request processed in 56987us (57213us total) trans 4294973795 rc 0/0 00000100:00100000:3.0:1789760763.521811:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6506 00000100:00100000:3.0:1789760763.521820:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 742959632185 time 743029097326 state 0x2 ref 0 00000001:00080000:1.0:1789760763.522433:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973795 is committed 00000100:00100000:2.0:1789760763.527852:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760763.528109:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512691200 00000100:00100000:1.0:1789760763.528122:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512691200 00000100:00100000:1.0:1789760763.528188:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 742959632185 time 743029097326 state 0x0 ref 1 00000100:00100000:1.0:1789760763.528205:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 743029357742 time 743029097326 state 0x1 ref 1 00000100:00100000:1.0:1789760763.528233:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 743029357742 time 743092532837 state 0x0 ref 1 00000100:00100000:1.0:1789760763.528246:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4130700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512691200:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760763.528294:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26529792, mtime: 0x6aad94fa, atime: 0x0, ctime: 0x6aad94fa, blocks: 0xd007 00010000:00010000:1.0:1789760763.528304:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2f245 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.528322:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=743s v=(0 0) 00010000:00010000:1.0:1789760763.528326:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760763.528332:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2f245 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26529792->26533887] (req 26529792->26533887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.528346:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2f245 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26529792->26533887] (req 26529792->26533887) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760763.563589:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26529792 -> 26533888 00002000:00010000:1.0:1789760763.563601:0:13063:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760762 -> 1789760763 00002000:00010000:1.0:1789760763.563603:0:13063:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760762 -> 1789760763 00010000:00010000:1.0:1789760763.563618:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2f245 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26529792->26533887] (req 26529792->26533887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.563631:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2f245 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26529792->26533887] (req 26529792->26533887) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.563645:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2f245 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26529792->26533887] (req 26529792->26533887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.563653:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760763.563658:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2f245 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26529792->26533887] (req 26529792->26533887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.563667:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2f245 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26529792->26533887] (req 26529792->26533887) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.563685:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2f245 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26529792->26533887] (req 26529792->26533887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.563693:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2f245 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26529792->26533887] (req 26529792->26533887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760763.563806:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4130700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2361:x1876699512691200:12345-192.168.203.56@tcp:4:dd.0 Request processed in 35557us (35969us total) trans 4294973796 rc 0/0 00000100:00100000:1.0:1789760763.563817:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6507 00000100:00100000:1.0:1789760763.563828:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 743029357742 time 743092532837 state 0x2 ref 0 00000001:00080000:3.0:1789760763.564135:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973796 is committed 00000100:00100000:0.0:1789760763.567201:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760763.567301:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512691328 00000100:00100000:1.0:1789760763.567311:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512691328 00000100:00100000:1.0:1789760763.567362:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 743029357742 time 743092532837 state 0x0 ref 1 00000100:00100000:1.0:1789760763.567377:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 743092793253 time 743092532837 state 0x1 ref 1 00000100:00100000:1.0:1789760763.567402:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 743092793253 time 743131702011 state 0x0 ref 1 00000100:00100000:1.0:1789760763.567413:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2879c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512691328:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760763.567459:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26533888, mtime: 0x6aad94fb, atime: 0x0, ctime: 0x6aad94fb, blocks: 0xd007 00010000:00010000:1.0:1789760763.567467:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2f24c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.567483:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=743s v=(0 0) 00010000:00010000:1.0:1789760763.567487:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760763.567493:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2f24c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26533888->26537983] (req 26533888->26537983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.567506:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2f24c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26533888->26537983] (req 26533888->26537983) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760763.599327:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973797 is committed 00002000:00010000:1.0:1789760763.599524:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26533888 -> 26537984 00010000:00010000:1.0:1789760763.599542:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2f24c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26533888->26537983] (req 26533888->26537983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.599556:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2f24c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26533888->26537983] (req 26533888->26537983) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.599642:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2f24c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26533888->26537983] (req 26533888->26537983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.599652:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760763.599659:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2f24c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26533888->26537983] (req 26533888->26537983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.599669:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2f24c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26533888->26537983] (req 26533888->26537983) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.599690:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2f24c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26533888->26537983] (req 26533888->26537983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.599737:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2f24c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26533888->26537983] (req 26533888->26537983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760763.599950:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2879c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512691328:12345-192.168.203.56@tcp:4:dd.0 Request processed in 32533us (32752us total) trans 4294973797 rc 0/0 00000100:00100000:1.0:1789760763.599961:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6508 00000100:00100000:1.0:1789760763.599972:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 743092793253 time 743131702011 state 0x2 ref 0 00000100:00100000:0.0:1789760763.605415:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760763.605540:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512691456 00000100:00100000:3.0:1789760763.605552:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512691456 00000100:00100000:3.0:1789760763.605684:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 743092793253 time 743131702011 state 0x0 ref 1 00000100:00100000:3.0:1789760763.605805:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 743131962427 time 743131702011 state 0x1 ref 1 00000100:00100000:3.0:1789760763.605838:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 743131962427 time 743170137660 state 0x0 ref 1 00000100:00100000:3.0:1789760763.605951:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2878e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512691456:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760763.606107:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26537984, mtime: 0x6aad94fb, atime: 0x0, ctime: 0x6aad94fb, blocks: 0xd007 00010000:00010000:3.0:1789760763.606117:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2f253 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.606132:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=743s v=(0 0) 00010000:00010000:3.0:1789760763.606137:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760763.606143:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2f253 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26537984->26542079] (req 26537984->26542079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.606156:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2f253 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26537984->26542079] (req 26537984->26542079) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760763.641377:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973798 is committed 00002000:00010000:3.0:1789760763.641476:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26537984 -> 26542080 00010000:00010000:3.0:1789760763.641490:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2f253 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26537984->26542079] (req 26537984->26542079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.641502:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2f253 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26537984->26542079] (req 26537984->26542079) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.641511:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2f253 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26537984->26542079] (req 26537984->26542079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.641518:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760763.641524:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2f253 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26537984->26542079] (req 26537984->26542079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.641533:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2f253 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26537984->26542079] (req 26537984->26542079) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.641553:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2f253 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26537984->26542079] (req 26537984->26542079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.641630:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2f253 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26537984->26542079] (req 26537984->26542079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760763.641832:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2878e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512691456:12345-192.168.203.56@tcp:4:dd.0 Request processed in 35879us (36422us total) trans 4294973798 rc 0/0 00000100:00100000:3.0:1789760763.641901:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6509 00000100:00100000:3.0:1789760763.641911:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 743131962427 time 743170137660 state 0x2 ref 0 00000100:00100000:2.0:1789760763.645079:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760763.645163:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512691584 00000100:00100000:1.0:1789760763.645180:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512691584 00000100:00100000:1.0:1789760763.645227:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 743131962427 time 743170137660 state 0x0 ref 1 00000100:00100000:1.0:1789760763.645242:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 743170398076 time 743170137660 state 0x1 ref 1 00000100:00100000:1.0:1789760763.645268:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 743170398076 time 743209568006 state 0x0 ref 1 00000100:00100000:1.0:1789760763.645280:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4131180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512691584:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760763.645325:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26542080, mtime: 0x6aad94fb, atime: 0x0, ctime: 0x6aad94fb, blocks: 0xd007 00010000:00010000:1.0:1789760763.645335:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2f25a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.645350:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=743s v=(0 0) 00010000:00010000:1.0:1789760763.645354:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760763.645361:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2f25a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26542080->26546175] (req 26542080->26546175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.645374:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2f25a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26542080->26546175] (req 26542080->26546175) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760763.673675:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973799 is committed 00002000:00010000:0.0:1789760763.673786:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26542080 -> 26546176 00010000:00010000:0.0:1789760763.673802:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2f25a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26542080->26546175] (req 26542080->26546175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.673816:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2f25a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26542080->26546175] (req 26542080->26546175) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.673950:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2f25a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26542080->26546175] (req 26542080->26546175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.673959:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760763.673964:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2f25a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26542080->26546175] (req 26542080->26546175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.673974:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2f25a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26542080->26546175] (req 26542080->26546175) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.673995:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2f25a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26542080->26546175] (req 26542080->26546175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.674003:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2f25a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26542080->26546175] (req 26542080->26546175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760763.674137:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4131180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512691584:12345-192.168.203.56@tcp:4:dd.0 Request processed in 28854us (29061us total) trans 4294973799 rc 0/0 00000100:00100000:0.0:1789760763.674148:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6510 00000100:00100000:0.0:1789760763.674157:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 743170398076 time 743209568006 state 0x2 ref 0 00000100:00100000:1.0:1789760763.677527:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760763.677680:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512691712 00000100:00100000:0.0:1789760763.677689:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512691712 00000100:00100000:0.0:1789760763.677815:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 743170398076 time 743209568006 state 0x0 ref 1 00000100:00100000:0.0:1789760763.677993:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 743209828422 time 743209568006 state 0x1 ref 1 00000100:00100000:0.0:1789760763.678076:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 743209828422 time 743242375140 state 0x0 ref 1 00000100:00100000:0.0:1789760763.678090:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac403ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512691712:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760763.678139:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26546176, mtime: 0x6aad94fb, atime: 0x0, ctime: 0x6aad94fb, blocks: 0xd007 00010000:00010000:0.0:1789760763.678148:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2f261 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.678164:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=743s v=(0 0) 00010000:00010000:0.0:1789760763.678181:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760763.678189:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2f261 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26546176->26550271] (req 26546176->26550271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.678203:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2f261 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26546176->26550271] (req 26546176->26550271) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760763.713958:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973800 is committed 00002000:00010000:2.0:1789760763.714099:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26546176 -> 26550272 00010000:00010000:2.0:1789760763.714113:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2f261 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26546176->26550271] (req 26546176->26550271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.714126:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2f261 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26546176->26550271] (req 26546176->26550271) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.714136:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2f261 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26546176->26550271] (req 26546176->26550271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.714144:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760763.714150:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2f261 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26546176->26550271] (req 26546176->26550271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.714160:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2f261 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26546176->26550271] (req 26546176->26550271) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.714191:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2f261 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26546176->26550271] (req 26546176->26550271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.714200:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2f261 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26546176->26550271] (req 26546176->26550271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760763.714288:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac403ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512691712:12345-192.168.203.56@tcp:4:dd.0 Request processed in 36196us (36765us total) trans 4294973800 rc 0/0 00000100:00100000:2.0:1789760763.714299:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6511 00000100:00100000:2.0:1789760763.714308:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 743209828422 time 743242375140 state 0x2 ref 0 00000100:00100000:2.0:1789760763.716966:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760763.717273:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512691840 00000100:00100000:3.0:1789760763.717283:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512691840 00000100:00100000:3.0:1789760763.717336:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 743209828422 time 743242375140 state 0x0 ref 1 00000100:00100000:3.0:1789760763.717351:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 743242635556 time 743242375140 state 0x1 ref 1 00000100:00100000:3.0:1789760763.717382:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 743242635556 time 743281681321 state 0x0 ref 1 00000100:00100000:3.0:1789760763.717395:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4130a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512691840:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760763.717441:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26550272, mtime: 0x6aad94fb, atime: 0x0, ctime: 0x6aad94fb, blocks: 0xd007 00010000:00010000:3.0:1789760763.717450:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2f268 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.717467:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=743s v=(0 0) 00010000:00010000:3.0:1789760763.717472:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760763.717478:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2f268 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26550272->26554367] (req 26550272->26554367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.717493:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2f268 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26550272->26554367] (req 26550272->26554367) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760763.749523:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973801 is committed 00002000:00010000:2.0:1789760763.749690:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26550272 -> 26554368 00010000:00010000:2.0:1789760763.749730:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2f268 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26550272->26554367] (req 26550272->26554367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.749744:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2f268 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26550272->26554367] (req 26550272->26554367) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.749760:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2f268 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26550272->26554367] (req 26550272->26554367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.749768:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760763.749773:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2f268 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26550272->26554367] (req 26550272->26554367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.749781:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2f268 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26550272->26554367] (req 26550272->26554367) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.749801:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2f268 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26550272->26554367] (req 26550272->26554367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.749810:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2f268 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26550272->26554367] (req 26550272->26554367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760763.749899:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4130a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512691840:12345-192.168.203.56@tcp:4:dd.0 Request processed in 32502us (32937us total) trans 4294973801 rc 0/0 00000100:00100000:2.0:1789760763.749909:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6512 00000100:00100000:2.0:1789760763.749919:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 743242635556 time 743281681321 state 0x2 ref 0 00000100:00100000:3.0:1789760763.752833:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760763.752979:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512691968 00000100:00100000:0.0:1789760763.752989:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512691968 00000100:00100000:0.0:1789760763.753044:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 743242635556 time 743281681321 state 0x0 ref 1 00000100:00100000:0.0:1789760763.753060:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 743281941737 time 743281681321 state 0x1 ref 1 00000100:00100000:0.0:1789760763.753096:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 743281941737 time 743317395614 state 0x0 ref 1 00000100:00100000:0.0:1789760763.753109:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febffb9c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512691968:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760763.753160:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26554368, mtime: 0x6aad94fb, atime: 0x0, ctime: 0x6aad94fb, blocks: 0xd007 00010000:00010000:0.0:1789760763.753180:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2f26f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.753197:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=743s v=(0 0) 00010000:00010000:0.0:1789760763.753202:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760763.753208:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2f26f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26554368->26558463] (req 26554368->26558463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.753222:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2f26f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26554368->26558463] (req 26554368->26558463) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760763.784059:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26554368 -> 26558464 00010000:00010000:0.0:1789760763.784078:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2f26f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26554368->26558463] (req 26554368->26558463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.784091:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2f26f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26554368->26558463] (req 26554368->26558463) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.784101:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2f26f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26554368->26558463] (req 26554368->26558463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.784108:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760763.784114:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2f26f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26554368->26558463] (req 26554368->26558463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.784125:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2f26f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26554368->26558463] (req 26554368->26558463) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.784145:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2f26f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26554368->26558463] (req 26554368->26558463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.784152:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2f26f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26554368->26558463] (req 26554368->26558463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760763.784257:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febffb9c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2361:x1876699512691968:12345-192.168.203.56@tcp:4:dd.0 Request processed in 31145us (31428us total) trans 4294973802 rc 0/0 00000100:00100000:0.0:1789760763.784267:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6513 00000100:00100000:0.0:1789760763.784277:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 743281941737 time 743317395614 state 0x2 ref 0 00000001:00080000:0.0:1789760763.784317:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973802 is committed 00000100:00100000:0.0:1789760763.787345:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760763.787438:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512692096 00000100:00100000:2.0:1789760763.787448:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512692096 00000100:00100000:2.0:1789760763.787503:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 743281941737 time 743317395614 state 0x0 ref 1 00000100:00100000:2.0:1789760763.787518:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 743317656030 time 743317395614 state 0x1 ref 1 00000100:00100000:2.0:1789760763.787552:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 743317656030 time 743351850908 state 0x0 ref 1 00000100:00100000:2.0:1789760763.787576:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac287b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512692096:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760763.787625:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26558464, mtime: 0x6aad94fb, atime: 0x0, ctime: 0x6aad94fb, blocks: 0xd007 00010000:00010000:2.0:1789760763.787634:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2f276 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.787651:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=743s v=(0 0) 00010000:00010000:2.0:1789760763.787655:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760763.787662:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2f276 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26558464->26562559] (req 26558464->26562559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.787675:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2f276 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26558464->26562559] (req 26558464->26562559) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760763.820503:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973803 is committed 00002000:00010000:0.0:1789760763.820623:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26558464 -> 26562560 00010000:00010000:0.0:1789760763.820640:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2f276 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26558464->26562559] (req 26558464->26562559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.820654:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2f276 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26558464->26562559] (req 26558464->26562559) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.820664:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2f276 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26558464->26562559] (req 26558464->26562559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.820673:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760763.820679:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2f276 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26558464->26562559] (req 26558464->26562559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.820688:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2f276 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26558464->26562559] (req 26558464->26562559) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.820743:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2f276 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26558464->26562559] (req 26558464->26562559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.820752:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2f276 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26558464->26562559] (req 26558464->26562559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760763.820849:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac287b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512692096:12345-192.168.203.56@tcp:4:dd.0 Request processed in 33270us (33497us total) trans 4294973803 rc 0/0 00000100:00100000:0.0:1789760763.820860:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6514 00000100:00100000:0.0:1789760763.820870:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 743317656030 time 743351850908 state 0x2 ref 0 00000100:00100000:2.0:1789760763.824287:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760763.824397:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512692224 00000100:00100000:3.0:1789760763.824408:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512692224 00000100:00100000:3.0:1789760763.824461:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 743317656030 time 743351850908 state 0x0 ref 1 00000100:00100000:3.0:1789760763.824477:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 743352111324 time 743351850908 state 0x1 ref 1 00000100:00100000:3.0:1789760763.824508:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 743352111324 time 743388807625 state 0x0 ref 1 00000100:00100000:3.0:1789760763.824521:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4132d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512692224:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760763.824679:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26562560, mtime: 0x6aad94fb, atime: 0x0, ctime: 0x6aad94fb, blocks: 0xd007 00010000:00010000:3.0:1789760763.824689:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2f27d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.824804:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=743s v=(0 0) 00010000:00010000:3.0:1789760763.824809:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760763.824816:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2f27d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26562560->26566655] (req 26562560->26566655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.824829:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2f27d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26562560->26566655] (req 26562560->26566655) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760763.851016:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973804 is committed 00002000:00010000:0.0:1789760763.851179:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26562560 -> 26566656 00010000:00010000:0.0:1789760763.851196:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2f27d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26562560->26566655] (req 26562560->26566655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.851209:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2f27d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26562560->26566655] (req 26562560->26566655) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.851224:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2f27d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26562560->26566655] (req 26562560->26566655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.851231:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760763.851236:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2f27d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26562560->26566655] (req 26562560->26566655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.851245:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2f27d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26562560->26566655] (req 26562560->26566655) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.851266:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2f27d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26562560->26566655] (req 26562560->26566655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.851350:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2f27d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26562560->26566655] (req 26562560->26566655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760763.851483:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4132d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512692224:12345-192.168.203.56@tcp:4:dd.0 Request processed in 26960us (27201us total) trans 4294973804 rc 0/0 00000100:00100000:0.0:1789760763.851494:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6515 00000100:00100000:0.0:1789760763.851502:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 743352111324 time 743388807625 state 0x2 ref 0 00000100:00100000:2.0:1789760763.857259:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760763.858117:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512692352 00000100:00100000:0.0:1789760763.858131:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512692352 00000100:00100000:0.0:1789760763.858198:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 743352111324 time 743388807625 state 0x0 ref 1 00000100:00100000:0.0:1789760763.858214:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 743389068041 time 743388807625 state 0x1 ref 1 00000100:00100000:0.0:1789760763.858249:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 743389068041 time 743422547837 state 0x0 ref 1 00000100:00100000:0.0:1789760763.858262:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4132300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512692352:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760763.858325:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26566656, mtime: 0x6aad94fb, atime: 0x0, ctime: 0x6aad94fb, blocks: 0xd007 00010000:00010000:0.0:1789760763.858335:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2f284 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.858352:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=743s v=(0 0) 00010000:00010000:0.0:1789760763.858356:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760763.858362:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2f284 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26566656->26570751] (req 26566656->26570751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.858377:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2f284 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26566656->26570751] (req 26566656->26570751) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760763.884001:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26566656 -> 26570752 00010000:00010000:0.0:1789760763.884020:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2f284 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26566656->26570751] (req 26566656->26570751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.884034:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2f284 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26566656->26570751] (req 26566656->26570751) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.884044:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2f284 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26566656->26570751] (req 26566656->26570751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.884062:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760763.884068:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2f284 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26566656->26570751] (req 26566656->26570751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.884078:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2f284 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26566656->26570751] (req 26566656->26570751) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.884104:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2f284 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26566656->26570751] (req 26566656->26570751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.884111:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2f284 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26566656->26570751] (req 26566656->26570751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760763.884116:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973805 is committed 00000100:00100000:0.0:1789760763.884216:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4132300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512692352:12345-192.168.203.56@tcp:4:dd.0 Request processed in 25952us (26961us total) trans 4294973805 rc 0/0 00000100:00100000:0.0:1789760763.884230:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6516 00000100:00100000:0.0:1789760763.884239:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 743389068041 time 743422547837 state 0x2 ref 0 00000100:00100000:0.0:1789760763.889875:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760763.890232:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512692480 00000100:00100000:2.0:1789760763.890243:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512692480 00000100:00100000:2.0:1789760763.890296:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 743389068041 time 743422547837 state 0x0 ref 1 00000100:00100000:2.0:1789760763.890311:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 743422808253 time 743422547837 state 0x1 ref 1 00000100:00100000:2.0:1789760763.891310:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 743422808253 time 743455606681 state 0x0 ref 1 00000100:00100000:2.0:1789760763.891330:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2878000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512692480:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760763.891383:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26570752, mtime: 0x6aad94fb, atime: 0x0, ctime: 0x6aad94fb, blocks: 0xd007 00010000:00010000:2.0:1789760763.891392:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2f28b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.891409:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=743s v=(0 0) 00010000:00010000:2.0:1789760763.891413:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760763.891419:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2f28b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26570752->26574847] (req 26570752->26574847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.891435:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2f28b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26570752->26574847] (req 26570752->26574847) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760763.917905:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26570752 -> 26574848 00010000:00010000:0.0:1789760763.917928:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2f28b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26570752->26574847] (req 26570752->26574847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.917941:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2f28b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26570752->26574847] (req 26570752->26574847) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760763.917943:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973806 is committed 00010000:00010000:0.0:1789760763.917951:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2f28b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26570752->26574847] (req 26570752->26574847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.917965:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760763.917970:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2f28b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26570752->26574847] (req 26570752->26574847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.917979:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2f28b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26570752->26574847] (req 26570752->26574847) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.917998:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2f28b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26570752->26574847] (req 26570752->26574847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.918005:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2f28b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26570752->26574847] (req 26570752->26574847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760763.918144:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2878000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512692480:12345-192.168.203.56@tcp:4:dd.0 Request processed in 26816us (28275us total) trans 4294973806 rc 0/0 00000100:00100000:0.0:1789760763.918159:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6517 00000100:00100000:0.0:1789760763.918179:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 743422808253 time 743455606681 state 0x2 ref 0 00000100:00100000:2.0:1789760763.920513:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760763.921040:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512692608 00000100:00100000:0.0:1789760763.921050:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512692608 00000100:00100000:0.0:1789760763.921101:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 743422808253 time 743455606681 state 0x0 ref 1 00000100:00100000:0.0:1789760763.921116:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 743455867097 time 743455606681 state 0x1 ref 1 00000100:00100000:0.0:1789760763.921191:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 743455867097 time 743485489729 state 0x0 ref 1 00000100:00100000:0.0:1789760763.921207:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4131f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512692608:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760763.921256:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26574848, mtime: 0x6aad94fb, atime: 0x0, ctime: 0x6aad94fb, blocks: 0xd007 00010000:00010000:0.0:1789760763.921265:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2f292 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.921281:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=743s v=(0 0) 00010000:00010000:0.0:1789760763.921286:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760763.921292:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2f292 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26574848->26578943] (req 26574848->26578943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.921305:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2f292 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26574848->26578943] (req 26574848->26578943) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760763.953583:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973807 is committed 00002000:00010000:0.0:1789760763.953920:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26574848 -> 26578944 00010000:00010000:0.0:1789760763.953935:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2f292 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26574848->26578943] (req 26574848->26578943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.953948:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2f292 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26574848->26578943] (req 26574848->26578943) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.953958:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2f292 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26574848->26578943] (req 26574848->26578943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.953965:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760763.953970:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2f292 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26574848->26578943] (req 26574848->26578943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.953979:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2f292 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26574848->26578943] (req 26574848->26578943) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.953998:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2f292 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26574848->26578943] (req 26574848->26578943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.954006:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2f292 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26574848->26578943] (req 26574848->26578943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760763.954143:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4131f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512692608:12345-192.168.203.56@tcp:4:dd.0 Request processed in 32933us (33632us total) trans 4294973807 rc 0/0 00000100:00100000:0.0:1789760763.954155:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6518 00000100:00100000:0.0:1789760763.954164:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 743455867097 time 743485489729 state 0x2 ref 0 00000100:00100000:2.0:1789760763.957165:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760763.957745:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512692736 00000100:00100000:0.0:1789760763.957756:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512692736 00000100:00100000:0.0:1789760763.957812:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 743455867097 time 743485489729 state 0x0 ref 1 00000100:00100000:0.0:1789760763.957829:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 743485750145 time 743485489729 state 0x1 ref 1 00000100:00100000:0.0:1789760763.957964:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 743485750145 time 743522262061 state 0x0 ref 1 00000100:00100000:0.0:1789760763.957979:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4130380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512692736:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760763.958027:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26578944, mtime: 0x6aad94fb, atime: 0x0, ctime: 0x6aad94fb, blocks: 0xd007 00010000:00010000:0.0:1789760763.958035:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f299 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.958065:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=743s v=(0 0) 00010000:00010000:0.0:1789760763.958070:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760763.958077:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f299 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26578944->26583039] (req 26578944->26583039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.958091:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f299 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26578944->26583039] (req 26578944->26583039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760763.999130:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973808 is committed 00002000:00010000:1.0:1789760763.999253:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26578944 -> 26583040 00010000:00010000:1.0:1789760763.999269:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f299 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26578944->26583039] (req 26578944->26583039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.999292:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f299 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26578944->26583039] (req 26578944->26583039) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.999301:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f299 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26578944->26583039] (req 26578944->26583039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.999308:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760763.999312:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f299 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26578944->26583039] (req 26578944->26583039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.999320:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f299 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26578944->26583039] (req 26578944->26583039) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.999342:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f299 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26578944->26583039] (req 26578944->26583039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.999350:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f299 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26578944->26583039] (req 26578944->26583039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760764.000234:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4130380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512692736:12345-192.168.203.56@tcp:4:dd.0 Request processed in 41461us (42282us total) trans 4294973808 rc 0/0 00000100:00100000:1.0:1789760764.000247:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6519 00000100:00100000:1.0:1789760764.000257:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 743485750145 time 743522262061 state 0x2 ref 0 00000100:00100000:2.0:1789760764.005098:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760764.005196:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512692864 00000100:00100000:0.0:1789760764.005206:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512692864 00000100:00100000:0.0:1789760764.005261:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 743485750145 time 743522262061 state 0x0 ref 1 00000100:00100000:0.0:1789760764.005277:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 743522522477 time 743522262061 state 0x1 ref 1 00000100:00100000:0.0:1789760764.005306:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 743522522477 time 743569606211 state 0x0 ref 1 00000100:00100000:0.0:1789760764.005320:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4133b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512692864:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760764.005368:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26583040, mtime: 0x6aad94fb, atime: 0x0, ctime: 0x6aad94fb, blocks: 0xd007 00010000:00010000:0.0:1789760764.005376:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2f2a0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.005392:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=743s v=(0 0) 00010000:00010000:0.0:1789760764.005396:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760764.005403:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2f2a0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26583040->26587135] (req 26583040->26587135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.005422:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2f2a0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26583040->26587135] (req 26583040->26587135) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760764.038938:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973809 is committed 00002000:00010000:1.0:1789760764.039045:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26583040 -> 26587136 00010000:00010000:1.0:1789760764.039062:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2f2a0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26583040->26587135] (req 26583040->26587135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.039075:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2f2a0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26583040->26587135] (req 26583040->26587135) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.039084:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2f2a0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26583040->26587135] (req 26583040->26587135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.039091:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760764.039096:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2f2a0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26583040->26587135] (req 26583040->26587135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.039111:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2f2a0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26583040->26587135] (req 26583040->26587135) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.039131:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2f2a0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26583040->26587135] (req 26583040->26587135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.039139:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2f2a0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26583040->26587135] (req 26583040->26587135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760764.039243:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4133b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512692864:12345-192.168.203.56@tcp:4:dd.0 Request processed in 33921us (34150us total) trans 4294973809 rc 0/0 00000100:00100000:1.0:1789760764.039254:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6520 00000100:00100000:1.0:1789760764.039262:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 743522522477 time 743569606211 state 0x2 ref 0 00000100:00100000:2.0:1789760764.043061:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760764.043158:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512692992 00000100:00100000:0.0:1789760764.043178:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512692992 00000100:00100000:0.0:1789760764.043232:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 743522522477 time 743569606211 state 0x0 ref 1 00000100:00100000:0.0:1789760764.043247:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 743569866627 time 743569606211 state 0x1 ref 1 00000100:00100000:0.0:1789760764.043279:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 743569866627 time 743607578754 state 0x0 ref 1 00000100:00100000:0.0:1789760764.043291:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4133800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512692992:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760764.043339:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26587136, mtime: 0x6aad94fb, atime: 0x0, ctime: 0x6aad94fb, blocks: 0xd007 00010000:00010000:0.0:1789760764.043347:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f2a7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.043363:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=743s v=(0 0) 00010000:00010000:0.0:1789760764.043367:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760764.043375:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f2a7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26587136->26591231] (req 26587136->26591231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.043388:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f2a7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26587136->26591231] (req 26587136->26591231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760764.086416:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973810 is committed 00002000:00010000:0.0:1789760764.086532:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26587136 -> 26591232 00010000:00010000:0.0:1789760764.086548:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f2a7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26587136->26591231] (req 26587136->26591231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.086561:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f2a7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26587136->26591231] (req 26587136->26591231) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.086570:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f2a7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26587136->26591231] (req 26587136->26591231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.086577:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760764.086582:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f2a7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26587136->26591231] (req 26587136->26591231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.086592:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f2a7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26587136->26591231] (req 26587136->26591231) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.086609:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f2a7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26587136->26591231] (req 26587136->26591231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.086617:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f2a7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26587136->26591231] (req 26587136->26591231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760764.086726:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4133800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512692992:12345-192.168.203.56@tcp:4:dd.0 Request processed in 43433us (43679us total) trans 4294973810 rc 0/0 00000100:00100000:0.0:1789760764.086736:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6521 00000100:00100000:0.0:1789760764.086746:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 743569866627 time 743607578754 state 0x2 ref 0 00000100:00100000:3.0:1789760764.092278:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760764.092751:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512693120 00000100:00100000:1.0:1789760764.092762:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512693120 00000100:00100000:1.0:1789760764.092814:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 743569866627 time 743607578754 state 0x0 ref 1 00000100:00100000:1.0:1789760764.092830:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 743607839170 time 743607578754 state 0x1 ref 1 00000100:00100000:1.0:1789760764.092865:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 743607839170 time 743657164877 state 0x0 ref 1 00000100:00100000:1.0:1789760764.092878:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febffb9f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512693120:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760764.092924:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26591232, mtime: 0x6aad94fb, atime: 0x0, ctime: 0x6aad94fb, blocks: 0xd007 00010000:00010000:1.0:1789760764.092932:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2f2ae lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.092948:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=743s v=(0 0) 00010000:00010000:1.0:1789760764.092952:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760764.092958:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2f2ae lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26591232->26595327] (req 26591232->26595327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.092972:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2f2ae lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26591232->26595327] (req 26591232->26595327) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760764.131270:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26591232 -> 26595328 00010000:00010000:1.0:1789760764.131290:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2f2ae lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26591232->26595327] (req 26591232->26595327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.131305:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2f2ae lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26591232->26595327] (req 26591232->26595327) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.131314:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2f2ae lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26591232->26595327] (req 26591232->26595327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.131322:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760764.131327:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2f2ae lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26591232->26595327] (req 26591232->26595327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.131335:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2f2ae lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26591232->26595327] (req 26591232->26595327) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.131354:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2f2ae lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26591232->26595327] (req 26591232->26595327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.131363:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2f2ae lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26591232->26595327] (req 26591232->26595327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760764.131440:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febffb9f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2360:x1876699512693120:12345-192.168.203.56@tcp:4:dd.0 Request processed in 38560us (39168us total) trans 4294973811 rc 0/0 00000100:00100000:1.0:1789760764.131451:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6522 00000100:00100000:1.0:1789760764.131460:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 743607839170 time 743657164877 state 0x2 ref 0 00000001:00080000:1.0:1789760764.131496:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973811 is committed 00000100:00100000:3.0:1789760764.137303:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760764.137727:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512693248 00000100:00100000:1.0:1789760764.137738:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512693248 00000100:00100000:1.0:1789760764.137792:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 743607839170 time 743657164877 state 0x0 ref 1 00000100:00100000:1.0:1789760764.137809:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 743657425293 time 743657164877 state 0x1 ref 1 00000100:00100000:1.0:1789760764.137843:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 743657425293 time 743702142593 state 0x0 ref 1 00000100:00100000:1.0:1789760764.137857:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac437f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512693248:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760764.137905:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26595328, mtime: 0x6aad94fb, atime: 0x0, ctime: 0x6aad94fb, blocks: 0xd007 00010000:00010000:1.0:1789760764.137915:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2f2b5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.137930:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=743s v=(0 0) 00010000:00010000:1.0:1789760764.137935:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760764.137941:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2f2b5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26595328->26599423] (req 26595328->26599423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.137956:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2f2b5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26595328->26599423] (req 26595328->26599423) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760764.183745:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973812 is committed 00002000:00010000:0.0:1789760764.184032:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26595328 -> 26599424 00010000:00010000:0.0:1789760764.184047:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2f2b5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26595328->26599423] (req 26595328->26599423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.184061:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2f2b5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26595328->26599423] (req 26595328->26599423) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.184070:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2f2b5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26595328->26599423] (req 26595328->26599423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.184078:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760764.184082:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2f2b5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26595328->26599423] (req 26595328->26599423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.184091:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2f2b5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26595328->26599423] (req 26595328->26599423) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.184111:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2f2b5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26595328->26599423] (req 26595328->26599423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.184119:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2f2b5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26595328->26599423] (req 26595328->26599423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760764.184220:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac437f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512693248:12345-192.168.203.56@tcp:4:dd.0 Request processed in 46361us (46921us total) trans 4294973812 rc 0/0 00000100:00100000:0.0:1789760764.184232:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6523 00000100:00100000:0.0:1789760764.184244:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 743657425293 time 743702142593 state 0x2 ref 0 00000100:00100000:2.0:1789760764.190180:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760764.190263:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512693376 00000100:00100000:0.0:1789760764.190273:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512693376 00000100:00100000:0.0:1789760764.190331:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 743657425293 time 743702142593 state 0x0 ref 1 00000100:00100000:0.0:1789760764.190346:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 743702403009 time 743702142593 state 0x1 ref 1 00000100:00100000:0.0:1789760764.190376:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 743702403009 time 743754676088 state 0x0 ref 1 00000100:00100000:0.0:1789760764.190389:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febea4b7450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512693376:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760764.190436:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26599424, mtime: 0x6aad94fb, atime: 0x0, ctime: 0x6aad94fb, blocks: 0xd007 00010000:00010000:0.0:1789760764.190445:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f2bc lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.190462:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=743s v=(0 0) 00010000:00010000:0.0:1789760764.190466:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760764.190472:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f2bc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26599424->26603519] (req 26599424->26603519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.190486:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f2bc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26599424->26603519] (req 26599424->26603519) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760764.244893:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973813 is committed 00002000:00010000:0.0:1789760764.245227:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26599424 -> 26603520 00010000:00010000:0.0:1789760764.245242:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f2bc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26599424->26603519] (req 26599424->26603519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.245260:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f2bc lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26599424->26603519] (req 26599424->26603519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.245269:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f2bc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26599424->26603519] (req 26599424->26603519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.245276:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760764.245280:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f2bc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26599424->26603519] (req 26599424->26603519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.245290:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f2bc lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26599424->26603519] (req 26599424->26603519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.245310:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f2bc lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26599424->26603519] (req 26599424->26603519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.245318:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f2bc lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26599424->26603519] (req 26599424->26603519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760764.246012:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febea4b7450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512693376:12345-192.168.203.56@tcp:4:dd.0 Request processed in 55619us (55845us total) trans 4294973813 rc 0/0 00000100:00100000:0.0:1789760764.246023:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6524 00000100:00100000:0.0:1789760764.246032:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 743702403009 time 743754676088 state 0x2 ref 0 00000100:00100000:3.0:1789760764.250186:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760764.250281:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512693504 00000100:00100000:1.0:1789760764.250290:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512693504 00000100:00100000:1.0:1789760764.250342:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 743702403009 time 743754676088 state 0x0 ref 1 00000100:00100000:1.0:1789760764.250359:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 743754936504 time 743754676088 state 0x1 ref 1 00000100:00100000:1.0:1789760764.250429:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 743754936504 time 743814728513 state 0x0 ref 1 00000100:00100000:1.0:1789760764.250441:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac437fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512693504:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760764.250491:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26603520, mtime: 0x6aad94fb, atime: 0x0, ctime: 0x6aad94fb, blocks: 0xd007 00010000:00010000:1.0:1789760764.250500:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2f2c3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.250516:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=743s v=(0 0) 00010000:00010000:1.0:1789760764.250520:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760764.250526:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2f2c3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26603520->26607615] (req 26603520->26607615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.250541:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2f2c3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26603520->26607615] (req 26603520->26607615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.275764:0:5835:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:1.0:1789760764.289756:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26603520 -> 26607616 00010000:00010000:1.0:1789760764.289777:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2f2c3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26603520->26607615] (req 26603520->26607615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.289790:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2f2c3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26603520->26607615] (req 26603520->26607615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.289798:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2f2c3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26603520->26607615] (req 26603520->26607615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.289805:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760764.289810:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2f2c3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26603520->26607615] (req 26603520->26607615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.289819:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2f2c3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26603520->26607615] (req 26603520->26607615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.289846:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2f2c3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26603520->26607615] (req 26603520->26607615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.289853:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2f2c3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26603520->26607615] (req 26603520->26607615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760764.289945:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac437fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2359:x1876699512693504:12345-192.168.203.56@tcp:4:dd.0 Request processed in 39501us (39776us total) trans 4294973814 rc 0/0 00000100:00100000:1.0:1789760764.289960:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6525 00000100:00100000:1.0:1789760764.289969:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 743754936504 time 743814728513 state 0x2 ref 0 00000001:00080000:2.0:1789760764.290455:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973814 is committed 00000100:00100000:3.0:1789760764.295646:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760764.295767:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512693632 00000100:00100000:0.0:1789760764.295777:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512693632 00000100:00100000:0.0:1789760764.295828:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 743754936504 time 743814728513 state 0x0 ref 1 00000100:00100000:0.0:1789760764.295853:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 743814988929 time 743814728513 state 0x1 ref 1 00000100:00100000:0.0:1789760764.295876:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 743814988929 time 743860176270 state 0x0 ref 1 00000100:00100000:0.0:1789760764.295888:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac437d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512693632:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760764.295934:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26607616, mtime: 0x6aad94fb, atime: 0x0, ctime: 0x6aad94fb, blocks: 0xd007 00010000:00010000:0.0:1789760764.295943:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f2ca lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.295958:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=743s v=(0 0) 00010000:00010000:0.0:1789760764.295963:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760764.295969:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f2ca lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26607616->26611711] (req 26607616->26611711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.295983:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f2ca lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26607616->26611711] (req 26607616->26611711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760764.324928:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973815 is committed 00002000:00010000:0.0:1789760764.326101:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26607616 -> 26611712 00010000:00010000:0.0:1789760764.326118:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f2ca lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26607616->26611711] (req 26607616->26611711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.326132:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f2ca lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26607616->26611711] (req 26607616->26611711) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.326147:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f2ca lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26607616->26611711] (req 26607616->26611711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.326154:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760764.326159:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f2ca lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26607616->26611711] (req 26607616->26611711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.326177:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f2ca lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26607616->26611711] (req 26607616->26611711) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.326198:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f2ca lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26607616->26611711] (req 26607616->26611711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.326206:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f2ca lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26607616->26611711] (req 26607616->26611711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760764.326422:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac437d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512693632:12345-192.168.203.56@tcp:4:dd.0 Request processed in 30529us (30783us total) trans 4294973815 rc 0/0 00000100:00100000:0.0:1789760764.326433:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6526 00000100:00100000:0.0:1789760764.326442:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 743814988929 time 743860176270 state 0x2 ref 0 00000100:00100000:3.0:1789760764.329226:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760764.329311:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512693760 00000100:00100000:0.0:1789760764.329322:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512693760 00000100:00100000:0.0:1789760764.329376:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 743814988929 time 743860176270 state 0x0 ref 1 00000100:00100000:0.0:1789760764.329390:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 743860436686 time 743860176270 state 0x1 ref 1 00000100:00100000:0.0:1789760764.329465:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 743860436686 time 743893763569 state 0x0 ref 1 00000100:00100000:0.0:1789760764.329479:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac437df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512693760:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760764.329535:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26611712, mtime: 0x6aad94fb, atime: 0x0, ctime: 0x6aad94fb, blocks: 0xd007 00010000:00010000:0.0:1789760764.329544:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2f2d1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.329561:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=743s v=(0 0) 00010000:00010000:0.0:1789760764.329566:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760764.329572:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2f2d1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26611712->26615807] (req 26611712->26615807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.329585:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2f2d1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26611712->26615807] (req 26611712->26615807) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760764.402271:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973816 is committed 00002000:00010000:3.0:1789760764.402412:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26611712 -> 26615808 00010000:00010000:3.0:1789760764.402427:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2f2d1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26611712->26615807] (req 26611712->26615807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.402442:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2f2d1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26611712->26615807] (req 26611712->26615807) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.402451:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2f2d1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26611712->26615807] (req 26611712->26615807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.402458:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760764.402464:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2f2d1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26611712->26615807] (req 26611712->26615807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.402473:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2f2d1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26611712->26615807] (req 26611712->26615807) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.402492:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2f2d1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26611712->26615807] (req 26611712->26615807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.402500:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2f2d1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26611712->26615807] (req 26611712->26615807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760764.402594:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac437df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512693760:12345-192.168.203.56@tcp:4:dd.0 Request processed in 73112us (73374us total) trans 4294973816 rc 0/0 00000100:00100000:3.0:1789760764.402607:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6527 00000100:00100000:3.0:1789760764.402616:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 743860436686 time 743893763569 state 0x2 ref 0 00000100:00100000:3.0:1789760764.411790:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760764.413250:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512693888 00000100:00100000:1.0:1789760764.413262:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512693888 00000100:00100000:1.0:1789760764.413326:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 743860436686 time 743893763569 state 0x0 ref 1 00000100:00100000:1.0:1789760764.413342:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 743894023985 time 743893763569 state 0x1 ref 1 00000100:00100000:1.0:1789760764.413380:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 743894023985 time 743977679300 state 0x0 ref 1 00000100:00100000:1.0:1789760764.413394:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac437e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512693888:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760764.413445:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26615808, mtime: 0x6aad94fb, atime: 0x0, ctime: 0x6aad94fb, blocks: 0xd007 00010000:00010000:1.0:1789760764.413454:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2f2d8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.413470:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=743s v=(0 0) 00010000:00010000:1.0:1789760764.413475:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760764.413481:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2f2d8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26615808->26619903] (req 26615808->26619903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.413495:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2f2d8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26615808->26619903] (req 26615808->26619903) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760764.455015:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973817 is committed 00002000:00010000:1.0:1789760764.455140:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26615808 -> 26619904 00010000:00010000:1.0:1789760764.455155:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2f2d8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26615808->26619903] (req 26615808->26619903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.455177:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2f2d8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26615808->26619903] (req 26615808->26619903) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.455187:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2f2d8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26615808->26619903] (req 26615808->26619903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.455195:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760764.455200:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2f2d8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26615808->26619903] (req 26615808->26619903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.455208:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2f2d8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26615808->26619903] (req 26615808->26619903) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.455229:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2f2d8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26615808->26619903] (req 26615808->26619903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.455237:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2f2d8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26615808->26619903] (req 26615808->26619903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760764.455327:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac437e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512693888:12345-192.168.203.56@tcp:4:dd.0 Request processed in 41932us (43541us total) trans 4294973817 rc 0/0 00000100:00100000:1.0:1789760764.455340:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6528 00000100:00100000:1.0:1789760764.455349:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 743894023985 time 743977679300 state 0x2 ref 0 00000100:00100000:2.0:1789760764.459186:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760764.459393:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512694016 00000100:00100000:0.0:1789760764.459404:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512694016 00000100:00100000:0.0:1789760764.459463:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 743894023985 time 743977679300 state 0x0 ref 1 00000100:00100000:0.0:1789760764.459479:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 743977939716 time 743977679300 state 0x1 ref 1 00000100:00100000:0.0:1789760764.459553:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 743977939716 time 744023851270 state 0x0 ref 1 00000100:00100000:0.0:1789760764.459577:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4130000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512694016:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760764.459637:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26619904, mtime: 0x6aad94fb, atime: 0x0, ctime: 0x6aad94fb, blocks: 0xd007 00010000:00010000:0.0:1789760764.459646:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f2df lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.459664:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=744s v=(0 0) 00010000:00010000:0.0:1789760764.459668:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760764.459675:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f2df lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26619904->26623999] (req 26619904->26623999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.459690:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f2df lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26619904->26623999] (req 26619904->26623999) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760764.492570:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973818 is committed 00002000:00010000:0.0:1789760764.492777:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26619904 -> 26624000 00010000:00010000:0.0:1789760764.492792:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f2df lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26619904->26623999] (req 26619904->26623999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.492806:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f2df lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26619904->26623999] (req 26619904->26623999) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.492827:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f2df lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26619904->26623999] (req 26619904->26623999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.492950:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760764.492955:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f2df lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26619904->26623999] (req 26619904->26623999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.492964:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f2df lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26619904->26623999] (req 26619904->26623999) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.492986:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f2df lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26619904->26623999] (req 26619904->26623999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.492994:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f2df lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26619904->26623999] (req 26619904->26623999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760764.493088:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4130000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512694016:12345-192.168.203.56@tcp:4:dd.0 Request processed in 33519us (33917us total) trans 4294973818 rc 0/0 00000100:00100000:0.0:1789760764.493097:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6529 00000100:00100000:0.0:1789760764.493108:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 743977939716 time 744023851270 state 0x2 ref 0 00000100:00100000:3.0:1789760764.495931:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760764.496295:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512694144 00000100:00100000:0.0:1789760764.496309:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512694144 00000100:00100000:0.0:1789760764.496363:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 743977939716 time 744023851270 state 0x0 ref 1 00000100:00100000:0.0:1789760764.496453:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 744024111686 time 744023851270 state 0x1 ref 1 00000100:00100000:0.0:1789760764.496489:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 744024111686 time 744060787688 state 0x0 ref 1 00000100:00100000:0.0:1789760764.496502:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac437f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512694144:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760764.496550:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26624000, mtime: 0x6aad94fb, atime: 0x0, ctime: 0x6aad94fb, blocks: 0xd007 00010000:00010000:0.0:1789760764.496559:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f2e6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.496574:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=744s v=(0 0) 00010000:00010000:0.0:1789760764.496580:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760764.496586:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f2e6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26624000->26628095] (req 26624000->26628095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.496600:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f2e6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26624000->26628095] (req 26624000->26628095) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760764.560263:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973819 is committed 00002000:00010000:1.0:1789760764.560433:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26624000 -> 26628096 00002000:00010000:1.0:1789760764.560437:0:6710:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760763 -> 1789760764 00002000:00010000:1.0:1789760764.560440:0:6710:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760763 -> 1789760764 00010000:00010000:1.0:1789760764.560453:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f2e6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26624000->26628095] (req 26624000->26628095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.560465:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f2e6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26624000->26628095] (req 26624000->26628095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.560476:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f2e6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26624000->26628095] (req 26624000->26628095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.560484:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760764.560488:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f2e6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26624000->26628095] (req 26624000->26628095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.560498:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f2e6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26624000->26628095] (req 26624000->26628095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.560517:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f2e6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26624000->26628095] (req 26624000->26628095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.560525:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f2e6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26624000->26628095] (req 26624000->26628095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760764.560620:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac437f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512694144:12345-192.168.203.56@tcp:4:dd.0 Request processed in 64116us (64693us total) trans 4294973819 rc 0/0 00000100:00100000:1.0:1789760764.560630:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6530 00000100:00100000:1.0:1789760764.560639:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 744024111686 time 744060787688 state 0x2 ref 0 00000100:00100000:3.0:1789760764.564465:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760764.564606:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512694272 00000100:00100000:1.0:1789760764.564616:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512694272 00000100:00100000:1.0:1789760764.564668:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 744024111686 time 744060787688 state 0x0 ref 1 00000100:00100000:1.0:1789760764.564684:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 744061048104 time 744060787688 state 0x1 ref 1 00000100:00100000:1.0:1789760764.564746:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 744061048104 time 744129045014 state 0x0 ref 1 00000100:00100000:1.0:1789760764.564758:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febff932a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512694272:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760764.564805:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26628096, mtime: 0x6aad94fc, atime: 0x0, ctime: 0x6aad94fc, blocks: 0xd007 00010000:00010000:1.0:1789760764.564814:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2f2ed lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.564831:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=744s v=(0 0) 00010000:00010000:1.0:1789760764.564835:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760764.564852:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2f2ed lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26628096->26632191] (req 26628096->26632191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.564867:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2f2ed lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26628096->26632191] (req 26628096->26632191) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760764.604178:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973820 is committed 00002000:00010000:1.0:1789760764.604326:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26628096 -> 26632192 00010000:00010000:1.0:1789760764.604342:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2f2ed lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26628096->26632191] (req 26628096->26632191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.604355:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2f2ed lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26628096->26632191] (req 26628096->26632191) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.604364:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2f2ed lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26628096->26632191] (req 26628096->26632191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.604371:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760764.604377:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2f2ed lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26628096->26632191] (req 26628096->26632191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.604386:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2f2ed lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26628096->26632191] (req 26628096->26632191) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.604405:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2f2ed lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26628096->26632191] (req 26628096->26632191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.604413:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2f2ed lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26628096->26632191] (req 26628096->26632191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760764.604504:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febff932a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512694272:12345-192.168.203.56@tcp:4:dd.0 Request processed in 39743us (40043us total) trans 4294973820 rc 0/0 00000100:00100000:1.0:1789760764.604517:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6531 00000100:00100000:1.0:1789760764.604526:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 744061048104 time 744129045014 state 0x2 ref 0 00000100:00100000:3.0:1789760764.606953:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760764.607789:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512694400 00000100:00100000:0.0:1789760764.607799:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512694400 00000100:00100000:0.0:1789760764.607993:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 744061048104 time 744129045014 state 0x0 ref 1 00000100:00100000:0.0:1789760764.608009:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 744129305430 time 744129045014 state 0x1 ref 1 00000100:00100000:0.0:1789760764.608112:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 744129305430 time 744172411012 state 0x0 ref 1 00000100:00100000:0.0:1789760764.608126:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febff930e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512694400:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760764.608180:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26632192, mtime: 0x6aad94fc, atime: 0x0, ctime: 0x6aad94fc, blocks: 0xd007 00010000:00010000:0.0:1789760764.608189:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f2f4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.608203:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=744s v=(0 0) 00010000:00010000:0.0:1789760764.608208:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760764.608214:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f2f4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26632192->26636287] (req 26632192->26636287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.608227:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f2f4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26632192->26636287] (req 26632192->26636287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760764.639304:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973821 is committed 00002000:00010000:1.0:1789760764.639412:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26632192 -> 26636288 00010000:00010000:1.0:1789760764.639428:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f2f4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26632192->26636287] (req 26632192->26636287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.639441:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f2f4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26632192->26636287] (req 26632192->26636287) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.639449:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f2f4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26632192->26636287] (req 26632192->26636287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.639456:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760764.639461:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f2f4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26632192->26636287] (req 26632192->26636287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.639470:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f2f4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26632192->26636287] (req 26632192->26636287) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.639487:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f2f4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26632192->26636287] (req 26632192->26636287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.639494:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f2f4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26632192->26636287] (req 26632192->26636287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760764.639784:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febff930e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512694400:12345-192.168.203.56@tcp:4:dd.0 Request processed in 31656us (32833us total) trans 4294973821 rc 0/0 00000100:00100000:1.0:1789760764.639795:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6532 00000100:00100000:1.0:1789760764.639805:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 744129305430 time 744172411012 state 0x2 ref 0 00000100:00100000:3.0:1789760764.642895:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760764.643272:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512694528 00000100:00100000:1.0:1789760764.643283:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512694528 00000100:00100000:1.0:1789760764.643335:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 744129305430 time 744172411012 state 0x0 ref 1 00000100:00100000:1.0:1789760764.643350:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 744172671428 time 744172411012 state 0x1 ref 1 00000100:00100000:1.0:1789760764.643380:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 744172671428 time 744207679896 state 0x0 ref 1 00000100:00100000:1.0:1789760764.643393:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5305f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512694528:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760764.643437:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26636288, mtime: 0x6aad94fc, atime: 0x0, ctime: 0x6aad94fc, blocks: 0xd007 00010000:00010000:1.0:1789760764.643447:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2f2fb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.643474:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=744s v=(0 0) 00010000:00010000:1.0:1789760764.643480:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760764.643487:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2f2fb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26636288->26640383] (req 26636288->26640383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.643501:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2f2fb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26636288->26640383] (req 26636288->26640383) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760764.688571:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973822 is committed 00002000:00010000:3.0:1789760764.688886:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26636288 -> 26640384 00010000:00010000:3.0:1789760764.688902:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2f2fb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26636288->26640383] (req 26636288->26640383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.688915:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2f2fb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26636288->26640383] (req 26636288->26640383) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.688925:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2f2fb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26636288->26640383] (req 26636288->26640383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.688932:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760764.688936:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2f2fb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26636288->26640383] (req 26636288->26640383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.688946:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2f2fb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26636288->26640383] (req 26636288->26640383) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.688971:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2f2fb lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26636288->26640383] (req 26636288->26640383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.688979:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2f2fb lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26636288->26640383] (req 26636288->26640383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760764.689085:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5305f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512694528:12345-192.168.203.56@tcp:4:dd.0 Request processed in 45683us (46187us total) trans 4294973822 rc 0/0 00000100:00100000:3.0:1789760764.689096:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6533 00000100:00100000:3.0:1789760764.689107:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 744172671428 time 744207679896 state 0x2 ref 0 00000100:00100000:3.0:1789760764.693863:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760764.694011:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512694656 00000100:00100000:0.0:1789760764.694022:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512694656 00000100:00100000:0.0:1789760764.694073:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 744172671428 time 744207679896 state 0x0 ref 1 00000100:00100000:0.0:1789760764.694087:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 744207940312 time 744207679896 state 0x1 ref 1 00000100:00100000:0.0:1789760764.694122:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 744207940312 time 744258421686 state 0x0 ref 1 00000100:00100000:0.0:1789760764.694135:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5307480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512694656:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760764.694196:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26640384, mtime: 0x6aad94fc, atime: 0x0, ctime: 0x6aad94fc, blocks: 0xd007 00010000:00010000:0.0:1789760764.694206:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2f302 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.694222:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=744s v=(0 0) 00010000:00010000:0.0:1789760764.694227:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760764.694233:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2f302 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26640384->26644479] (req 26640384->26644479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.694245:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2f302 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26640384->26644479] (req 26640384->26644479) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760764.732438:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26640384 -> 26644480 00010000:00010000:0.0:1789760764.732457:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2f302 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26640384->26644479] (req 26640384->26644479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.732474:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2f302 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26640384->26644479] (req 26640384->26644479) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.732483:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2f302 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26640384->26644479] (req 26640384->26644479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.732490:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760764.732495:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2f302 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26640384->26644479] (req 26640384->26644479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.732503:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2f302 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26640384->26644479] (req 26640384->26644479) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.732520:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2f302 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26640384->26644479] (req 26640384->26644479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.732528:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2f302 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26640384->26644479] (req 26640384->26644479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760764.732631:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5307480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2361:x1876699512694656:12345-192.168.203.56@tcp:4:dd.0 Request processed in 38494us (38772us total) trans 4294973823 rc 0/0 00000100:00100000:0.0:1789760764.732641:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6534 00000100:00100000:0.0:1789760764.732650:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 744207940312 time 744258421686 state 0x2 ref 0 00000001:00080000:3.0:1789760764.733316:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973823 is committed 00000100:00100000:0.0:1789760764.737127:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760764.738270:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512694784 00000100:00100000:1.0:1789760764.738283:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512694784 00000100:00100000:1.0:1789760764.738343:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 744207940312 time 744258421686 state 0x0 ref 1 00000100:00100000:1.0:1789760764.738358:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 744258682102 time 744258421686 state 0x1 ref 1 00000100:00100000:1.0:1789760764.738392:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 744258682102 time 744302691032 state 0x0 ref 1 00000100:00100000:1.0:1789760764.738406:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac287a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512694784:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760764.738465:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26644480, mtime: 0x6aad94fc, atime: 0x0, ctime: 0x6aad94fc, blocks: 0xd007 00010000:00010000:1.0:1789760764.738474:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2f309 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.738491:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=744s v=(0 0) 00010000:00010000:1.0:1789760764.738496:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760764.738502:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2f309 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26644480->26648575] (req 26644480->26648575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.738517:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2f309 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26644480->26648575] (req 26644480->26648575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760764.781981:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26644480 -> 26648576 00010000:00010000:1.0:1789760764.782004:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2f309 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26644480->26648575] (req 26644480->26648575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.782024:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2f309 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26644480->26648575] (req 26644480->26648575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.782033:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2f309 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26644480->26648575] (req 26644480->26648575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.782040:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760764.782045:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2f309 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26644480->26648575] (req 26644480->26648575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.782059:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2f309 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26644480->26648575] (req 26644480->26648575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.782081:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2f309 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26644480->26648575] (req 26644480->26648575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.782090:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2f309 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26644480->26648575] (req 26644480->26648575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760764.782208:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac287a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2360:x1876699512694784:12345-192.168.203.56@tcp:4:dd.0 Request processed in 43800us (45086us total) trans 4294973824 rc 0/0 00000100:00100000:1.0:1789760764.782219:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6535 00000100:00100000:1.0:1789760764.782227:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 744258682102 time 744302691032 state 0x2 ref 0 00000001:00080000:3.0:1789760764.782457:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973824 is committed 00000100:00100000:2.0:1789760764.790307:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760764.790491:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512694912 00000100:00100000:3.0:1789760764.790504:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512694912 00000100:00100000:3.0:1789760764.790590:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 744258682102 time 744302691032 state 0x0 ref 1 00000100:00100000:3.0:1789760764.790612:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 744302951448 time 744302691032 state 0x1 ref 1 00000100:00100000:3.0:1789760764.790651:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 744302951448 time 744354950666 state 0x0 ref 1 00000100:00100000:3.0:1789760764.790667:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4133480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512694912:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760764.790781:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26648576, mtime: 0x6aad94fc, atime: 0x0, ctime: 0x6aad94fc, blocks: 0xd007 00010000:00010000:3.0:1789760764.790792:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2f310 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.790815:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=744s v=(0 0) 00010000:00010000:3.0:1789760764.790820:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760764.790828:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2f310 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26648576->26652671] (req 26648576->26652671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.790966:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2f310 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26648576->26652671] (req 26648576->26652671) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760764.829043:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26648576 -> 26652672 00010000:00010000:0.0:1789760764.829064:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2f310 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26648576->26652671] (req 26648576->26652671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.829077:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2f310 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26648576->26652671] (req 26648576->26652671) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.829086:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2f310 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26648576->26652671] (req 26648576->26652671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.829093:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760764.829098:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2f310 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26648576->26652671] (req 26648576->26652671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.829108:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2f310 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26648576->26652671] (req 26648576->26652671) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.829127:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2f310 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26648576->26652671] (req 26648576->26652671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.829135:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2f310 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26648576->26652671] (req 26648576->26652671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760764.829237:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4133480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2361:x1876699512694912:12345-192.168.203.56@tcp:4:dd.0 Request processed in 38569us (38941us total) trans 4294973825 rc 0/0 00000100:00100000:0.0:1789760764.829248:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6536 00000100:00100000:0.0:1789760764.829258:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 744302951448 time 744354950666 state 0x2 ref 0 00000001:00080000:3.0:1789760764.830010:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973825 is committed 00000100:00100000:3.0:1789760764.835404:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760764.835553:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512695040 00000100:00100000:0.0:1789760764.835564:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512695040 00000100:00100000:0.0:1789760764.835682:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 744302951448 time 744354950666 state 0x0 ref 1 00000100:00100000:0.0:1789760764.835782:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 744355211082 time 744354950666 state 0x1 ref 1 00000100:00100000:0.0:1789760764.835857:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 744355211082 time 744400155617 state 0x0 ref 1 00000100:00100000:0.0:1789760764.835871:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5304700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512695040:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760764.835921:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26652672, mtime: 0x6aad94fc, atime: 0x0, ctime: 0x6aad94fc, blocks: 0xd007 00010000:00010000:0.0:1789760764.835929:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2f317 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.835946:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=744s v=(0 0) 00010000:00010000:0.0:1789760764.835951:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760764.835957:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2f317 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26652672->26656767] (req 26652672->26656767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.835970:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2f317 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26652672->26656767] (req 26652672->26656767) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760764.887196:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973826 is committed 00002000:00010000:0.0:1789760764.887287:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26652672 -> 26656768 00010000:00010000:0.0:1789760764.887301:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2f317 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26652672->26656767] (req 26652672->26656767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.887313:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2f317 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26652672->26656767] (req 26652672->26656767) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.887323:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2f317 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26652672->26656767] (req 26652672->26656767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.887330:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760764.887335:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2f317 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26652672->26656767] (req 26652672->26656767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.887344:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2f317 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26652672->26656767] (req 26652672->26656767) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.887361:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2f317 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26652672->26656767] (req 26652672->26656767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.887368:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2f317 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26652672->26656767] (req 26652672->26656767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760764.887481:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5304700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512695040:12345-192.168.203.56@tcp:4:dd.0 Request processed in 51608us (52082us total) trans 4294973826 rc 0/0 00000100:00100000:0.0:1789760764.887490:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6537 00000100:00100000:0.0:1789760764.887499:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 744355211082 time 744400155617 state 0x2 ref 0 00000100:00100000:1.0:1789760764.890241:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760764.890347:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512695168 00000100:00100000:2.0:1789760764.890357:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512695168 00000100:00100000:2.0:1789760764.890409:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 744355211082 time 744400155617 state 0x0 ref 1 00000100:00100000:2.0:1789760764.890427:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 744400416033 time 744400155617 state 0x1 ref 1 00000100:00100000:2.0:1789760764.890491:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 744400416033 time 744454791317 state 0x0 ref 1 00000100:00100000:2.0:1789760764.890504:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac403a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512695168:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760764.890614:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26656768, mtime: 0x6aad94fc, atime: 0x0, ctime: 0x6aad94fc, blocks: 0xd007 00010000:00010000:2.0:1789760764.890637:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2f31e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.890652:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=744s v=(0 0) 00010000:00010000:2.0:1789760764.890657:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760764.890663:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2f31e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26656768->26660863] (req 26656768->26660863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.890677:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2f31e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26656768->26660863] (req 26656768->26660863) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760764.920329:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973827 is committed 00002000:00010000:2.0:1789760764.920425:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26656768 -> 26660864 00010000:00010000:2.0:1789760764.920441:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2f31e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26656768->26660863] (req 26656768->26660863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.920454:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2f31e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26656768->26660863] (req 26656768->26660863) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.920464:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2f31e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26656768->26660863] (req 26656768->26660863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.920477:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760764.920482:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2f31e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26656768->26660863] (req 26656768->26660863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.920491:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2f31e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26656768->26660863] (req 26656768->26660863) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.920509:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2f31e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26656768->26660863] (req 26656768->26660863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.920517:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2f31e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26656768->26660863] (req 26656768->26660863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760764.920610:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac403a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512695168:12345-192.168.203.56@tcp:4:dd.0 Request processed in 30103us (30372us total) trans 4294973827 rc 0/0 00000100:00100000:2.0:1789760764.920620:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6538 00000100:00100000:2.0:1789760764.920629:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 744400416033 time 744454791317 state 0x2 ref 0 00000100:00100000:0.0:1789760764.923960:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760764.924145:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512695296 00000100:00100000:3.0:1789760764.924155:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512695296 00000100:00100000:0.0:1789760764.924282:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 744400416033 time 744454791317 state 0x0 ref 1 00000100:00100000:0.0:1789760764.924302:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 744455051733 time 744454791317 state 0x1 ref 1 00000100:00100000:0.0:1789760764.924366:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 744455051733 time 744488665369 state 0x0 ref 1 00000100:00100000:0.0:1789760764.924380:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2879c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512695296:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760764.924440:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26660864, mtime: 0x6aad94fc, atime: 0x0, ctime: 0x6aad94fc, blocks: 0xd007 00010000:00010000:0.0:1789760764.924449:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2f325 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.924466:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=744s v=(0 0) 00010000:00010000:0.0:1789760764.924470:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760764.924477:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2f325 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26660864->26664959] (req 26660864->26664959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.924490:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2f325 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26660864->26664959] (req 26660864->26664959) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760764.958318:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26660864 -> 26664960 00010000:00010000:0.0:1789760764.958337:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2f325 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26660864->26664959] (req 26660864->26664959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.958351:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2f325 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26660864->26664959] (req 26660864->26664959) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.958361:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2f325 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26660864->26664959] (req 26660864->26664959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.958375:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760764.958380:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2f325 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26660864->26664959] (req 26660864->26664959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.958389:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2f325 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26660864->26664959] (req 26660864->26664959) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.958409:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2f325 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26660864->26664959] (req 26660864->26664959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.958416:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2f325 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26660864->26664959] (req 26660864->26664959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760764.958512:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2879c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2361:x1876699512695296:12345-192.168.203.56@tcp:4:dd.0 Request processed in 34131us (34558us total) trans 4294973828 rc 0/0 00000100:00100000:0.0:1789760764.958524:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6539 00000100:00100000:0.0:1789760764.958534:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 744455051733 time 744488665369 state 0x2 ref 0 00000001:00080000:3.0:1789760764.960682:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973828 is committed 00000100:00100000:0.0:1789760764.961590:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760764.961673:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512695424 00000100:00100000:1.0:1789760764.961682:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512695424 00000100:00100000:1.0:1789760764.961762:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 744455051733 time 744488665369 state 0x0 ref 1 00000100:00100000:1.0:1789760764.961776:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 744488925785 time 744488665369 state 0x1 ref 1 00000100:00100000:1.0:1789760764.961804:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 744488925785 time 744526103887 state 0x0 ref 1 00000100:00100000:1.0:1789760764.961816:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2878e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512695424:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760764.961875:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26664960, mtime: 0x6aad94fc, atime: 0x0, ctime: 0x6aad94fc, blocks: 0xd007 00010000:00010000:1.0:1789760764.961884:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2f32c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.961900:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=744s v=(0 0) 00010000:00010000:1.0:1789760764.961905:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760764.961910:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2f32c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26664960->26669055] (req 26664960->26669055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.961924:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2f32c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26664960->26669055] (req 26664960->26669055) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760764.989269:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973829 is committed 00002000:00010000:3.0:1789760764.989386:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26664960 -> 26669056 00010000:00010000:3.0:1789760764.989402:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2f32c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26664960->26669055] (req 26664960->26669055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.989416:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2f32c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26664960->26669055] (req 26664960->26669055) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.989425:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2f32c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26664960->26669055] (req 26664960->26669055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.989437:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760764.989443:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2f32c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26664960->26669055] (req 26664960->26669055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.989458:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2f32c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26664960->26669055] (req 26664960->26669055) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.989476:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2f32c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26664960->26669055] (req 26664960->26669055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.989484:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2f32c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26664960->26669055] (req 26664960->26669055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760764.989636:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2878e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512695424:12345-192.168.203.56@tcp:4:dd.0 Request processed in 27818us (28047us total) trans 4294973829 rc 0/0 00000100:00100000:3.0:1789760764.989647:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6540 00000100:00100000:3.0:1789760764.989656:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 744488925785 time 744526103887 state 0x2 ref 0 00000100:00100000:0.0:1789760764.994571:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760764.994654:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512695552 00000100:00100000:2.0:1789760764.994665:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512695552 00000100:00100000:2.0:1789760764.994742:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 744488925785 time 744526103887 state 0x0 ref 1 00000100:00100000:2.0:1789760764.994758:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 744526364303 time 744526103887 state 0x1 ref 1 00000100:00100000:2.0:1789760764.994793:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 744526364303 time 744559091913 state 0x0 ref 1 00000100:00100000:2.0:1789760764.994810:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac287b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512695552:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760764.994856:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26669056, mtime: 0x6aad94fc, atime: 0x0, ctime: 0x6aad94fc, blocks: 0xd007 00010000:00010000:2.0:1789760764.994865:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2f333 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.994886:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=744s v=(0 0) 00010000:00010000:2.0:1789760764.994891:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760764.994898:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2f333 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26669056->26673151] (req 26669056->26673151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.994913:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2f333 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26669056->26673151] (req 26669056->26673151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760765.048182:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973830 is committed 00002000:00010000:1.0:1789760765.048319:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26669056 -> 26673152 00010000:00010000:1.0:1789760765.048333:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2f333 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26669056->26673151] (req 26669056->26673151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.048346:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2f333 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26669056->26673151] (req 26669056->26673151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.048360:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2f333 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26669056->26673151] (req 26669056->26673151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.048368:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760765.048372:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2f333 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26669056->26673151] (req 26669056->26673151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.048382:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2f333 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26669056->26673151] (req 26669056->26673151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.048402:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2f333 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26669056->26673151] (req 26669056->26673151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.048409:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2f333 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26669056->26673151] (req 26669056->26673151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760765.048497:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac287b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512695552:12345-192.168.203.56@tcp:4:dd.0 Request processed in 53685us (53929us total) trans 4294973830 rc 0/0 00000100:00100000:1.0:1789760765.048506:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6541 00000100:00100000:1.0:1789760765.048515:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 744526364303 time 744559091913 state 0x2 ref 0 00000100:00100000:0.0:1789760765.051556:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760765.051809:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512695680 00000100:00100000:2.0:1789760765.051819:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512695680 00000100:00100000:2.0:1789760765.051963:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 744526364303 time 744559091913 state 0x0 ref 1 00000100:00100000:2.0:1789760765.051978:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 744559352329 time 744559091913 state 0x1 ref 1 00000100:00100000:2.0:1789760765.052031:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 744559352329 time 744616330920 state 0x0 ref 1 00000100:00100000:2.0:1789760765.052043:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2878000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512695680:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760765.052084:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26673152, mtime: 0x6aad94fc, atime: 0x0, ctime: 0x6aad94fc, blocks: 0xd007 00010000:00010000:2.0:1789760765.052092:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2f33a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.052108:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=744s v=(0 0) 00010000:00010000:2.0:1789760765.052113:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760765.052119:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2f33a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26673152->26677247] (req 26673152->26677247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.052131:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2f33a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26673152->26677247] (req 26673152->26677247) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:1.0:1789760765.052506:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:2.0:1789760765.094921:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973831 is committed 00002000:00010000:1.0:1789760765.095109:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26673152 -> 26677248 00010000:00010000:1.0:1789760765.095132:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2f33a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26673152->26677247] (req 26673152->26677247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.095153:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2f33a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26673152->26677247] (req 26673152->26677247) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.095170:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2f33a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26673152->26677247] (req 26673152->26677247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.095178:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760765.095182:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2f33a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26673152->26677247] (req 26673152->26677247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.095191:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2f33a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26673152->26677247] (req 26673152->26677247) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.095208:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2f33a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26673152->26677247] (req 26673152->26677247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.095216:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2f33a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26673152->26677247] (req 26673152->26677247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760765.095299:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2878000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512695680:12345-192.168.203.56@tcp:4:dd.0 Request processed in 43253us (43746us total) trans 4294973831 rc 0/0 00000100:00100000:1.0:1789760765.095309:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6542 00000100:00100000:1.0:1789760765.095318:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 744559352329 time 744616330920 state 0x2 ref 0 00000100:00100000:2.0:1789760765.102046:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760765.102114:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512695808 00000100:00100000:3.0:1789760765.102123:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512695808 00000100:00100000:3.0:1789760765.102186:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 744559352329 time 744616330920 state 0x0 ref 1 00000100:00100000:3.0:1789760765.102221:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 744616591336 time 744616330920 state 0x1 ref 1 00000100:00100000:3.0:1789760765.102258:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 744616591336 time 744666557200 state 0x0 ref 1 00000100:00100000:3.0:1789760765.102270:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4130700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512695808:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760765.102317:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26677248, mtime: 0x6aad94fc, atime: 0x0, ctime: 0x6aad94fc, blocks: 0xd007 00010000:00010000:3.0:1789760765.102327:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2f341 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.102342:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=744s v=(0 0) 00010000:00010000:3.0:1789760765.102347:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760765.102353:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2f341 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26677248->26681343] (req 26677248->26681343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.102513:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2f341 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26677248->26681343] (req 26677248->26681343) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760765.140051:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973832 is committed 00002000:00010000:1.0:1789760765.141909:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26677248 -> 26681344 00010000:00010000:1.0:1789760765.141928:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2f341 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26677248->26681343] (req 26677248->26681343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.141942:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2f341 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26677248->26681343] (req 26677248->26681343) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.141949:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2f341 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26677248->26681343] (req 26677248->26681343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.141956:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760765.141961:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2f341 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26677248->26681343] (req 26677248->26681343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.141969:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2f341 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26677248->26681343] (req 26677248->26681343) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.141987:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2f341 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26677248->26681343] (req 26677248->26681343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.141994:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2f341 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26677248->26681343] (req 26677248->26681343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760765.142091:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4130700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512695808:12345-192.168.203.56@tcp:4:dd.0 Request processed in 39819us (40050us total) trans 4294973832 rc 0/0 00000100:00100000:1.0:1789760765.142101:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6543 00000100:00100000:1.0:1789760765.142110:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 744616591336 time 744666557200 state 0x2 ref 0 00000100:00100000:2.0:1789760765.149259:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760765.149355:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512695936 00000100:00100000:1.0:1789760765.149367:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512695936 00000100:00100000:1.0:1789760765.149428:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 744616591336 time 744666557200 state 0x0 ref 1 00000100:00100000:1.0:1789760765.149444:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 744666817616 time 744666557200 state 0x1 ref 1 00000100:00100000:1.0:1789760765.149475:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 744666817616 time 744713774659 state 0x0 ref 1 00000100:00100000:1.0:1789760765.149488:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4131180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512695936:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760765.149536:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26681344, mtime: 0x6aad94fc, atime: 0x0, ctime: 0x6aad94fc, blocks: 0xd007 00010000:00010000:1.0:1789760765.149545:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2f348 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.149561:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=744s v=(0 0) 00010000:00010000:1.0:1789760765.149566:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760765.149573:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2f348 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26681344->26685439] (req 26681344->26685439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.149586:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2f348 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26681344->26685439] (req 26681344->26685439) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760765.186256:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26681344 -> 26685440 00010000:00010000:1.0:1789760765.186276:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2f348 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26681344->26685439] (req 26681344->26685439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.186289:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2f348 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26681344->26685439] (req 26681344->26685439) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.186298:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2f348 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26681344->26685439] (req 26681344->26685439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.186305:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760765.186309:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2f348 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26681344->26685439] (req 26681344->26685439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.186318:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2f348 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26681344->26685439] (req 26681344->26685439) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.186335:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2f348 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26681344->26685439] (req 26681344->26685439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.186343:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2f348 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26681344->26685439] (req 26681344->26685439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760765.186430:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4131180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2360:x1876699512695936:12345-192.168.203.56@tcp:4:dd.0 Request processed in 36940us (37178us total) trans 4294973833 rc 0/0 00000100:00100000:1.0:1789760765.186440:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6544 00000100:00100000:1.0:1789760765.186449:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 744666817616 time 744713774659 state 0x2 ref 0 00000001:00080000:3.0:1789760765.187009:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973833 is committed 00000100:00100000:2.0:1789760765.188279:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760765.188324:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760765.188396:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760765.188470:0:8229:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512696192 00000100:00100000:1.0:1789760765.188480:0:8229:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512696192 00000100:00100000:1.0:1789760765.188538:0:8229:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512696320 00000100:00100000:1.0:1789760765.188542:0:8229:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512696320 00000100:00100000:1.0:1789760765.188568:0:8229:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 494 00000100:00100000:1.0:1789760765.188577:0:8229:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4130a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+8:2359:x1876699512696192:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:1.0:1789760765.188635:0:8229:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4130a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+8:2359:x1876699512696192:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 59us (362us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760765.188641:0:8229:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 494 00000100:00100000:1.0:1789760765.188656:0:8229:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 495 00000100:00100000:1.0:1789760765.188661:0:8229:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4132d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:d2c40150-e97b-40da-952b-cb9f18bf7e96+5:2358:x1876699512696320:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:1.0:1789760765.188692:0:8229:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4132d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:d2c40150-e97b-40da-952b-cb9f18bf7e96+5:2358:x1876699512696320:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 30us (296us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760765.188720:0:8229:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 495 00000100:00080000:2.0:1789760765.188862:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789760765.190095:0:5842:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512696064 00000100:00100000:3.0:1789760765.190104:0:5842:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512696064 00000100:00100000:2.0:1789760765.190270:0:5842:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 406 00000100:00100000:2.0:1789760765.190279:0:5842:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febe00c6450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:e6cf7267-9b19-462a-a263-14115df15dab+13:2358:x1876699512696064:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:2.0:1789760765.190341:0:5842:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febe00c6450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:e6cf7267-9b19-462a-a263-14115df15dab+13:2358:x1876699512696064:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 62us (2016us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760765.190347:0:5842:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 406 00000100:00100000:0.0:1789760765.191020:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760765.191102:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512696448 00000100:00100000:1.0:1789760765.191107:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512696448 00000100:00100000:1.0:1789760765.191148:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 744666817616 time 744713774659 state 0x0 ref 1 00000100:00100000:1.0:1789760765.191172:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 744714035075 time 744713774659 state 0x1 ref 1 00000100:00100000:1.0:1789760765.191233:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 744714035075 time 744755531997 state 0x0 ref 1 00000100:00100000:1.0:1789760765.191244:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac287b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512696448:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760765.191284:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26685440, mtime: 0x6aad94fc, atime: 0x0, ctime: 0x6aad94fc, blocks: 0xd007 00010000:00010000:1.0:1789760765.191292:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2f34f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.191306:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=744s v=(0 0) 00010000:00010000:1.0:1789760765.191310:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760765.191316:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2f34f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26685440->26689535] (req 26685440->26689535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.191330:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2f34f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26685440->26689535] (req 26685440->26689535) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760765.223173:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973834 is committed 00002000:00010000:1.0:1789760765.223339:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26685440 -> 26689536 00010000:00010000:1.0:1789760765.223354:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2f34f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26685440->26689535] (req 26685440->26689535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.223367:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2f34f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26685440->26689535] (req 26685440->26689535) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.223377:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2f34f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26685440->26689535] (req 26685440->26689535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.223385:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760765.223390:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2f34f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26685440->26689535] (req 26685440->26689535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.223399:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2f34f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26685440->26689535] (req 26685440->26689535) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.223419:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2f34f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26685440->26689535] (req 26685440->26689535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.223427:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2f34f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26685440->26689535] (req 26685440->26689535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760765.223546:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac287b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512696448:12345-192.168.203.56@tcp:4:dd.0 Request processed in 32299us (32524us total) trans 4294973834 rc 0/0 00000100:00100000:1.0:1789760765.223556:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6545 00000100:00100000:1.0:1789760765.223566:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 744714035075 time 744755531997 state 0x2 ref 0 00000100:00100000:3.0:1789760765.236093:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760765.236307:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512696576 00000100:00100000:1.0:1789760765.236316:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512696576 00000100:00100000:1.0:1789760765.236365:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 744714035075 time 744755531997 state 0x0 ref 1 00000100:00100000:1.0:1789760765.236381:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 744755792413 time 744755531997 state 0x1 ref 1 00000100:00100000:1.0:1789760765.236408:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 744755792413 time 744800708526 state 0x0 ref 1 00000100:00100000:1.0:1789760765.236421:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5304380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512696576:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760765.236470:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26689536, mtime: 0x6aad94fc, atime: 0x0, ctime: 0x6aad94fc, blocks: 0xd007 00010000:00010000:1.0:1789760765.236479:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2f356 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.236495:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=744s v=(0 0) 00010000:00010000:1.0:1789760765.236499:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760765.236506:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2f356 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26689536->26693631] (req 26689536->26693631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.236519:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2f356 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26689536->26693631] (req 26689536->26693631) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.298935:0:7647:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:3.0:1789760765.317010:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973835 is committed 00002000:00010000:0.0:1789760765.317040:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26689536 -> 26693632 00010000:00010000:0.0:1789760765.317055:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2f356 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26689536->26693631] (req 26689536->26693631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.317067:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2f356 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26689536->26693631] (req 26689536->26693631) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.317075:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2f356 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26689536->26693631] (req 26689536->26693631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.317081:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760765.317085:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2f356 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26689536->26693631] (req 26689536->26693631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.317093:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2f356 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26689536->26693631] (req 26689536->26693631) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.317113:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2f356 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26689536->26693631] (req 26689536->26693631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.317121:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2f356 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26689536->26693631] (req 26689536->26693631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760765.317918:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5304380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512696576:12345-192.168.203.56@tcp:4:dd.0 Request processed in 81237us (81573us total) trans 4294973835 rc 0/0 00000100:00100000:0.0:1789760765.317935:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6546 00000100:00100000:0.0:1789760765.317946:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 744755792413 time 744800708526 state 0x2 ref 0 00000100:00100000:2.0:1789760765.325995:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760765.326097:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512696704 00000100:00100000:1.0:1789760765.326106:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512696704 00000100:00100000:1.0:1789760765.326168:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 744755792413 time 744800708526 state 0x0 ref 1 00000100:00100000:1.0:1789760765.326184:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 744800968942 time 744800708526 state 0x1 ref 1 00000100:00100000:1.0:1789760765.326211:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 744800968942 time 744890511096 state 0x0 ref 1 00000100:00100000:1.0:1789760765.326223:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4132300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512696704:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760765.326282:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26693632, mtime: 0x6aad94fc, atime: 0x0, ctime: 0x6aad94fc, blocks: 0xd007 00010000:00010000:1.0:1789760765.326291:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2f35d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.326307:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=744s v=(0 0) 00010000:00010000:1.0:1789760765.326312:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760765.326319:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2f35d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26693632->26697727] (req 26693632->26697727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.326332:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2f35d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26693632->26697727] (req 26693632->26697727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760765.392426:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973836 is committed 00002000:00010000:1.0:1789760765.392530:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26693632 -> 26697728 00010000:00010000:1.0:1789760765.392545:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2f35d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26693632->26697727] (req 26693632->26697727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.392574:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2f35d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26693632->26697727] (req 26693632->26697727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.392583:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2f35d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26693632->26697727] (req 26693632->26697727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.392591:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760765.392595:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2f35d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26693632->26697727] (req 26693632->26697727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.392604:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2f35d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26693632->26697727] (req 26693632->26697727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.392632:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2f35d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26693632->26697727] (req 26693632->26697727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.392641:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2f35d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26693632->26697727] (req 26693632->26697727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760765.392765:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4132300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512696704:12345-192.168.203.56@tcp:4:dd.0 Request processed in 66538us (66775us total) trans 4294973836 rc 0/0 00000100:00100000:1.0:1789760765.392775:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6547 00000100:00100000:1.0:1789760765.392784:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 744800968942 time 744890511096 state 0x2 ref 0 00000100:00100000:2.0:1789760765.399674:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760765.399806:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512696832 00000100:00100000:1.0:1789760765.399817:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512696832 00000100:00100000:1.0:1789760765.399873:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 744800968942 time 744890511096 state 0x0 ref 1 00000100:00100000:1.0:1789760765.399888:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 744890771512 time 744890511096 state 0x1 ref 1 00000100:00100000:1.0:1789760765.399919:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 744890771512 time 744964219262 state 0x0 ref 1 00000100:00100000:1.0:1789760765.399932:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4131f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512696832:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760765.399981:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26697728, mtime: 0x6aad94fc, atime: 0x0, ctime: 0x6aad94fc, blocks: 0xd007 00010000:00010000:1.0:1789760765.399990:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2f364 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.400005:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=744s v=(0 0) 00010000:00010000:1.0:1789760765.400010:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760765.400017:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2f364 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26697728->26701823] (req 26697728->26701823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.400031:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2f364 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26697728->26701823] (req 26697728->26701823) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760765.472905:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973837 is committed 00002000:00010000:3.0:1789760765.473367:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26697728 -> 26701824 00010000:00010000:3.0:1789760765.473392:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2f364 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26697728->26701823] (req 26697728->26701823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.473405:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2f364 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26697728->26701823] (req 26697728->26701823) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.473414:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2f364 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26697728->26701823] (req 26697728->26701823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.473422:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760765.473427:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2f364 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26697728->26701823] (req 26697728->26701823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.473437:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2f364 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26697728->26701823] (req 26697728->26701823) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.473507:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2f364 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26697728->26701823] (req 26697728->26701823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.473515:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2f364 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26697728->26701823] (req 26697728->26701823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760765.473603:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4131f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512696832:12345-192.168.203.56@tcp:4:dd.0 Request processed in 73669us (73933us total) trans 4294973837 rc 0/0 00000100:00100000:3.0:1789760765.473614:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6548 00000100:00100000:3.0:1789760765.473623:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 744890771512 time 744964219262 state 0x2 ref 0 00000100:00100000:2.0:1789760765.477633:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760765.477743:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512696960 00000100:00100000:1.0:1789760765.477753:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512696960 00000100:00100000:1.0:1789760765.477806:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 744890771512 time 744964219262 state 0x0 ref 1 00000100:00100000:1.0:1789760765.477823:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 744964479678 time 744964219262 state 0x1 ref 1 00000100:00100000:1.0:1789760765.477855:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 744964479678 time 745042155336 state 0x0 ref 1 00000100:00100000:1.0:1789760765.477871:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4130380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512696960:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760765.477923:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26701824, mtime: 0x6aad94fc, atime: 0x0, ctime: 0x6aad94fc, blocks: 0xd007 00010000:00010000:1.0:1789760765.477932:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2f36b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.477948:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=745s v=(0 0) 00010000:00010000:1.0:1789760765.477953:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760765.477960:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2f36b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26701824->26705919] (req 26701824->26705919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.477973:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2f36b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26701824->26705919] (req 26701824->26705919) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760765.522570:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973838 is committed 00002000:00010000:2.0:1789760765.522756:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26701824 -> 26705920 00010000:00010000:2.0:1789760765.522776:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2f36b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26701824->26705919] (req 26701824->26705919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.522790:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2f36b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26701824->26705919] (req 26701824->26705919) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.522798:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2f36b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26701824->26705919] (req 26701824->26705919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.522805:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760765.522810:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2f36b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26701824->26705919] (req 26701824->26705919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.522819:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2f36b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26701824->26705919] (req 26701824->26705919) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.522839:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2f36b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26701824->26705919] (req 26701824->26705919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.522847:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2f36b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26701824->26705919] (req 26701824->26705919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760765.522948:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4130380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512696960:12345-192.168.203.56@tcp:4:dd.0 Request processed in 45073us (45320us total) trans 4294973838 rc 0/0 00000100:00100000:2.0:1789760765.522962:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6549 00000100:00100000:2.0:1789760765.522971:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 744964479678 time 745042155336 state 0x2 ref 0 00000100:00100000:2.0:1789760765.531149:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760765.531754:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512697088 00000100:00100000:1.0:1789760765.531764:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512697088 00000100:00100000:1.0:1789760765.531821:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 744964479678 time 745042155336 state 0x0 ref 1 00000100:00100000:1.0:1789760765.531836:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 745042415752 time 745042155336 state 0x1 ref 1 00000100:00100000:1.0:1789760765.531863:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 745042415752 time 745096163014 state 0x0 ref 1 00000100:00100000:1.0:1789760765.531877:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4133b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512697088:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760765.531924:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26705920, mtime: 0x6aad94fc, atime: 0x0, ctime: 0x6aad94fc, blocks: 0xd007 00010000:00010000:1.0:1789760765.531933:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2f372 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.531955:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=745s v=(0 0) 00010000:00010000:1.0:1789760765.531959:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760765.531965:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2f372 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26705920->26710015] (req 26705920->26710015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.531978:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2f372 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26705920->26710015] (req 26705920->26710015) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760765.574487:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973839 is committed 00002000:00010000:1.0:1789760765.574593:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26705920 -> 26710016 00002000:00010000:1.0:1789760765.574599:0:6708:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760764 -> 1789760765 00002000:00010000:1.0:1789760765.574601:0:6708:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760764 -> 1789760765 00010000:00010000:1.0:1789760765.574616:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2f372 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26705920->26710015] (req 26705920->26710015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.574630:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2f372 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26705920->26710015] (req 26705920->26710015) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.574638:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2f372 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26705920->26710015] (req 26705920->26710015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.574644:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760765.574650:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2f372 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26705920->26710015] (req 26705920->26710015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.574659:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2f372 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26705920->26710015] (req 26705920->26710015) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.574677:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2f372 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26705920->26710015] (req 26705920->26710015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.574767:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2f372 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26705920->26710015] (req 26705920->26710015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760765.574960:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4133b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512697088:12345-192.168.203.56@tcp:4:dd.0 Request processed in 43081us (43817us total) trans 4294973839 rc 0/0 00000100:00100000:1.0:1789760765.574971:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6550 00000100:00100000:1.0:1789760765.574980:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 745042415752 time 745096163014 state 0x2 ref 0 00000100:00100000:0.0:1789760765.578393:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760765.578473:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512697216 00000100:00100000:3.0:1789760765.578483:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512697216 00000100:00100000:3.0:1789760765.578533:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 745042415752 time 745096163014 state 0x0 ref 1 00000100:00100000:3.0:1789760765.578549:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 745096423430 time 745096163014 state 0x1 ref 1 00000100:00100000:3.0:1789760765.578579:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 745096423430 time 745142878938 state 0x0 ref 1 00000100:00100000:3.0:1789760765.578592:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2878700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512697216:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760765.578642:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26710016, mtime: 0x6aad94fd, atime: 0x0, ctime: 0x6aad94fd, blocks: 0xd007 00010000:00010000:3.0:1789760765.578651:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2f379 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.578667:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=745s v=(0 0) 00010000:00010000:3.0:1789760765.578671:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760765.578677:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2f379 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26710016->26714111] (req 26710016->26714111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.578820:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2f379 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26710016->26714111] (req 26710016->26714111) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760765.625639:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26710016 -> 26714112 00010000:00010000:3.0:1789760765.625660:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2f379 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26710016->26714111] (req 26710016->26714111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.625675:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2f379 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26710016->26714111] (req 26710016->26714111) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.625684:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2f379 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26710016->26714111] (req 26710016->26714111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.625691:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760765.625766:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2f379 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26710016->26714111] (req 26710016->26714111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.625777:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2f379 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26710016->26714111] (req 26710016->26714111) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.625800:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2f379 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26710016->26714111] (req 26710016->26714111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.625809:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2f379 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26710016->26714111] (req 26710016->26714111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760765.626022:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2878700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2361:x1876699512697216:12345-192.168.203.56@tcp:4:dd.0 Request processed in 47426us (47632us total) trans 4294973840 rc 0/0 00000100:00100000:3.0:1789760765.626043:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6551 00000100:00100000:3.0:1789760765.626053:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 745096423430 time 745142878938 state 0x2 ref 0 00000001:00080000:1.0:1789760765.626202:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973840 is committed 00000100:00100000:2.0:1789760765.638152:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760765.638354:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512697344 00000100:00100000:1.0:1789760765.638367:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512697344 00000100:00100000:1.0:1789760765.638557:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 745096423430 time 745142878938 state 0x0 ref 1 00000100:00100000:1.0:1789760765.638575:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 745143139354 time 745142878938 state 0x1 ref 1 00000100:00100000:1.0:1789760765.638608:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 745143139354 time 745202907771 state 0x0 ref 1 00000100:00100000:1.0:1789760765.638622:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4133800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512697344:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760765.638672:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26714112, mtime: 0x6aad94fd, atime: 0x0, ctime: 0x6aad94fd, blocks: 0xd007 00010000:00010000:1.0:1789760765.638680:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2f380 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.638724:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=745s v=(0 0) 00010000:00010000:1.0:1789760765.638729:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760765.638736:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2f380 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26714112->26718207] (req 26714112->26718207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.638750:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2f380 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26714112->26718207] (req 26714112->26718207) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760765.682574:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973841 is committed 00002000:00010000:0.0:1789760765.682737:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26714112 -> 26718208 00010000:00010000:0.0:1789760765.682752:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2f380 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26714112->26718207] (req 26714112->26718207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.682765:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2f380 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26714112->26718207] (req 26714112->26718207) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.682773:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2f380 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26714112->26718207] (req 26714112->26718207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.682779:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760765.682783:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2f380 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26714112->26718207] (req 26714112->26718207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.682791:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2f380 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26714112->26718207] (req 26714112->26718207) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.682808:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2f380 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26714112->26718207] (req 26714112->26718207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.682816:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2f380 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26714112->26718207] (req 26714112->26718207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760765.682977:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4133800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512697344:12345-192.168.203.56@tcp:4:dd.0 Request processed in 44352us (44830us total) trans 4294973841 rc 0/0 00000100:00100000:0.0:1789760765.682988:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6552 00000100:00100000:0.0:1789760765.682999:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 745143139354 time 745202907771 state 0x2 ref 0 00000100:00100000:0.0:1789760765.688012:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760765.689033:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512697472 00000100:00100000:1.0:1789760765.689045:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512697472 00000100:00100000:1.0:1789760765.689098:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 745143139354 time 745202907771 state 0x0 ref 1 00000100:00100000:1.0:1789760765.689185:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 745203168187 time 745202907771 state 0x1 ref 1 00000100:00100000:1.0:1789760765.689319:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 745203168187 time 745253617252 state 0x0 ref 1 00000100:00100000:1.0:1789760765.689333:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac287aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512697472:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760765.689386:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26718208, mtime: 0x6aad94fd, atime: 0x0, ctime: 0x6aad94fd, blocks: 0xd007 00010000:00010000:1.0:1789760765.689396:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2f387 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.689412:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=745s v=(0 0) 00010000:00010000:1.0:1789760765.689416:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760765.689422:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2f387 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26718208->26722303] (req 26718208->26722303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.689442:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2f387 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26718208->26722303] (req 26718208->26722303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760765.722126:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973842 is committed 00002000:00010000:1.0:1789760765.722244:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26718208 -> 26722304 00010000:00010000:1.0:1789760765.722260:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2f387 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26718208->26722303] (req 26718208->26722303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.722273:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2f387 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26718208->26722303] (req 26718208->26722303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.722283:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2f387 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26718208->26722303] (req 26718208->26722303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.722291:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760765.722296:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2f387 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26718208->26722303] (req 26718208->26722303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.722305:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2f387 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26718208->26722303] (req 26718208->26722303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.722320:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2f387 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26718208->26722303] (req 26718208->26722303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.722325:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2f387 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26718208->26722303] (req 26718208->26722303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760765.722620:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac287aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512697472:12345-192.168.203.56@tcp:4:dd.0 Request processed in 33283us (34610us total) trans 4294973842 rc 0/0 00000100:00100000:1.0:1789760765.722631:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6553 00000100:00100000:1.0:1789760765.722641:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 745203168187 time 745253617252 state 0x2 ref 0 00000100:00100000:1.0:1789760765.725608:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760765.725804:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512697600 00000100:00100000:3.0:1789760765.725814:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512697600 00000100:00100000:3.0:1789760765.725868:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 745203168187 time 745253617252 state 0x0 ref 1 00000100:00100000:3.0:1789760765.725885:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 745253877668 time 745253617252 state 0x1 ref 1 00000100:00100000:3.0:1789760765.725966:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 745253877668 time 745290264847 state 0x0 ref 1 00000100:00100000:3.0:1789760765.725980:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4039c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512697600:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760765.726032:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26722304, mtime: 0x6aad94fd, atime: 0x0, ctime: 0x6aad94fd, blocks: 0xd007 00010000:00010000:3.0:1789760765.726041:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2f38e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.726060:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=745s v=(0 0) 00010000:00010000:3.0:1789760765.726064:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760765.726068:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2f38e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26722304->26726399] (req 26722304->26726399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.726078:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2f38e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26722304->26726399] (req 26722304->26726399) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760765.761593:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973843 is committed 00002000:00010000:3.0:1789760765.761686:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26722304 -> 26726400 00010000:00010000:3.0:1789760765.761722:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2f38e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26722304->26726399] (req 26722304->26726399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.761735:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2f38e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26722304->26726399] (req 26722304->26726399) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.761745:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2f38e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26722304->26726399] (req 26722304->26726399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.761752:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760765.761757:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2f38e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26722304->26726399] (req 26722304->26726399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.761765:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2f38e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26722304->26726399] (req 26722304->26726399) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.761787:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2f38e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26722304->26726399] (req 26722304->26726399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.761796:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2f38e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26722304->26726399] (req 26722304->26726399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760765.761904:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4039c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512697600:12345-192.168.203.56@tcp:4:dd.0 Request processed in 35921us (36301us total) trans 4294973843 rc 0/0 00000100:00100000:3.0:1789760765.761914:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6554 00000100:00100000:3.0:1789760765.761924:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 745253877668 time 745290264847 state 0x2 ref 0 00000100:00100000:2.0:1789760765.765818:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760765.765870:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512697728 00000100:00100000:0.0:1789760765.765889:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512697728 00000100:00100000:0.0:1789760765.765945:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 745253877668 time 745290264847 state 0x0 ref 1 00000100:00100000:0.0:1789760765.765960:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 745290525263 time 745290264847 state 0x1 ref 1 00000100:00100000:0.0:1789760765.765988:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 745290525263 time 745330288074 state 0x0 ref 1 00000100:00100000:0.0:1789760765.766002:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febc7b33480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512697728:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760765.766053:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26726400, mtime: 0x6aad94fd, atime: 0x0, ctime: 0x6aad94fd, blocks: 0xd007 00010000:00010000:0.0:1789760765.766062:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2f395 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.766078:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=745s v=(0 0) 00010000:00010000:0.0:1789760765.766082:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760765.766089:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2f395 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26726400->26730495] (req 26726400->26730495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.766102:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2f395 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26726400->26730495] (req 26726400->26730495) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760765.812519:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973844 is committed 00002000:00010000:1.0:1789760765.812774:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26726400 -> 26730496 00010000:00010000:1.0:1789760765.812791:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2f395 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26726400->26730495] (req 26726400->26730495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.812803:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2f395 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26726400->26730495] (req 26726400->26730495) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.812812:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2f395 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26726400->26730495] (req 26726400->26730495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.812820:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760765.812825:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2f395 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26726400->26730495] (req 26726400->26730495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.812834:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2f395 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26726400->26730495] (req 26726400->26730495) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.812854:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2f395 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26726400->26730495] (req 26726400->26730495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.812862:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2f395 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26726400->26730495] (req 26726400->26730495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760765.812958:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febc7b33480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512697728:12345-192.168.203.56@tcp:4:dd.0 Request processed in 46953us (47144us total) trans 4294973844 rc 0/0 00000100:00100000:1.0:1789760765.812969:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6555 00000100:00100000:1.0:1789760765.812979:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 745290525263 time 745330288074 state 0x2 ref 0 00000100:00100000:0.0:1789760765.815850:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760765.816114:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512697856 00000100:00100000:1.0:1789760765.816123:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512697856 00000100:00100000:1.0:1789760765.816185:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 745290525263 time 745330288074 state 0x0 ref 1 00000100:00100000:1.0:1789760765.816199:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 745330548490 time 745330288074 state 0x1 ref 1 00000100:00100000:1.0:1789760765.816229:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 745330548490 time 745380528828 state 0x0 ref 1 00000100:00100000:1.0:1789760765.816241:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2879880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512697856:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760765.816287:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26730496, mtime: 0x6aad94fd, atime: 0x0, ctime: 0x6aad94fd, blocks: 0xd007 00010000:00010000:1.0:1789760765.816295:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2f39c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.816311:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=745s v=(0 0) 00010000:00010000:1.0:1789760765.816315:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760765.816321:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2f39c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26730496->26734591] (req 26730496->26734591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.816340:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2f39c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26730496->26734591] (req 26730496->26734591) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760765.871760:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973845 is committed 00002000:00010000:1.0:1789760765.871885:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26730496 -> 26734592 00010000:00010000:1.0:1789760765.871901:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2f39c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26730496->26734591] (req 26730496->26734591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.871913:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2f39c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26730496->26734591] (req 26730496->26734591) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.871923:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2f39c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26730496->26734591] (req 26730496->26734591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.871931:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760765.871935:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2f39c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26730496->26734591] (req 26730496->26734591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.871945:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2f39c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26730496->26734591] (req 26730496->26734591) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.871965:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2f39c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26730496->26734591] (req 26730496->26734591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.871973:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2f39c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26730496->26734591] (req 26730496->26734591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760765.872078:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2879880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512697856:12345-192.168.203.56@tcp:4:dd.0 Request processed in 55832us (56241us total) trans 4294973845 rc 0/0 00000100:00100000:1.0:1789760765.872088:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6556 00000100:00100000:1.0:1789760765.872098:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 745330548490 time 745380528828 state 0x2 ref 0 00000100:00100000:0.0:1789760765.879975:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760765.880076:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512697984 00000100:00100000:3.0:1789760765.880086:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512697984 00000100:00100000:3.0:1789760765.880140:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 745330548490 time 745380528828 state 0x0 ref 1 00000100:00100000:3.0:1789760765.880166:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 745380789244 time 745380528828 state 0x1 ref 1 00000100:00100000:3.0:1789760765.880199:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 745380789244 time 745444499050 state 0x0 ref 1 00000100:00100000:3.0:1789760765.880213:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2879180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512697984:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760765.880260:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26734592, mtime: 0x6aad94fd, atime: 0x0, ctime: 0x6aad94fd, blocks: 0xd007 00010000:00010000:3.0:1789760765.880269:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2f3a3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.880286:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=745s v=(0 0) 00010000:00010000:3.0:1789760765.880291:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760765.880298:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2f3a3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26734592->26738687] (req 26734592->26738687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.880311:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2f3a3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26734592->26738687] (req 26734592->26738687) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760765.923186:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973846 is committed 00002000:00010000:1.0:1789760765.923340:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26734592 -> 26738688 00010000:00010000:1.0:1789760765.923355:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2f3a3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26734592->26738687] (req 26734592->26738687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.923381:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2f3a3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26734592->26738687] (req 26734592->26738687) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.923392:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2f3a3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26734592->26738687] (req 26734592->26738687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.923400:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760765.923405:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2f3a3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26734592->26738687] (req 26734592->26738687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.923415:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2f3a3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26734592->26738687] (req 26734592->26738687) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.923436:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2f3a3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26734592->26738687] (req 26734592->26738687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.923451:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2f3a3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26734592->26738687] (req 26734592->26738687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760765.923539:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2879180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512697984:12345-192.168.203.56@tcp:4:dd.0 Request processed in 43323us (43569us total) trans 4294973846 rc 0/0 00000100:00100000:1.0:1789760765.923548:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6557 00000100:00100000:1.0:1789760765.923558:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 745380789244 time 745444499050 state 0x2 ref 0 00000100:00100000:3.0:1789760765.926296:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760765.926430:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512698112 00000100:00100000:1.0:1789760765.926441:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512698112 00000100:00100000:1.0:1789760765.926494:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 745380789244 time 745444499050 state 0x0 ref 1 00000100:00100000:1.0:1789760765.926509:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 745444759466 time 745444499050 state 0x1 ref 1 00000100:00100000:1.0:1789760765.926540:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 745444759466 time 745490839726 state 0x0 ref 1 00000100:00100000:1.0:1789760765.926560:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5306680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512698112:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760765.926608:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26738688, mtime: 0x6aad94fd, atime: 0x0, ctime: 0x6aad94fd, blocks: 0xd007 00010000:00010000:1.0:1789760765.926616:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2f3aa lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.926633:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=745s v=(0 0) 00010000:00010000:1.0:1789760765.926638:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760765.926645:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2f3aa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26738688->26742783] (req 26738688->26742783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.926665:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2f3aa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26738688->26742783] (req 26738688->26742783) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760765.987763:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973847 is committed 00002000:00010000:1.0:1789760765.991021:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26738688 -> 26742784 00010000:00010000:1.0:1789760765.991041:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2f3aa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26738688->26742783] (req 26738688->26742783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.991052:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2f3aa lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26738688->26742783] (req 26738688->26742783) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.991061:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2f3aa lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26738688->26742783] (req 26738688->26742783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.991069:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760765.991073:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2f3aa lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26738688->26742783] (req 26738688->26742783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.991081:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2f3aa lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26738688->26742783] (req 26738688->26742783) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.991101:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2f3aa lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26738688->26742783] (req 26738688->26742783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.991109:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2f3aa lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26738688->26742783] (req 26738688->26742783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760765.991242:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5306680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512698112:12345-192.168.203.56@tcp:4:dd.0 Request processed in 64679us (64950us total) trans 4294973847 rc 0/0 00000100:00100000:1.0:1789760765.991252:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6558 00000100:00100000:1.0:1789760765.991263:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 745444759466 time 745490839726 state 0x2 ref 0 00000100:00100000:0.0:1789760766.003006:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760766.003105:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512698240 00000100:00100000:1.0:1789760766.003115:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512698240 00000100:00100000:1.0:1789760766.003181:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 745444759466 time 745490839726 state 0x0 ref 1 00000100:00100000:1.0:1789760766.003197:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 745491100142 time 745490839726 state 0x1 ref 1 00000100:00100000:1.0:1789760766.003230:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 745491100142 time 745567529383 state 0x0 ref 1 00000100:00100000:1.0:1789760766.003242:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2879500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512698240:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760766.003293:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26742784, mtime: 0x6aad94fd, atime: 0x0, ctime: 0x6aad94fd, blocks: 0xd007 00010000:00010000:1.0:1789760766.003303:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2f3b1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.003320:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=745s v=(0 0) 00010000:00010000:1.0:1789760766.003324:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760766.003331:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2f3b1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26742784->26746879] (req 26742784->26746879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.003344:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2f3b1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26742784->26746879] (req 26742784->26746879) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760766.045031:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973848 is committed 00002000:00010000:1.0:1789760766.045133:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26742784 -> 26746880 00010000:00010000:1.0:1789760766.045149:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2f3b1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26742784->26746879] (req 26742784->26746879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.045174:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2f3b1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26742784->26746879] (req 26742784->26746879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.045184:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2f3b1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26742784->26746879] (req 26742784->26746879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.045192:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760766.045197:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2f3b1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26742784->26746879] (req 26742784->26746879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.045206:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2f3b1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26742784->26746879] (req 26742784->26746879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.045302:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2f3b1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26742784->26746879] (req 26742784->26746879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.045310:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2f3b1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26742784->26746879] (req 26742784->26746879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760766.045441:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2879500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512698240:12345-192.168.203.56@tcp:4:dd.0 Request processed in 42195us (42440us total) trans 4294973848 rc 0/0 00000100:00100000:1.0:1789760766.045454:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6559 00000100:00100000:1.0:1789760766.045465:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 745491100142 time 745567529383 state 0x2 ref 0 00000100:00100000:0.0:1789760766.048960:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760766.049062:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512698368 00000100:00100000:1.0:1789760766.049072:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512698368 00000100:00100000:1.0:1789760766.049120:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 745491100142 time 745567529383 state 0x0 ref 1 00000100:00100000:1.0:1789760766.049134:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 745567789799 time 745567529383 state 0x1 ref 1 00000100:00100000:1.0:1789760766.049169:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 745567789799 time 745613468584 state 0x0 ref 1 00000100:00100000:1.0:1789760766.049180:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febd02e0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512698368:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760766.049329:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26746880, mtime: 0x6aad94fd, atime: 0x0, ctime: 0x6aad94fd, blocks: 0xd007 00010000:00010000:1.0:1789760766.049338:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2f3b8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.049354:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=745s v=(0 0) 00010000:00010000:1.0:1789760766.049358:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760766.049364:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2f3b8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26746880->26750975] (req 26746880->26750975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.049378:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2f3b8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26746880->26750975] (req 26746880->26750975) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760766.104609:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973849 is committed 00002000:00010000:1.0:1789760766.104811:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26746880 -> 26750976 00010000:00010000:1.0:1789760766.104843:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2f3b8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26746880->26750975] (req 26746880->26750975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.104857:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2f3b8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26746880->26750975] (req 26746880->26750975) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.104865:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2f3b8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26746880->26750975] (req 26746880->26750975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.104873:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760766.104877:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2f3b8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26746880->26750975] (req 26746880->26750975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.104886:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2f3b8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26746880->26750975] (req 26746880->26750975) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.104905:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2f3b8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26746880->26750975] (req 26746880->26750975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.104912:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2f3b8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26746880->26750975] (req 26746880->26750975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760766.105003:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febd02e0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512698368:12345-192.168.203.56@tcp:4:dd.0 Request processed in 55820us (56304us total) trans 4294973849 rc 0/0 00000100:00100000:1.0:1789760766.105012:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6560 00000100:00100000:1.0:1789760766.105022:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 745567789799 time 745613468584 state 0x2 ref 0 00000100:00100000:3.0:1789760766.122064:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760766.122434:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512698496 00000100:00100000:1.0:1789760766.122446:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512698496 00000100:00100000:1.0:1789760766.122500:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 745567789799 time 745613468584 state 0x0 ref 1 00000100:00100000:1.0:1789760766.122515:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 745613729000 time 745613468584 state 0x1 ref 1 00000100:00100000:1.0:1789760766.122545:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 745613729000 time 745686844440 state 0x0 ref 1 00000100:00100000:1.0:1789760766.122556:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5307100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512698496:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760766.122607:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26750976, mtime: 0x6aad94fd, atime: 0x0, ctime: 0x6aad94fd, blocks: 0xd007 00010000:00010000:1.0:1789760766.122615:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2f3bf lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.122641:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=745s v=(0 0) 00010000:00010000:1.0:1789760766.122646:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760766.122653:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2f3bf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26750976->26755071] (req 26750976->26755071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.123131:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2f3bf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26750976->26755071] (req 26750976->26755071) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760766.190073:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973850 is committed 00002000:00010000:2.0:1789760766.190193:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26750976 -> 26755072 00010000:00010000:2.0:1789760766.190210:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2f3bf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26750976->26755071] (req 26750976->26755071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.190224:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2f3bf lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26750976->26755071] (req 26750976->26755071) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.190234:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2f3bf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26750976->26755071] (req 26750976->26755071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.190241:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760766.190246:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2f3bf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26750976->26755071] (req 26750976->26755071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.190257:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2f3bf lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26750976->26755071] (req 26750976->26755071) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.190277:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2f3bf lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26750976->26755071] (req 26750976->26755071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.190292:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2f3bf lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26750976->26755071] (req 26750976->26755071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760766.190383:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5307100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512698496:12345-192.168.203.56@tcp:4:dd.0 Request processed in 67823us (68325us total) trans 4294973850 rc 0/0 00000100:00100000:2.0:1789760766.190394:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6561 00000100:00100000:2.0:1789760766.190404:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 745613729000 time 745686844440 state 0x2 ref 0 00000100:00100000:2.0:1789760766.193448:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760766.193538:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512698624 00000100:00100000:1.0:1789760766.193605:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512698624 00000100:00100000:1.0:1789760766.193660:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 745613729000 time 745686844440 state 0x0 ref 1 00000100:00100000:1.0:1789760766.193675:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 745687104856 time 745686844440 state 0x1 ref 1 00000100:00100000:1.0:1789760766.193760:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 745687104856 time 745758058619 state 0x0 ref 1 00000100:00100000:1.0:1789760766.193774:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febc7b32680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512698624:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760766.193819:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26755072, mtime: 0x6aad94fd, atime: 0x0, ctime: 0x6aad94fd, blocks: 0xd007 00010000:00010000:1.0:1789760766.193828:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2f3c6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.193844:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=745s v=(0 0) 00010000:00010000:1.0:1789760766.193848:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760766.193854:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2f3c6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26755072->26759167] (req 26755072->26759167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.193868:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2f3c6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26755072->26759167] (req 26755072->26759167) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760766.226755:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26755072 -> 26759168 00010000:00010000:1.0:1789760766.226776:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2f3c6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26755072->26759167] (req 26755072->26759167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.226790:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2f3c6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26755072->26759167] (req 26755072->26759167) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.226799:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2f3c6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26755072->26759167] (req 26755072->26759167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.226807:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760766.226811:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2f3c6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26755072->26759167] (req 26755072->26759167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.226830:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2f3c6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26755072->26759167] (req 26755072->26759167) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.226849:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2f3c6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26755072->26759167] (req 26755072->26759167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.226857:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2f3c6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26755072->26759167] (req 26755072->26759167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760766.226951:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febc7b32680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2360:x1876699512698624:12345-192.168.203.56@tcp:4:dd.0 Request processed in 33173us (33507us total) trans 4294973851 rc 0/0 00000100:00100000:1.0:1789760766.226961:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6562 00000100:00100000:1.0:1789760766.226973:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 745687104856 time 745758058619 state 0x2 ref 0 00000001:00080000:3.0:1789760766.227108:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973851 is committed 00000100:00100000:3.0:1789760766.231136:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760766.231180:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512698752 00000100:00100000:3.0:1789760766.231190:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512698752 00000100:00100000:3.0:1789760766.231243:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 745687104856 time 745758058619 state 0x0 ref 1 00000100:00100000:3.0:1789760766.231258:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 745758319035 time 745758058619 state 0x1 ref 1 00000100:00100000:3.0:1789760766.231283:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 745758319035 time 745795582751 state 0x0 ref 1 00000100:00100000:3.0:1789760766.231295:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5304a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512698752:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760766.231341:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26759168, mtime: 0x6aad94fd, atime: 0x0, ctime: 0x6aad94fd, blocks: 0xd007 00010000:00010000:3.0:1789760766.231351:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2f3cd lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.231367:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=745s v=(0 0) 00010000:00010000:3.0:1789760766.231372:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760766.231378:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2f3cd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26759168->26763263] (req 26759168->26763263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.231392:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2f3cd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26759168->26763263] (req 26759168->26763263) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760766.273523:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26759168 -> 26763264 00000001:00080000:3.0:1789760766.273554:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973852 is committed 00010000:00010000:1.0:1789760766.273614:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2f3cd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26759168->26763263] (req 26759168->26763263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.273836:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2f3cd lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26759168->26763263] (req 26759168->26763263) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.273846:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2f3cd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26759168->26763263] (req 26759168->26763263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.273853:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760766.273859:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2f3cd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26759168->26763263] (req 26759168->26763263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.273869:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2f3cd lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26759168->26763263] (req 26759168->26763263) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.273943:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2f3cd lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26759168->26763263] (req 26759168->26763263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.273952:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2f3cd lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26759168->26763263] (req 26759168->26763263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760766.274050:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5304a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512698752:12345-192.168.203.56@tcp:4:dd.0 Request processed in 42751us (42916us total) trans 4294973852 rc 0/0 00000100:00100000:1.0:1789760766.274059:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6563 00000100:00100000:1.0:1789760766.274069:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 745758319035 time 745795582751 state 0x2 ref 0 00000100:00100000:2.0:1789760766.276395:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760766.276523:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512698880 00000100:00100000:1.0:1789760766.276532:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512698880 00000100:00100000:1.0:1789760766.276684:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 745758319035 time 745795582751 state 0x0 ref 1 00000100:00100000:1.0:1789760766.276758:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 745795843167 time 745795582751 state 0x1 ref 1 00000100:00100000:1.0:1789760766.276823:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 745795843167 time 745841122069 state 0x0 ref 1 00000100:00100000:1.0:1789760766.276836:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac531df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512698880:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760766.276885:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26763264, mtime: 0x6aad94fd, atime: 0x0, ctime: 0x6aad94fd, blocks: 0xd007 00010000:00010000:1.0:1789760766.276894:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2f3d4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.276909:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=745s v=(0 0) 00010000:00010000:1.0:1789760766.276913:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760766.276919:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2f3d4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26763264->26767359] (req 26763264->26767359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.276933:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2f3d4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26763264->26767359] (req 26763264->26767359) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.325016:0:5836:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:2.0:1789760766.327369:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26763264 -> 26767360 00010000:00010000:2.0:1789760766.327387:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2f3d4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26763264->26767359] (req 26763264->26767359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.327400:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2f3d4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26763264->26767359] (req 26763264->26767359) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.327410:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2f3d4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26763264->26767359] (req 26763264->26767359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.327418:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760766.327422:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2f3d4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26763264->26767359] (req 26763264->26767359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.327432:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2f3d4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26763264->26767359] (req 26763264->26767359) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.327451:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2f3d4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26763264->26767359] (req 26763264->26767359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.327458:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2f3d4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26763264->26767359] (req 26763264->26767359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760766.327560:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac531df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2360:x1876699512698880:12345-192.168.203.56@tcp:4:dd.0 Request processed in 50722us (51168us total) trans 4294973853 rc 0/0 00000100:00100000:2.0:1789760766.327571:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6564 00000100:00100000:2.0:1789760766.327582:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 745795843167 time 745841122069 state 0x2 ref 0 00000001:00080000:3.0:1789760766.327654:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973853 is committed 00000100:00100000:2.0:1789760766.330791:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760766.330966:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512699008 00000100:00100000:1.0:1789760766.330977:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512699008 00000100:00100000:1.0:1789760766.331030:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 745795843167 time 745841122069 state 0x0 ref 1 00000100:00100000:1.0:1789760766.331044:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 745841382485 time 745841122069 state 0x1 ref 1 00000100:00100000:1.0:1789760766.331074:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 745841382485 time 745895374257 state 0x0 ref 1 00000100:00100000:1.0:1789760766.331086:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac531f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512699008:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760766.331131:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26767360, mtime: 0x6aad94fd, atime: 0x0, ctime: 0x6aad94fd, blocks: 0xd007 00010000:00010000:1.0:1789760766.331140:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2f3db lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.331169:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=745s v=(0 0) 00010000:00010000:1.0:1789760766.331173:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760766.331180:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2f3db lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26767360->26771455] (req 26767360->26771455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.331194:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2f3db lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26767360->26771455] (req 26767360->26771455) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760766.374461:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973854 is committed 00002000:00010000:0.0:1789760766.376780:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26767360 -> 26771456 00010000:00010000:0.0:1789760766.376801:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2f3db lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26767360->26771455] (req 26767360->26771455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.376815:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2f3db lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26767360->26771455] (req 26767360->26771455) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.376824:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2f3db lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26767360->26771455] (req 26767360->26771455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.376831:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760766.376835:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2f3db lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26767360->26771455] (req 26767360->26771455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.376845:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2f3db lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26767360->26771455] (req 26767360->26771455) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.376863:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2f3db lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26767360->26771455] (req 26767360->26771455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.376871:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2f3db lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26767360->26771455] (req 26767360->26771455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760766.376972:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac531f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512699008:12345-192.168.203.56@tcp:4:dd.0 Request processed in 45882us (46185us total) trans 4294973854 rc 0/0 00000100:00100000:0.0:1789760766.376986:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6565 00000100:00100000:0.0:1789760766.376996:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 745841382485 time 745895374257 state 0x2 ref 0 00000100:00100000:0.0:1789760766.381528:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760766.381714:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512699136 00000100:00100000:1.0:1789760766.381725:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512699136 00000100:00100000:1.0:1789760766.381779:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 745841382485 time 745895374257 state 0x0 ref 1 00000100:00100000:1.0:1789760766.381796:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 745895634673 time 745895374257 state 0x1 ref 1 00000100:00100000:1.0:1789760766.381827:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 745895634673 time 745946126501 state 0x0 ref 1 00000100:00100000:1.0:1789760766.381840:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2875c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512699136:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760766.381900:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26771456, mtime: 0x6aad94fd, atime: 0x0, ctime: 0x6aad94fd, blocks: 0xd007 00010000:00010000:1.0:1789760766.381922:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2f3e2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.381938:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=745s v=(0 0) 00010000:00010000:1.0:1789760766.381942:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760766.381949:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2f3e2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26771456->26775551] (req 26771456->26775551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.381963:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2f3e2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26771456->26775551] (req 26771456->26775551) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760766.417807:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973855 is committed 00002000:00010000:0.0:1789760766.417976:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26771456 -> 26775552 00010000:00010000:0.0:1789760766.417993:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2f3e2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26771456->26775551] (req 26771456->26775551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.418005:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2f3e2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26771456->26775551] (req 26771456->26775551) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.418013:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2f3e2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26771456->26775551] (req 26771456->26775551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.418020:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760766.418025:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2f3e2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26771456->26775551] (req 26771456->26775551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.418034:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2f3e2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26771456->26775551] (req 26771456->26775551) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.418053:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2f3e2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26771456->26775551] (req 26771456->26775551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.418061:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2f3e2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26771456->26775551] (req 26771456->26775551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760766.418201:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2875c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512699136:12345-192.168.203.56@tcp:4:dd.0 Request processed in 36358us (36678us total) trans 4294973855 rc 0/0 00000100:00100000:0.0:1789760766.418212:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6566 00000100:00100000:0.0:1789760766.418223:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 745895634673 time 745946126501 state 0x2 ref 0 00000100:00100000:3.0:1789760766.422834:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760766.422987:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512699264 00000100:00100000:2.0:1789760766.422997:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512699264 00000100:00100000:2.0:1789760766.423052:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 745895634673 time 745946126501 state 0x0 ref 1 00000100:00100000:2.0:1789760766.423067:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 745946386917 time 745946126501 state 0x1 ref 1 00000100:00100000:2.0:1789760766.423096:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 745946386917 time 745987395850 state 0x0 ref 1 00000100:00100000:2.0:1789760766.423107:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5305880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512699264:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760766.423326:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26775552, mtime: 0x6aad94fd, atime: 0x0, ctime: 0x6aad94fd, blocks: 0xd007 00010000:00010000:2.0:1789760766.423336:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2f3e9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.423351:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=745s v=(0 0) 00010000:00010000:2.0:1789760766.423356:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760766.423364:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2f3e9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26775552->26779647] (req 26775552->26779647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.423377:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2f3e9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26775552->26779647] (req 26775552->26779647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760766.461249:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973856 is committed 00002000:00010000:3.0:1789760766.461346:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26775552 -> 26779648 00010000:00010000:3.0:1789760766.461361:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2f3e9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26775552->26779647] (req 26775552->26779647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.461373:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2f3e9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26775552->26779647] (req 26775552->26779647) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.461382:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2f3e9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26775552->26779647] (req 26775552->26779647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.461389:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760766.461394:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2f3e9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26775552->26779647] (req 26775552->26779647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.461403:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2f3e9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26775552->26779647] (req 26775552->26779647) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.461426:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2f3e9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26775552->26779647] (req 26775552->26779647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.461433:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2f3e9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26775552->26779647] (req 26775552->26779647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760766.461552:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5305880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512699264:12345-192.168.203.56@tcp:4:dd.0 Request processed in 38441us (38728us total) trans 4294973856 rc 0/0 00000100:00100000:3.0:1789760766.461562:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6567 00000100:00100000:3.0:1789760766.461571:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 745946386917 time 745987395850 state 0x2 ref 0 00000100:00100000:2.0:1789760766.464811:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760766.465046:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512699392 00000100:00100000:0.0:1789760766.465059:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512699392 00000100:00100000:0.0:1789760766.465121:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 745946386917 time 745987395850 state 0x0 ref 1 00000100:00100000:0.0:1789760766.465136:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 745987656266 time 745987395850 state 0x1 ref 1 00000100:00100000:0.0:1789760766.465177:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 745987656266 time 746029477032 state 0x0 ref 1 00000100:00100000:0.0:1789760766.465189:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac531c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512699392:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760766.465240:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26779648, mtime: 0x6aad94fd, atime: 0x0, ctime: 0x6aad94fd, blocks: 0xd007 00010000:00010000:0.0:1789760766.465250:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f3f0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.465265:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=746s v=(0 0) 00010000:00010000:0.0:1789760766.465270:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760766.465277:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f3f0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26779648->26783743] (req 26779648->26783743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.465292:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f3f0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26779648->26783743] (req 26779648->26783743) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760766.494644:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973857 is committed 00002000:00010000:2.0:1789760766.494797:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26779648 -> 26783744 00010000:00010000:2.0:1789760766.494812:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f3f0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26779648->26783743] (req 26779648->26783743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.494882:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f3f0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26779648->26783743] (req 26779648->26783743) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.494893:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f3f0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26779648->26783743] (req 26779648->26783743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.494900:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760766.494905:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f3f0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26779648->26783743] (req 26779648->26783743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.494915:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f3f0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26779648->26783743] (req 26779648->26783743) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.494938:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f3f0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26779648->26783743] (req 26779648->26783743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.494946:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f3f0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26779648->26783743] (req 26779648->26783743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760766.495068:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac531c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512699392:12345-192.168.203.56@tcp:4:dd.0 Request processed in 29876us (30261us total) trans 4294973857 rc 0/0 00000100:00100000:2.0:1789760766.495078:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6568 00000100:00100000:2.0:1789760766.495088:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 745987656266 time 746029477032 state 0x2 ref 0 00000100:00100000:0.0:1789760766.498475:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760766.498563:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512699520 00000100:00100000:1.0:1789760766.498573:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512699520 00000100:00100000:1.0:1789760766.498628:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 745987656266 time 746029477032 state 0x0 ref 1 00000100:00100000:1.0:1789760766.498643:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 746029737448 time 746029477032 state 0x1 ref 1 00000100:00100000:1.0:1789760766.498672:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 746029737448 time 746062971532 state 0x0 ref 1 00000100:00100000:1.0:1789760766.498683:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2877100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512699520:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760766.498744:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26783744, mtime: 0x6aad94fd, atime: 0x0, ctime: 0x6aad94fd, blocks: 0xd007 00010000:00010000:1.0:1789760766.498753:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2f3f7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.498770:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=746s v=(0 0) 00010000:00010000:1.0:1789760766.498774:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760766.498781:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2f3f7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26783744->26787839] (req 26783744->26787839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.498794:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2f3f7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26783744->26787839] (req 26783744->26787839) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760766.520931:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973858 is committed 00002000:00010000:1.0:1789760766.521029:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26783744 -> 26787840 00002000:00010000:1.0:1789760766.521034:0:6710:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760765 -> 1789760766 00002000:00010000:1.0:1789760766.521046:0:6710:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760765 -> 1789760766 00010000:00010000:1.0:1789760766.521064:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2f3f7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26783744->26787839] (req 26783744->26787839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.521078:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2f3f7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26783744->26787839] (req 26783744->26787839) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.521088:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2f3f7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26783744->26787839] (req 26783744->26787839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.521096:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760766.521102:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2f3f7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26783744->26787839] (req 26783744->26787839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.521116:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2f3f7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26783744->26787839] (req 26783744->26787839) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.521135:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2f3f7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26783744->26787839] (req 26783744->26787839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.521143:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2f3f7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26783744->26787839] (req 26783744->26787839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760766.521286:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2877100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512699520:12345-192.168.203.56@tcp:4:dd.0 Request processed in 22599us (22813us total) trans 4294973858 rc 0/0 00000100:00100000:1.0:1789760766.521297:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6569 00000100:00100000:1.0:1789760766.521307:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 746029737448 time 746062971532 state 0x2 ref 0 00000100:00100000:0.0:1789760766.523648:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760766.523737:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512699648 00000100:00100000:3.0:1789760766.523747:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512699648 00000100:00100000:3.0:1789760766.523800:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 746029737448 time 746062971532 state 0x0 ref 1 00000100:00100000:3.0:1789760766.523817:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 746063231948 time 746062971532 state 0x1 ref 1 00000100:00100000:3.0:1789760766.523858:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 746063231948 time 746088157439 state 0x0 ref 1 00000100:00100000:3.0:1789760766.523872:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac3302680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512699648:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760766.523918:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26787840, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0xd007 00010000:00010000:3.0:1789760766.523925:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2f3fe lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.523938:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=746s v=(0 0) 00010000:00010000:3.0:1789760766.523942:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760766.523946:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2f3fe lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26787840->26791935] (req 26787840->26791935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.523956:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2f3fe lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26787840->26791935] (req 26787840->26791935) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760766.560589:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26787840 -> 26791936 00010000:00010000:1.0:1789760766.560611:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2f3fe lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26787840->26791935] (req 26787840->26791935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.560629:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2f3fe lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26787840->26791935] (req 26787840->26791935) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.560638:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2f3fe lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26787840->26791935] (req 26787840->26791935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.560645:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760766.560650:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2f3fe lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26787840->26791935] (req 26787840->26791935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.560658:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2f3fe lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26787840->26791935] (req 26787840->26791935) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.560677:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2f3fe lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26787840->26791935] (req 26787840->26791935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.560686:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2f3fe lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26787840->26791935] (req 26787840->26791935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760766.560791:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973859 is committed 00000100:00100000:1.0:1789760766.560887:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac3302680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512699648:12345-192.168.203.56@tcp:4:dd.0 Request processed in 37013us (37243us total) trans 4294973859 rc 0/0 00000100:00100000:1.0:1789760766.560897:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6570 00000100:00100000:1.0:1789760766.560905:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 746063231948 time 746088157439 state 0x2 ref 0 00000100:00100000:2.0:1789760766.565413:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760766.565484:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512699776 00000100:00100000:0.0:1789760766.565495:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512699776 00000100:00100000:0.0:1789760766.565566:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 746063231948 time 746088157439 state 0x0 ref 1 00000100:00100000:0.0:1789760766.565582:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 746088417855 time 746088157439 state 0x1 ref 1 00000100:00100000:0.0:1789760766.565614:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 746088417855 time 746129913368 state 0x0 ref 1 00000100:00100000:0.0:1789760766.565628:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac531d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512699776:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760766.565684:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26791936, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0xd007 00010000:00010000:0.0:1789760766.565954:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f405 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.565975:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=746s v=(0 0) 00010000:00010000:0.0:1789760766.565980:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760766.565988:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f405 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26791936->26796031] (req 26791936->26796031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.566005:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f405 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26791936->26796031] (req 26791936->26796031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760766.603975:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26791936 -> 26796032 00010000:00010000:2.0:1789760766.603994:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f405 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26791936->26796031] (req 26791936->26796031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.604008:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f405 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26791936->26796031] (req 26791936->26796031) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.604016:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f405 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26791936->26796031] (req 26791936->26796031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.604023:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760766.604028:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f405 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26791936->26796031] (req 26791936->26796031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.604038:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f405 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26791936->26796031] (req 26791936->26796031) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.604058:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f405 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26791936->26796031] (req 26791936->26796031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.604064:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f405 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26791936->26796031] (req 26791936->26796031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760766.604175:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac531d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2358:x1876699512699776:12345-192.168.203.56@tcp:4:dd.0 Request processed in 38545us (38770us total) trans 4294973860 rc 0/0 00000100:00100000:2.0:1789760766.604185:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6571 00000100:00100000:2.0:1789760766.604194:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 746088417855 time 746129913368 state 0x2 ref 0 00000001:00080000:1.0:1789760766.604529:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973860 is committed 00000100:00100000:2.0:1789760766.608904:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760766.609190:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512699904 00000100:00100000:1.0:1789760766.609200:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512699904 00000100:00100000:1.0:1789760766.609323:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 746088417855 time 746129913368 state 0x0 ref 1 00000100:00100000:1.0:1789760766.609340:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 746130173784 time 746129913368 state 0x1 ref 1 00000100:00100000:1.0:1789760766.609373:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 746130173784 time 746173672617 state 0x0 ref 1 00000100:00100000:1.0:1789760766.609385:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac531e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512699904:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760766.609434:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26796032, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0xd007 00010000:00010000:1.0:1789760766.609442:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2f40c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.609457:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=746s v=(0 0) 00010000:00010000:1.0:1789760766.609461:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760766.609467:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2f40c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26796032->26800127] (req 26796032->26800127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.609480:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2f40c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26796032->26800127] (req 26796032->26800127) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760766.657852:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973861 is committed 00002000:00010000:0.0:1789760766.657953:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26796032 -> 26800128 00010000:00010000:0.0:1789760766.657969:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2f40c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26796032->26800127] (req 26796032->26800127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.657983:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2f40c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26796032->26800127] (req 26796032->26800127) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.657992:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2f40c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26796032->26800127] (req 26796032->26800127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.658001:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760766.658006:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2f40c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26796032->26800127] (req 26796032->26800127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.658015:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2f40c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26796032->26800127] (req 26796032->26800127) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.658033:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2f40c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26796032->26800127] (req 26796032->26800127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.658038:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2f40c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26796032->26800127] (req 26796032->26800127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760766.658127:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac531e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512699904:12345-192.168.203.56@tcp:4:dd.0 Request processed in 48737us (49228us total) trans 4294973861 rc 0/0 00000100:00100000:0.0:1789760766.658137:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6572 00000100:00100000:0.0:1789760766.658157:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 746130173784 time 746173672617 state 0x2 ref 0 00000100:00100000:3.0:1789760766.660751:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760766.660863:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512700032 00000100:00100000:1.0:1789760766.660872:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512700032 00000100:00100000:1.0:1789760766.660923:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 746130173784 time 746173672617 state 0x0 ref 1 00000100:00100000:1.0:1789760766.660939:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 746173933033 time 746173672617 state 0x1 ref 1 00000100:00100000:1.0:1789760766.660969:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 746173933033 time 746225268296 state 0x0 ref 1 00000100:00100000:1.0:1789760766.660982:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5306300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512700032:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760766.661035:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26800128, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0xd007 00010000:00010000:1.0:1789760766.661045:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2f413 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.661060:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=746s v=(0 0) 00010000:00010000:1.0:1789760766.661064:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760766.661070:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2f413 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26800128->26804223] (req 26800128->26804223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.661084:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2f413 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26800128->26804223] (req 26800128->26804223) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760766.690611:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26800128 -> 26804224 00010000:00010000:1.0:1789760766.690637:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2f413 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26800128->26804223] (req 26800128->26804223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.690650:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2f413 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26800128->26804223] (req 26800128->26804223) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.690659:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2f413 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26800128->26804223] (req 26800128->26804223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.690666:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760766.690671:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2f413 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26800128->26804223] (req 26800128->26804223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.690680:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2f413 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26800128->26804223] (req 26800128->26804223) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.690731:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2f413 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26800128->26804223] (req 26800128->26804223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.690740:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2f413 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26800128->26804223] (req 26800128->26804223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760766.690837:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5306300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2360:x1876699512700032:12345-192.168.203.56@tcp:4:dd.0 Request processed in 29851us (30089us total) trans 4294973862 rc 0/0 00000100:00100000:1.0:1789760766.690848:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6573 00000100:00100000:1.0:1789760766.690858:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 746173933033 time 746225268296 state 0x2 ref 0 00000001:00080000:3.0:1789760766.690988:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973862 is committed 00000100:00100000:2.0:1789760766.694761:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760766.694938:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512700160 00000100:00100000:0.0:1789760766.694948:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512700160 00000100:00100000:0.0:1789760766.695006:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 746173933033 time 746225268296 state 0x0 ref 1 00000100:00100000:0.0:1789760766.695022:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 746225528712 time 746225268296 state 0x1 ref 1 00000100:00100000:0.0:1789760766.695052:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 746225528712 time 746259352243 state 0x0 ref 1 00000100:00100000:0.0:1789760766.695064:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac531ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512700160:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760766.695115:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26804224, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0xd007 00010000:00010000:0.0:1789760766.695124:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2f41a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.695141:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=746s v=(0 0) 00010000:00010000:0.0:1789760766.695154:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760766.695161:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2f41a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26804224->26808319] (req 26804224->26808319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.695175:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2f41a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26804224->26808319] (req 26804224->26808319) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760766.761348:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973863 is committed 00002000:00010000:2.0:1789760766.761434:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26804224 -> 26808320 00010000:00010000:2.0:1789760766.761453:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2f41a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26804224->26808319] (req 26804224->26808319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.761466:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2f41a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26804224->26808319] (req 26804224->26808319) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.761475:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2f41a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26804224->26808319] (req 26804224->26808319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.761482:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760766.761487:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2f41a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26804224->26808319] (req 26804224->26808319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.761497:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2f41a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26804224->26808319] (req 26804224->26808319) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.761516:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2f41a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26804224->26808319] (req 26804224->26808319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.761593:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2f41a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26804224->26808319] (req 26804224->26808319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760766.761849:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac531ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512700160:12345-192.168.203.56@tcp:4:dd.0 Request processed in 66780us (67090us total) trans 4294973863 rc 0/0 00000100:00100000:2.0:1789760766.761860:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6574 00000100:00100000:2.0:1789760766.761871:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 746225528712 time 746259352243 state 0x2 ref 0 00000100:00100000:0.0:1789760766.766817:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760766.767098:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512700288 00000100:00100000:1.0:1789760766.767109:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512700288 00000100:00100000:1.0:1789760766.767179:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 746225528712 time 746259352243 state 0x0 ref 1 00000100:00100000:1.0:1789760766.767195:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 746259612659 time 746259352243 state 0x1 ref 1 00000100:00100000:1.0:1789760766.767228:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 746259612659 time 746331527566 state 0x0 ref 1 00000100:00100000:1.0:1789760766.767251:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac3300e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512700288:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760766.767299:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26808320, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0xd007 00010000:00010000:1.0:1789760766.767309:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2f421 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.767325:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=746s v=(0 0) 00010000:00010000:1.0:1789760766.767330:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760766.767337:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2f421 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26808320->26812415] (req 26808320->26812415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.767351:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2f421 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26808320->26812415] (req 26808320->26812415) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760766.809114:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973864 is committed 00002000:00010000:0.0:1789760766.809241:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26808320 -> 26812416 00010000:00010000:0.0:1789760766.809257:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2f421 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26808320->26812415] (req 26808320->26812415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.809270:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2f421 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26808320->26812415] (req 26808320->26812415) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.809284:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2f421 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26808320->26812415] (req 26808320->26812415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.809292:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760766.809297:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2f421 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26808320->26812415] (req 26808320->26812415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.809306:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2f421 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26808320->26812415] (req 26808320->26812415) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.809326:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2f421 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26808320->26812415] (req 26808320->26812415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.809333:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2f421 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26808320->26812415] (req 26808320->26812415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760766.809445:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac3300e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512700288:12345-192.168.203.56@tcp:4:dd.0 Request processed in 42192us (42637us total) trans 4294973864 rc 0/0 00000100:00100000:0.0:1789760766.809455:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6575 00000100:00100000:0.0:1789760766.809465:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 746259612659 time 746331527566 state 0x2 ref 0 00000100:00100000:3.0:1789760766.814947:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760766.815030:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512700416 00000100:00100000:1.0:1789760766.815039:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512700416 00000100:00100000:1.0:1789760766.815088:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 746259612659 time 746331527566 state 0x0 ref 1 00000100:00100000:1.0:1789760766.815103:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 746331787982 time 746331527566 state 0x1 ref 1 00000100:00100000:1.0:1789760766.815130:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 746331787982 time 746379430631 state 0x0 ref 1 00000100:00100000:1.0:1789760766.815142:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5307800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512700416:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760766.815198:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26812416, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0xd007 00010000:00010000:1.0:1789760766.815214:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2f428 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.815229:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=746s v=(0 0) 00010000:00010000:1.0:1789760766.815238:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760766.815245:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2f428 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26812416->26816511] (req 26812416->26816511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.815259:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2f428 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26812416->26816511] (req 26812416->26816511) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760766.870166:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973865 is committed 00002000:00010000:0.0:1789760766.870387:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26812416 -> 26816512 00010000:00010000:0.0:1789760766.870403:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2f428 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26812416->26816511] (req 26812416->26816511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.870421:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2f428 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26812416->26816511] (req 26812416->26816511) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.870431:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2f428 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26812416->26816511] (req 26812416->26816511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.870439:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760766.870444:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2f428 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26812416->26816511] (req 26812416->26816511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.870455:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2f428 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26812416->26816511] (req 26812416->26816511) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.870558:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2f428 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26812416->26816511] (req 26812416->26816511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.870567:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2f428 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26812416->26816511] (req 26812416->26816511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760766.871407:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5307800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512700416:12345-192.168.203.56@tcp:4:dd.0 Request processed in 56259us (56462us total) trans 4294973865 rc 0/0 00000100:00100000:0.0:1789760766.871419:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6576 00000100:00100000:0.0:1789760766.871430:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 746331787982 time 746379430631 state 0x2 ref 0 00000100:00100000:1.0:1789760766.880988:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760766.881762:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512700544 00000100:00100000:2.0:1789760766.881773:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512700544 00000100:00100000:2.0:1789760766.881823:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 746331787982 time 746379430631 state 0x0 ref 1 00000100:00100000:2.0:1789760766.881839:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 746379691047 time 746379430631 state 0x1 ref 1 00000100:00100000:2.0:1789760766.881920:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 746379691047 time 746446218874 state 0x0 ref 1 00000100:00100000:2.0:1789760766.881936:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4038e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512700544:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760766.881985:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26816512, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0xd007 00010000:00010000:2.0:1789760766.881994:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2f42f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.882011:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=746s v=(0 0) 00010000:00010000:2.0:1789760766.882015:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760766.882022:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2f42f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26816512->26820607] (req 26816512->26820607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.882035:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2f42f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26816512->26820607] (req 26816512->26820607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760766.898898:0:3312:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8feac4059180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3312:1876699540228480:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1789760766.898972:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760766.898975:0:3311:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8feac403b100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:6df98d54-eb52-4c5c-8fc9-ee9d74f90f98:3311:1876699540228352:0@lo:400:kworker.0 00000100:00100000:2.0:1789760766.899013:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760766.899041:0:3311:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8feac403bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3311:1876699540228864:0@lo:400:kworker.0 00000100:00100000:2.0:1789760766.899064:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760766.899091:0:3311:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1789760766.899097:0:3311:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8feac403b800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3311:1876699540228992:0@lo:400:kworker.0 00000100:00100000:2.0:1789760766.899123:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760766.899167:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540228352 00000100:00100000:2.0:1789760766.899176:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540228352 00000100:00100000:2.0:1789760766.899229:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 407 00000100:00100000:2.0:1789760766.899250:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febe00c7c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:6df98d54-eb52-4c5c-8fc9-ee9d74f90f98+19:3311:x1876699540228352:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760766.899310:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febe00c7c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:6df98d54-eb52-4c5c-8fc9-ee9d74f90f98+19:3311:x1876699540228352:12345-0@lo:400:kworker.0 Request processed in 62us (296us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760766.899318:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 407 00000100:00100000:2.0:1789760766.899349:0:10102:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540228864 00000100:00100000:2.0:1789760766.899353:0:10102:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540228864 00000100:00100000:2.0:1789760766.899396:0:10102:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 188 00000100:00100000:2.0:1789760766.899403:0:10102:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac531ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3311:x1876699540228864:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760766.899438:0:10102:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac531ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3311:x1876699540228864:12345-0@lo:400:kworker.0 Request processed in 35us (372us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760766.899444:0:10102:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 188 00000100:00100000:2.0:1789760766.899462:0:8229:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540228992 00000100:00100000:2.0:1789760766.899465:0:8229:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540228992 00000100:00100000:2.0:1789760766.899495:0:8229:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 496 00000100:00100000:2.0:1789760766.899502:0:8229:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac531e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3311:x1876699540228992:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760766.899533:0:8229:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac531e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3311:x1876699540228992:12345-0@lo:400:kworker.0 Request processed in 32us (410us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760766.899539:0:8229:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 496 00000100:00080000:2.0:1789760766.899559:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:2.0:1789760766.899563:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1789760766.899600:0:3311:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8feac403b800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3311:1876699540228992:0@lo:400:kworker.0 00000100:00100000:2.0:1789760766.899611:0:3311:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8feac403b100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:6df98d54-eb52-4c5c-8fc9-ee9d74f90f98:3311:1876699540228352:0@lo:400:kworker.0 00000100:00100000:2.0:1789760766.899620:0:3311:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8feac403bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3311:1876699540228864:0@lo:400:kworker.0 00000100:00100000:1.0:1789760766.900403:0:3309:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8feac4039f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3309:1876699540228608:0@lo:400:kworker.0 00000100:00100000:1.0:1789760766.900451:0:3309:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760766.900529:0:3310:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8feac4038380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3310:1876699540228736:0@lo:400:kworker.0 00000100:00100000:1.0:1789760766.900571:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760766.900613:0:3309:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8feac4038700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3309:1876699540229120:0@lo:400:kworker.0 00000100:00100000:1.0:1789760766.900644:0:3309:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760766.900650:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540228608 00000100:00100000:2.0:1789760766.900659:0:5847:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540228608 00000100:00100000:1.0:1789760766.900693:0:13064:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540229120 00000100:00100000:1.0:1789760766.900719:0:13064:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540229120 00000100:00100000:2.0:1789760766.900720:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540228736 00000100:00100000:2.0:1789760766.900727:0:5847:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540228736 00000100:00100000:2.0:1789760766.900766:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 497 00000100:00100000:1.0:1789760766.900775:0:13064:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 189 00000100:00100000:2.0:1789760766.900778:0:5847:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac403aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3309:x1876699540228608:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760766.900788:0:13064:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4039180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3309:x1876699540229120:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760766.901375:0:6707:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540228480 00000100:00100000:0.0:1789760766.901383:0:6707:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540228480 00000100:00100000:0.0:1789760766.901417:0:6707:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 206 00000100:00100000:0.0:1789760766.901425:0:6707:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febea48a850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3312:x1876699540228480:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1789760766.901443:0:6707:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 528285696 pending 1703936 free 3831496704 avail 3819962368 00000100:00100000:1.0:1789760766.901719:0:13064:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4039180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3309:x1876699540229120:12345-0@lo:400:kworker.0 Request processed in 914us (1054us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760766.901728:0:13064:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 189 00000100:00100000:2.0:1789760766.901927:0:5847:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac403aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3309:x1876699540228608:12345-0@lo:400:kworker.0 Request processed in 1150us (1473us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760766.901936:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 497 00000100:00100000:2.0:1789760766.901953:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 498 00000100:00100000:2.0:1789760766.901962:0:5847:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4039880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3310:x1876699540228736:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760766.902421:0:6707:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febea48a850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3312:x1876699540228480:12345-0@lo:13:osp-pre-0-0.0 Request processed in 994us (3447us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760766.902430:0:6707:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 206 00000100:00100000:2.0:1789760766.902441:0:5847:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4039880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3310:x1876699540228736:12345-0@lo:400:kworker.0 Request processed in 480us (1867us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760766.902449:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 498 00000100:00080000:2.0:1789760766.902483:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:2.0:1789760766.902488:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1789760766.902522:0:3309:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8feac4039f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3309:1876699540228608:0@lo:400:kworker.0 00000100:00100000:2.0:1789760766.902534:0:3309:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8feac4038700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3309:1876699540229120:0@lo:400:kworker.0 00000100:00100000:2.0:1789760766.902577:0:3312:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8feac4059180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3312:1876699540228480:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1789760766.902730:0:3310:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8feac4038380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3310:1876699540228736:0@lo:400:kworker.0 00002000:00010000:0.0:1789760766.951165:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26816512 -> 26820608 00010000:00010000:0.0:1789760766.951187:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2f42f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26816512->26820607] (req 26816512->26820607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.951201:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2f42f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26816512->26820607] (req 26816512->26820607) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.951209:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2f42f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26816512->26820607] (req 26816512->26820607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.951216:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760766.951221:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2f42f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26816512->26820607] (req 26816512->26820607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.951229:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2f42f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26816512->26820607] (req 26816512->26820607) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.951248:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2f42f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26816512->26820607] (req 26816512->26820607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.951262:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2f42f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26816512->26820607] (req 26816512->26820607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760766.951369:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4038e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2359:x1876699512700544:12345-192.168.203.56@tcp:4:dd.0 Request processed in 69431us (70386us total) trans 4294973866 rc 0/0 00000100:00100000:0.0:1789760766.951381:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6577 00000100:00100000:0.0:1789760766.951393:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 746379691047 time 746446218874 state 0x2 ref 0 00000001:00080000:1.0:1789760766.952298:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973866 is committed 00000100:00100000:2.0:1789760766.958272:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760766.958540:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512700672 00000100:00100000:0.0:1789760766.958551:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512700672 00000100:00100000:0.0:1789760766.958613:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 746379691047 time 746446218874 state 0x0 ref 1 00000100:00100000:0.0:1789760766.958629:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 746446479290 time 746446218874 state 0x1 ref 1 00000100:00100000:0.0:1789760766.958660:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 746446479290 time 746522959399 state 0x0 ref 1 00000100:00100000:0.0:1789760766.958672:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac531dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512700672:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760766.958759:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26820608, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0xd007 00010000:00010000:0.0:1789760766.958769:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f436 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.958787:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=746s v=(0 0) 00010000:00010000:0.0:1789760766.958792:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760766.958799:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f436 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26820608->26824703] (req 26820608->26824703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.958831:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f436 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26820608->26824703] (req 26820608->26824703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760766.986244:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26820608 -> 26824704 00010000:00010000:0.0:1789760766.986265:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f436 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26820608->26824703] (req 26820608->26824703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.986279:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f436 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26820608->26824703] (req 26820608->26824703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.986287:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f436 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26820608->26824703] (req 26820608->26824703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.986295:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760766.986300:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f436 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26820608->26824703] (req 26820608->26824703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.986309:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f436 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26820608->26824703] (req 26820608->26824703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.986473:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f436 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26820608->26824703] (req 26820608->26824703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.986482:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f436 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26820608->26824703] (req 26820608->26824703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760766.986621:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac531dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2358:x1876699512700672:12345-192.168.203.56@tcp:4:dd.0 Request processed in 27945us (28357us total) trans 4294973867 rc 0/0 00000100:00100000:0.0:1789760766.986631:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6578 00000100:00100000:0.0:1789760766.986641:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 746446479290 time 746522959399 state 0x2 ref 0 00000001:00080000:1.0:1789760766.986765:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973867 is committed 00000100:00100000:3.0:1789760766.991921:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760766.994684:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512700800 00000100:00100000:0.0:1789760766.995043:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512700800 00000100:00100000:0.0:1789760766.995101:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 746446479290 time 746522959399 state 0x0 ref 1 00000100:00100000:0.0:1789760766.995117:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 746523219815 time 746522959399 state 0x1 ref 1 00000100:00100000:0.0:1789760766.995197:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 746523219815 time 746559494933 state 0x0 ref 1 00000100:00100000:0.0:1789760766.995211:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5306a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512700800:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760766.995262:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26824704, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0xd007 00010000:00010000:0.0:1789760766.995271:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f43d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.995288:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=746s v=(0 0) 00010000:00010000:0.0:1789760766.995292:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760766.995298:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f43d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26824704->26828799] (req 26824704->26828799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.995312:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f43d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26824704->26828799] (req 26824704->26828799) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760767.030226:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26824704 -> 26828800 00010000:00010000:0.0:1789760767.030247:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f43d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26824704->26828799] (req 26824704->26828799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.030260:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f43d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26824704->26828799] (req 26824704->26828799) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.030268:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f43d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26824704->26828799] (req 26824704->26828799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.030275:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760767.030280:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f43d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26824704->26828799] (req 26824704->26828799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.030288:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f43d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26824704->26828799] (req 26824704->26828799) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.030308:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f43d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26824704->26828799] (req 26824704->26828799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.030389:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f43d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26824704->26828799] (req 26824704->26828799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760767.030495:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5306a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2359:x1876699512700800:12345-192.168.203.56@tcp:4:dd.0 Request processed in 35281us (38578us total) trans 4294973868 rc 0/0 00000100:00100000:0.0:1789760767.030510:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6579 00000100:00100000:0.0:1789760767.030521:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 746523219815 time 746559494933 state 0x2 ref 0 00000001:00080000:1.0:1789760767.031582:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973868 is committed 00000100:00100000:3.0:1789760767.033780:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760767.033925:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512700928 00000100:00100000:0.0:1789760767.033935:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512700928 00000100:00100000:0.0:1789760767.034003:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 746523219815 time 746559494933 state 0x0 ref 1 00000100:00100000:0.0:1789760767.034019:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 746559755349 time 746559494933 state 0x1 ref 1 00000100:00100000:0.0:1789760767.034044:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 746559755349 time 746598343750 state 0x0 ref 1 00000100:00100000:0.0:1789760767.034057:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5305180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512700928:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760767.034120:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26828800, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0xd007 00010000:00010000:0.0:1789760767.034130:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f444 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.034156:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=746s v=(0 0) 00010000:00010000:0.0:1789760767.034162:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760767.034168:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f444 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26828800->26832895] (req 26828800->26832895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.034182:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f444 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26828800->26832895] (req 26828800->26832895) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1789760767.034997:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00010000:0.0:1789760767.095269:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26828800 -> 26832896 00010000:00010000:0.0:1789760767.095289:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f444 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26828800->26832895] (req 26828800->26832895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.095302:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f444 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26828800->26832895] (req 26828800->26832895) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.095310:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f444 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26828800->26832895] (req 26828800->26832895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.095318:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760767.095323:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f444 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26828800->26832895] (req 26828800->26832895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.095332:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f444 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26828800->26832895] (req 26828800->26832895) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.095353:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f444 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26828800->26832895] (req 26828800->26832895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.095360:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f444 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26828800->26832895] (req 26828800->26832895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760767.095472:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5305180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2358:x1876699512700928:12345-192.168.203.56@tcp:4:dd.0 Request processed in 61412us (61706us total) trans 4294973869 rc 0/0 00000100:00100000:0.0:1789760767.095484:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6580 00000100:00100000:0.0:1789760767.095494:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 746559755349 time 746598343750 state 0x2 ref 0 00000001:00080000:3.0:1789760767.096227:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973869 is committed 00000100:00100000:2.0:1789760767.102490:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760767.102632:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512701056 00000100:00100000:0.0:1789760767.102643:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512701056 00000100:00100000:0.0:1789760767.102740:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 746559755349 time 746598343750 state 0x0 ref 1 00000100:00100000:0.0:1789760767.102756:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 746598604166 time 746598343750 state 0x1 ref 1 00000100:00100000:0.0:1789760767.102796:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 746598604166 time 746667095884 state 0x0 ref 1 00000100:00100000:0.0:1789760767.102808:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febea4b4450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512701056:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760767.102858:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26832896, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0xd007 00010000:00010000:0.0:1789760767.102867:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f44b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.102884:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=746s v=(0 0) 00010000:00010000:0.0:1789760767.102889:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760767.102895:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f44b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26832896->26836991] (req 26832896->26836991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.102910:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f44b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26832896->26836991] (req 26832896->26836991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760767.151560:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973870 is committed 00002000:00010000:0.0:1789760767.151754:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26832896 -> 26836992 00010000:00010000:0.0:1789760767.151915:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f44b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26832896->26836991] (req 26832896->26836991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.151928:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f44b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26832896->26836991] (req 26832896->26836991) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.151943:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f44b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26832896->26836991] (req 26832896->26836991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.151950:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760767.151955:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f44b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26832896->26836991] (req 26832896->26836991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.151965:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f44b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26832896->26836991] (req 26832896->26836991) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.151984:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f44b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26832896->26836991] (req 26832896->26836991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.151992:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f44b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26832896->26836991] (req 26832896->26836991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760767.152087:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febea4b4450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512701056:12345-192.168.203.56@tcp:4:dd.0 Request processed in 49276us (49603us total) trans 4294973870 rc 0/0 00000100:00100000:0.0:1789760767.152097:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6581 00000100:00100000:0.0:1789760767.152107:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 746598604166 time 746667095884 state 0x2 ref 0 00000100:00100000:1.0:1789760767.155784:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760767.155938:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512701184 00000100:00100000:0.0:1789760767.155948:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512701184 00000100:00100000:0.0:1789760767.156004:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 746598604166 time 746667095884 state 0x0 ref 1 00000100:00100000:0.0:1789760767.156019:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 746667356300 time 746667095884 state 0x1 ref 1 00000100:00100000:0.0:1789760767.156050:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 746667356300 time 746720349276 state 0x0 ref 1 00000100:00100000:0.0:1789760767.156061:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4038380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512701184:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760767.156112:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26836992, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0xd007 00010000:00010000:0.0:1789760767.156122:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f452 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.156140:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=746s v=(0 0) 00010000:00010000:0.0:1789760767.156156:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760767.156163:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f452 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26836992->26841087] (req 26836992->26841087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.156176:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f452 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26836992->26841087] (req 26836992->26841087) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760767.198048:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973871 is committed 00002000:00010000:0.0:1789760767.198114:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26836992 -> 26841088 00010000:00010000:0.0:1789760767.198131:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f452 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26836992->26841087] (req 26836992->26841087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.198157:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f452 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26836992->26841087] (req 26836992->26841087) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.198166:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f452 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26836992->26841087] (req 26836992->26841087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.198174:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760767.198179:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f452 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26836992->26841087] (req 26836992->26841087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.198189:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f452 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26836992->26841087] (req 26836992->26841087) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.198208:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f452 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26836992->26841087] (req 26836992->26841087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.198216:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f452 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26836992->26841087] (req 26836992->26841087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760767.198310:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4038380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512701184:12345-192.168.203.56@tcp:4:dd.0 Request processed in 42245us (42530us total) trans 4294973871 rc 0/0 00000100:00100000:0.0:1789760767.198320:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6582 00000100:00100000:0.0:1789760767.198330:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 746667356300 time 746720349276 state 0x2 ref 0 00000100:00100000:1.0:1789760767.201845:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760767.205922:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512701312 00000100:00100000:0.0:1789760767.205936:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512701312 00000100:00100000:2.0:1789760767.206104:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 746667356300 time 746720349276 state 0x0 ref 1 00000100:00100000:2.0:1789760767.206121:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 746720609692 time 746720349276 state 0x1 ref 1 00000100:00100000:2.0:1789760767.206163:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 746720609692 time 746770462819 state 0x0 ref 1 00000100:00100000:2.0:1789760767.206178:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4039500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512701312:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760767.206226:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26841088, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0xd007 00010000:00010000:2.0:1789760767.206236:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2f459 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.206563:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=746s v=(0 0) 00010000:00010000:2.0:1789760767.206569:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760767.206577:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2f459 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26841088->26845183] (req 26841088->26845183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.206595:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2f459 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26841088->26845183] (req 26841088->26845183) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760767.244801:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973872 is committed 00002000:00010000:0.0:1789760767.245140:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26841088 -> 26845184 00010000:00010000:0.0:1789760767.245166:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2f459 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26841088->26845183] (req 26841088->26845183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.245180:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2f459 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26841088->26845183] (req 26841088->26845183) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.245192:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2f459 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26841088->26845183] (req 26841088->26845183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.245204:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760767.245209:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2f459 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26841088->26845183] (req 26841088->26845183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.245217:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2f459 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26841088->26845183] (req 26841088->26845183) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.245237:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2f459 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26841088->26845183] (req 26841088->26845183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.245246:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2f459 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26841088->26845183] (req 26841088->26845183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760767.245335:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4039500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512701312:12345-192.168.203.56@tcp:4:dd.0 Request processed in 39156us (43497us total) trans 4294973872 rc 0/0 00000100:00100000:0.0:1789760767.245345:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6583 00000100:00100000:0.0:1789760767.245354:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 746720609692 time 746770462819 state 0x2 ref 0 00000100:00100000:2.0:1789760767.250441:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760767.250518:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512701440 00000100:00100000:0.0:1789760767.250526:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512701440 00000100:00100000:0.0:1789760767.250574:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 746720609692 time 746770462819 state 0x0 ref 1 00000100:00100000:0.0:1789760767.250586:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 746770723235 time 746770462819 state 0x1 ref 1 00000100:00100000:0.0:1789760767.250766:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 746770723235 time 746815064595 state 0x0 ref 1 00000100:00100000:0.0:1789760767.250784:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac531ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512701440:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760767.250835:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26845184, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0xd007 00010000:00010000:0.0:1789760767.250844:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f460 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.250861:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=746s v=(0 0) 00010000:00010000:0.0:1789760767.250866:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760767.250871:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f460 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26845184->26849279] (req 26845184->26849279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.250885:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f460 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26845184->26849279] (req 26845184->26849279) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760767.289381:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973873 is committed 00002000:00010000:0.0:1789760767.289511:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26845184 -> 26849280 00010000:00010000:0.0:1789760767.289527:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f460 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26845184->26849279] (req 26845184->26849279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.289540:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f460 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26845184->26849279] (req 26845184->26849279) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.289549:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f460 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26845184->26849279] (req 26845184->26849279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.289555:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760767.289559:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f460 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26845184->26849279] (req 26845184->26849279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.289569:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f460 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26845184->26849279] (req 26845184->26849279) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.289588:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f460 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26845184->26849279] (req 26845184->26849279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.289595:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f460 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26845184->26849279] (req 26845184->26849279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760767.289770:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac531ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512701440:12345-192.168.203.56@tcp:4:dd.0 Request processed in 38984us (39333us total) trans 4294973873 rc 0/0 00000100:00100000:0.0:1789760767.289780:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6584 00000100:00100000:0.0:1789760767.289790:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 746770723235 time 746815064595 state 0x2 ref 0 00000100:00100000:2.0:1789760767.292435:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760767.292516:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512701568 00000100:00100000:0.0:1789760767.292526:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512701568 00000100:00100000:0.0:1789760767.292682:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 746770723235 time 746815064595 state 0x0 ref 1 00000100:00100000:0.0:1789760767.292887:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 746815325011 time 746815064595 state 0x1 ref 1 00000100:00100000:0.0:1789760767.292917:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 746815325011 time 746857216702 state 0x0 ref 1 00000100:00100000:0.0:1789760767.292930:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac531f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512701568:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760767.292976:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26849280, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0xd007 00010000:00010000:0.0:1789760767.292984:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2f467 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.292999:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=746s v=(0 0) 00010000:00010000:0.0:1789760767.293004:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760767.293010:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2f467 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26849280->26853375] (req 26849280->26853375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.293023:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2f467 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26849280->26853375] (req 26849280->26853375) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760767.324663:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26849280 -> 26853376 00010000:00010000:0.0:1789760767.324683:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2f467 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26849280->26853375] (req 26849280->26853375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.324728:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2f467 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26849280->26853375] (req 26849280->26853375) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.324739:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2f467 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26849280->26853375] (req 26849280->26853375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.324746:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760767.324751:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2f467 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26849280->26853375] (req 26849280->26853375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.324760:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2f467 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26849280->26853375] (req 26849280->26853375) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.324780:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2f467 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26849280->26853375] (req 26849280->26853375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.324788:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2f467 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26849280->26853375] (req 26849280->26853375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760767.324933:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac531f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2360:x1876699512701568:12345-192.168.203.56@tcp:4:dd.0 Request processed in 32000us (32502us total) trans 4294973874 rc 0/0 00000100:00100000:0.0:1789760767.324943:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6585 00000100:00100000:0.0:1789760767.324952:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 746815325011 time 746857216702 state 0x2 ref 0 00000001:00080000:1.0:1789760767.331195:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973874 is committed 00000100:00100000:1.0:1789760767.332028:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760767.332111:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512701696 00000100:00100000:2.0:1789760767.332121:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512701696 00000100:00100000:2.0:1789760767.332187:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 746815325011 time 746857216702 state 0x0 ref 1 00000100:00100000:2.0:1789760767.332202:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 746857477118 time 746857216702 state 0x1 ref 1 00000100:00100000:2.0:1789760767.332232:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 746857477118 time 746896531508 state 0x0 ref 1 00000100:00100000:2.0:1789760767.332244:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4038a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512701696:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760767.332290:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26853376, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0xd007 00010000:00010000:2.0:1789760767.332299:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2f46e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.332319:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=746s v=(0 0) 00010000:00010000:2.0:1789760767.332323:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760767.332329:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2f46e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26853376->26857471] (req 26853376->26857471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.332343:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2f46e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26853376->26857471] (req 26853376->26857471) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.351276:0:5835:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:2.0:1789760767.365046:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26853376 -> 26857472 00010000:00010000:2.0:1789760767.365067:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2f46e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26853376->26857471] (req 26853376->26857471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.365080:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2f46e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26853376->26857471] (req 26853376->26857471) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.365089:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2f46e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26853376->26857471] (req 26853376->26857471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.365096:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760767.365101:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2f46e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26853376->26857471] (req 26853376->26857471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.365109:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2f46e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26853376->26857471] (req 26853376->26857471) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.365128:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2f46e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26853376->26857471] (req 26853376->26857471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.365137:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2f46e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26853376->26857471] (req 26853376->26857471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760767.365249:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4038a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2361:x1876699512701696:12345-192.168.203.56@tcp:4:dd.0 Request processed in 33002us (33221us total) trans 4294973875 rc 0/0 00000100:00100000:2.0:1789760767.365260:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6586 00000100:00100000:2.0:1789760767.365270:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 746857477118 time 746896531508 state 0x2 ref 0 00000001:00080000:0.0:1789760767.368006:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973875 is committed 00000100:00100000:2.0:1789760767.368357:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760767.368426:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512701824 00000100:00100000:0.0:1789760767.368435:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512701824 00000100:00100000:0.0:1789760767.368485:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 746857477118 time 746896531508 state 0x0 ref 1 00000100:00100000:0.0:1789760767.368499:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 746896791924 time 746896531508 state 0x1 ref 1 00000100:00100000:0.0:1789760767.368624:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 746896791924 time 746932920972 state 0x0 ref 1 00000100:00100000:0.0:1789760767.368639:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac531df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512701824:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760767.368692:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26857472, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0xd007 00010000:00010000:0.0:1789760767.368721:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2f475 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.368735:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=746s v=(0 0) 00010000:00010000:0.0:1789760767.368740:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760767.368747:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2f475 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26857472->26861567] (req 26857472->26861567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.368760:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2f475 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26857472->26861567] (req 26857472->26861567) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760767.402127:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26857472 -> 26861568 00000001:00080000:1.0:1789760767.402152:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973876 is committed 00010000:00010000:0.0:1789760767.402156:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2f475 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26857472->26861567] (req 26857472->26861567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.402169:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2f475 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26857472->26861567] (req 26857472->26861567) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.402178:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2f475 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26857472->26861567] (req 26857472->26861567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.402186:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760767.402191:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2f475 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26857472->26861567] (req 26857472->26861567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.402200:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2f475 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26857472->26861567] (req 26857472->26861567) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.402219:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2f475 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26857472->26861567] (req 26857472->26861567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.402227:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2f475 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26857472->26861567] (req 26857472->26861567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760767.402321:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac531df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512701824:12345-192.168.203.56@tcp:4:dd.0 Request processed in 33681us (33964us total) trans 4294973876 rc 0/0 00000100:00100000:0.0:1789760767.402330:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6587 00000100:00100000:0.0:1789760767.402339:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 746896791924 time 746932920972 state 0x2 ref 0 00000100:00100000:1.0:1789760767.405500:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760767.405757:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512701952 00000100:00100000:0.0:1789760767.405767:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512701952 00000100:00100000:0.0:1789760767.406007:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 746896791924 time 746932920972 state 0x0 ref 1 00000100:00100000:0.0:1789760767.406024:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 746933181388 time 746932920972 state 0x1 ref 1 00000100:00100000:0.0:1789760767.406052:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 746933181388 time 746970352420 state 0x0 ref 1 00000100:00100000:0.0:1789760767.406065:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac403ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512701952:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760767.406113:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26861568, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0xd007 00010000:00010000:0.0:1789760767.406122:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f47c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.406138:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=746s v=(0 0) 00010000:00010000:0.0:1789760767.406153:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760767.406159:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f47c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26861568->26865663] (req 26861568->26865663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.406173:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f47c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26861568->26865663] (req 26861568->26865663) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760767.411122:0:3309:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1789760767.411155:0:3309:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8feac4059880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3309:1876699540229248:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789760767.411584:0:3309:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760767.411995:0:6706:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540229248 00000100:00100000:0.0:1789760767.412004:0:6706:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540229248 00000100:00100000:0.0:1789760767.412053:0:6706:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 207 00000100:00100000:0.0:1789760767.412062:0:6706:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac531f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3309:x1876699540229248:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1789760767.412098:0:6706:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3859808256 avail 3857711104 00000100:00100000:0.0:1789760767.412157:0:6706:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac531f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3309:x1876699540229248:12345-0@lo:13:osp-pre-1-0.0 Request processed in 95us (572us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760767.413261:0:6706:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 207 00000100:00100000:0.0:1789760767.413334:0:3309:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8feac4059880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3309:1876699540229248:0@lo:13:osp-pre-1-0.0 00000001:00080000:2.0:1789760767.439480:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973877 is committed 00002000:00010000:1.0:1789760767.439585:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26861568 -> 26865664 00010000:00010000:1.0:1789760767.439600:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f47c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26861568->26865663] (req 26861568->26865663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.439615:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f47c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26861568->26865663] (req 26861568->26865663) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.439625:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f47c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26861568->26865663] (req 26861568->26865663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.439632:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760767.439637:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f47c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26861568->26865663] (req 26861568->26865663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.439647:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f47c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26861568->26865663] (req 26861568->26865663) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.439670:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f47c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26861568->26865663] (req 26861568->26865663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.439682:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f47c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26861568->26865663] (req 26861568->26865663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760767.439796:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac403ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512701952:12345-192.168.203.56@tcp:4:dd.0 Request processed in 33728us (34300us total) trans 4294973877 rc 0/0 00000100:00100000:1.0:1789760767.439806:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6588 00000100:00100000:1.0:1789760767.439827:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 746933181388 time 746970352420 state 0x2 ref 0 00000100:00100000:1.0:1789760767.443297:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760767.443382:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512702080 00000100:00100000:2.0:1789760767.443391:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512702080 00000100:00100000:2.0:1789760767.443434:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 746933181388 time 746970352420 state 0x0 ref 1 00000100:00100000:2.0:1789760767.443447:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 746970612836 time 746970352420 state 0x1 ref 1 00000100:00100000:2.0:1789760767.443503:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 746970612836 time 747007803530 state 0x0 ref 1 00000100:00100000:2.0:1789760767.443512:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac403a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512702080:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760767.443557:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26865664, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0xd007 00010000:00010000:2.0:1789760767.443564:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2f483 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.443581:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=747s v=(0 0) 00010000:00010000:2.0:1789760767.443584:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760767.443588:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2f483 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26865664->26869759] (req 26865664->26869759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.443598:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2f483 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26865664->26869759] (req 26865664->26869759) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760767.517035:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26865664 -> 26869760 00000001:00080000:2.0:1789760767.517037:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973878 is committed 00010000:00010000:1.0:1789760767.517056:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2f483 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26865664->26869759] (req 26865664->26869759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.517075:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2f483 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26865664->26869759] (req 26865664->26869759) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.517085:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2f483 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26865664->26869759] (req 26865664->26869759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.517093:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760767.517098:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2f483 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26865664->26869759] (req 26865664->26869759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.517198:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2f483 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26865664->26869759] (req 26865664->26869759) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.517224:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2f483 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26865664->26869759] (req 26865664->26869759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.517232:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2f483 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26865664->26869759] (req 26865664->26869759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760767.517327:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac403a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512702080:12345-192.168.203.56@tcp:4:dd.0 Request processed in 73810us (74035us total) trans 4294973878 rc 0/0 00000100:00100000:1.0:1789760767.517338:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6589 00000100:00100000:1.0:1789760767.517348:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 746970612836 time 747007803530 state 0x2 ref 0 00000100:00100000:2.0:1789760767.521247:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760767.521341:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512702208 00000100:00100000:0.0:1789760767.521351:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512702208 00000100:00100000:0.0:1789760767.521405:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 746970612836 time 747007803530 state 0x0 ref 1 00000100:00100000:0.0:1789760767.521419:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 747008063946 time 747007803530 state 0x1 ref 1 00000100:00100000:0.0:1789760767.521449:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 747008063946 time 747085749405 state 0x0 ref 1 00000100:00100000:0.0:1789760767.521460:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac531c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512702208:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760767.521506:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26869760, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0xd007 00010000:00010000:0.0:1789760767.521515:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2f48a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.521531:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=747s v=(0 0) 00010000:00010000:0.0:1789760767.521535:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760767.521543:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2f48a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26869760->26873855] (req 26869760->26873855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.521556:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2f48a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26869760->26873855] (req 26869760->26873855) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760767.554612:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973879 is committed 00002000:00010000:0.0:1789760767.554735:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26869760 -> 26873856 00002000:00010000:0.0:1789760767.554740:0:6710:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760766 -> 1789760767 00002000:00010000:0.0:1789760767.554743:0:6710:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760766 -> 1789760767 00010000:00010000:0.0:1789760767.554757:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2f48a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26869760->26873855] (req 26869760->26873855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.554771:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2f48a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26869760->26873855] (req 26869760->26873855) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.554781:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2f48a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26869760->26873855] (req 26869760->26873855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.554789:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760767.554795:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2f48a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26869760->26873855] (req 26869760->26873855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.554804:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2f48a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26869760->26873855] (req 26869760->26873855) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.554835:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2f48a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26869760->26873855] (req 26869760->26873855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.554866:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2f48a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26869760->26873855] (req 26869760->26873855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760767.554958:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac531c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512702208:12345-192.168.203.56@tcp:4:dd.0 Request processed in 33494us (33715us total) trans 4294973879 rc 0/0 00000100:00100000:0.0:1789760767.554968:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6590 00000100:00100000:0.0:1789760767.554978:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 747008063946 time 747085749405 state 0x2 ref 0 00000100:00100000:2.0:1789760767.561534:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760767.561684:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512702336 00000100:00100000:1.0:1789760767.561954:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512702336 00000100:00100000:2.0:1789760767.562076:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 747008063946 time 747085749405 state 0x0 ref 1 00000100:00100000:2.0:1789760767.562091:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 747086009821 time 747085749405 state 0x1 ref 1 00000100:00100000:2.0:1789760767.562167:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 747086009821 time 747126466960 state 0x0 ref 1 00000100:00100000:2.0:1789760767.562180:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac531d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512702336:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760767.562224:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26873856, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0xd007 00010000:00010000:2.0:1789760767.562233:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2f491 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.562248:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=747s v=(0 0) 00010000:00010000:2.0:1789760767.562253:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760767.562259:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2f491 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26873856->26877951] (req 26873856->26877951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.562274:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2f491 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26873856->26877951] (req 26873856->26877951) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760767.596731:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26873856 -> 26877952 00010000:00010000:3.0:1789760767.596750:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2f491 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26873856->26877951] (req 26873856->26877951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.596772:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2f491 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26873856->26877951] (req 26873856->26877951) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.596780:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2f491 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26873856->26877951] (req 26873856->26877951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.596787:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760767.596793:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2f491 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26873856->26877951] (req 26873856->26877951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.596802:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2f491 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26873856->26877951] (req 26873856->26877951) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.596829:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2f491 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26873856->26877951] (req 26873856->26877951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.596837:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2f491 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26873856->26877951] (req 26873856->26877951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760767.596919:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac531d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2360:x1876699512702336:12345-192.168.203.56@tcp:4:dd.0 Request processed in 34736us (35388us total) trans 4294973880 rc 0/0 00000100:00100000:3.0:1789760767.596929:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6591 00000100:00100000:3.0:1789760767.596939:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 747086009821 time 747126466960 state 0x2 ref 0 00000001:00080000:1.0:1789760767.597068:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973880 is committed 00000100:00100000:1.0:1789760767.600586:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760767.600690:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512702464 00000100:00100000:2.0:1789760767.600911:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512702464 00000100:00100000:2.0:1789760767.600970:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 747086009821 time 747126466960 state 0x0 ref 1 00000100:00100000:2.0:1789760767.600986:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 747126727376 time 747126466960 state 0x1 ref 1 00000100:00100000:2.0:1789760767.601014:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 747126727376 time 747165314106 state 0x0 ref 1 00000100:00100000:2.0:1789760767.601026:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4039c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512702464:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760767.601075:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26877952, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0xd007 00010000:00010000:2.0:1789760767.601084:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2f498 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.601100:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=747s v=(0 0) 00010000:00010000:2.0:1789760767.601104:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760767.601111:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2f498 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26877952->26882047] (req 26877952->26882047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.601125:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2f498 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26877952->26882047] (req 26877952->26882047) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760767.627157:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973881 is committed 00002000:00010000:1.0:1789760767.627319:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26877952 -> 26882048 00010000:00010000:1.0:1789760767.627335:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2f498 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26877952->26882047] (req 26877952->26882047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.627348:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2f498 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26877952->26882047] (req 26877952->26882047) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.627358:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2f498 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26877952->26882047] (req 26877952->26882047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.627381:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760767.627387:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2f498 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26877952->26882047] (req 26877952->26882047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.627397:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2f498 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26877952->26882047] (req 26877952->26882047) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.627420:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2f498 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26877952->26882047] (req 26877952->26882047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.627429:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2f498 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26877952->26882047] (req 26877952->26882047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760767.627527:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4039c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512702464:12345-192.168.203.56@tcp:4:dd.0 Request processed in 26498us (26948us total) trans 4294973881 rc 0/0 00000100:00100000:1.0:1789760767.627537:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6592 00000100:00100000:1.0:1789760767.627548:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 747126727376 time 747165314106 state 0x2 ref 0 00000100:00100000:2.0:1789760767.630683:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760767.630812:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512702592 00000100:00100000:0.0:1789760767.630823:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512702592 00000100:00100000:0.0:1789760767.630881:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 747126727376 time 747165314106 state 0x0 ref 1 00000100:00100000:0.0:1789760767.630898:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 747165574522 time 747165314106 state 0x1 ref 1 00000100:00100000:0.0:1789760767.630982:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 747165574522 time 747195279570 state 0x0 ref 1 00000100:00100000:0.0:1789760767.630996:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac531e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512702592:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760767.631050:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26882048, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0xd007 00010000:00010000:0.0:1789760767.631059:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2f49f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.631076:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=747s v=(0 0) 00010000:00010000:0.0:1789760767.631081:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760767.631088:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2f49f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26882048->26886143] (req 26882048->26886143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.631107:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2f49f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26882048->26886143] (req 26882048->26886143) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760767.664840:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973882 is committed 00002000:00010000:1.0:1789760767.665098:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26882048 -> 26886144 00010000:00010000:1.0:1789760767.665113:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2f49f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26882048->26886143] (req 26882048->26886143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.665126:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2f49f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26882048->26886143] (req 26882048->26886143) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.665154:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2f49f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26882048->26886143] (req 26882048->26886143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.665162:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760767.665168:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2f49f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26882048->26886143] (req 26882048->26886143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.665178:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2f49f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26882048->26886143] (req 26882048->26886143) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.665198:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2f49f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26882048->26886143] (req 26882048->26886143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.665206:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2f49f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26882048->26886143] (req 26882048->26886143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760767.665301:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac531e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512702592:12345-192.168.203.56@tcp:4:dd.0 Request processed in 34304us (34622us total) trans 4294973882 rc 0/0 00000100:00100000:1.0:1789760767.665311:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6593 00000100:00100000:1.0:1789760767.665320:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 747165574522 time 747195279570 state 0x2 ref 0 00000100:00100000:2.0:1789760767.668986:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760767.669078:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512702720 00000100:00100000:3.0:1789760767.669088:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512702720 00000100:00100000:3.0:1789760767.669151:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 747165574522 time 747195279570 state 0x0 ref 1 00000100:00100000:3.0:1789760767.669167:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 747195539986 time 747195279570 state 0x1 ref 1 00000100:00100000:3.0:1789760767.669193:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 747195539986 time 747233492936 state 0x0 ref 1 00000100:00100000:3.0:1789760767.669204:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac531ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512702720:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760767.669251:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26886144, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0xd007 00010000:00010000:3.0:1789760767.669259:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2f4a6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.669275:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=747s v=(0 0) 00010000:00010000:3.0:1789760767.669280:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760767.669287:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2f4a6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26886144->26890239] (req 26886144->26890239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.669301:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2f4a6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26886144->26890239] (req 26886144->26890239) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760767.694130:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973883 is committed 00002000:00010000:2.0:1789760767.694245:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26886144 -> 26890240 00010000:00010000:2.0:1789760767.694262:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2f4a6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26886144->26890239] (req 26886144->26890239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.694274:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2f4a6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26886144->26890239] (req 26886144->26890239) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.694285:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2f4a6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26886144->26890239] (req 26886144->26890239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.694292:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760767.694296:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2f4a6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26886144->26890239] (req 26886144->26890239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.694305:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2f4a6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26886144->26890239] (req 26886144->26890239) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.694323:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2f4a6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26886144->26890239] (req 26886144->26890239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.694330:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2f4a6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26886144->26890239] (req 26886144->26890239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760767.694428:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac531ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512702720:12345-192.168.203.56@tcp:4:dd.0 Request processed in 25220us (25446us total) trans 4294973883 rc 0/0 00000100:00100000:2.0:1789760767.694441:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6594 00000100:00100000:2.0:1789760767.694451:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 747195539986 time 747233492936 state 0x2 ref 0 00000100:00100000:0.0:1789760767.698300:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760767.698436:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512702848 00000100:00100000:1.0:1789760767.698446:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512702848 00000100:00100000:1.0:1789760767.698500:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 747195539986 time 747233492936 state 0x0 ref 1 00000100:00100000:1.0:1789760767.698515:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 747233753352 time 747233492936 state 0x1 ref 1 00000100:00100000:1.0:1789760767.698546:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 747233753352 time 747262845324 state 0x0 ref 1 00000100:00100000:1.0:1789760767.698558:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4059880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512702848:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760767.698604:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26890240, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0xd007 00010000:00010000:1.0:1789760767.698612:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2f4ad lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.698628:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=747s v=(0 0) 00010000:00010000:1.0:1789760767.698632:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760767.698638:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2f4ad lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26890240->26894335] (req 26890240->26894335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.698652:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2f4ad lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26890240->26894335] (req 26890240->26894335) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760767.738443:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973884 is committed 00002000:00010000:3.0:1789760767.738594:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26890240 -> 26894336 00010000:00010000:3.0:1789760767.738611:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2f4ad lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26890240->26894335] (req 26890240->26894335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.738624:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2f4ad lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26890240->26894335] (req 26890240->26894335) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.738633:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2f4ad lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26890240->26894335] (req 26890240->26894335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.738640:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760767.738644:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2f4ad lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26890240->26894335] (req 26890240->26894335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.738653:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2f4ad lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26890240->26894335] (req 26890240->26894335) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.738671:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2f4ad lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26890240->26894335] (req 26890240->26894335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.738679:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2f4ad lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26890240->26894335] (req 26890240->26894335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760767.739088:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4059880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512702848:12345-192.168.203.56@tcp:4:dd.0 Request processed in 40525us (40790us total) trans 4294973884 rc 0/0 00000100:00100000:2.0:1789760767.739099:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6595 00000100:00100000:2.0:1789760767.739109:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 747233753352 time 747262845324 state 0x2 ref 0 00000100:00100000:2.0:1789760767.743023:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760767.743169:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512702976 00000100:00100000:1.0:1789760767.743180:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512702976 00000100:00100000:1.0:1789760767.743244:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 747233753352 time 747262845324 state 0x0 ref 1 00000100:00100000:1.0:1789760767.743259:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 747263105740 time 747262845324 state 0x1 ref 1 00000100:00100000:1.0:1789760767.743294:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 747263105740 time 747307593068 state 0x0 ref 1 00000100:00100000:1.0:1789760767.743308:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac531dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512702976:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760767.743364:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26894336, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0xd007 00010000:00010000:1.0:1789760767.743385:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2f4b4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.743402:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=747s v=(0 0) 00010000:00010000:1.0:1789760767.743407:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760767.743414:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2f4b4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26894336->26898431] (req 26894336->26898431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.743428:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2f4b4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26894336->26898431] (req 26894336->26898431) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760767.791000:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973885 is committed 00002000:00010000:1.0:1789760767.791201:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26894336 -> 26898432 00010000:00010000:1.0:1789760767.791219:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2f4b4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26894336->26898431] (req 26894336->26898431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.791232:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2f4b4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26894336->26898431] (req 26894336->26898431) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.791240:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2f4b4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26894336->26898431] (req 26894336->26898431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.791248:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760767.791253:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2f4b4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26894336->26898431] (req 26894336->26898431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.791261:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2f4b4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26894336->26898431] (req 26894336->26898431) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.791286:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2f4b4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26894336->26898431] (req 26894336->26898431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.791294:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2f4b4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26894336->26898431] (req 26894336->26898431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760767.791426:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac531dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512702976:12345-192.168.203.56@tcp:4:dd.0 Request processed in 48115us (48408us total) trans 4294973885 rc 0/0 00000100:00100000:1.0:1789760767.791437:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6596 00000100:00100000:1.0:1789760767.791447:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 747263105740 time 747307593068 state 0x2 ref 0 00000100:00100000:3.0:1789760767.794337:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760767.795673:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512703104 00000100:00100000:2.0:1789760767.795687:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512703104 00000100:00100000:2.0:1789760767.796102:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 747263105740 time 747307593068 state 0x0 ref 1 00000100:00100000:2.0:1789760767.796191:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 747307853484 time 747307593068 state 0x1 ref 1 00000100:00100000:2.0:1789760767.796263:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 747307853484 time 747360561425 state 0x0 ref 1 00000100:00100000:2.0:1789760767.796279:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5305500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512703104:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760767.796328:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26898432, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0xd007 00010000:00010000:2.0:1789760767.796336:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2f4bb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.796352:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=747s v=(0 0) 00010000:00010000:2.0:1789760767.796356:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760767.796361:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2f4bb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26898432->26902527] (req 26898432->26902527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.796392:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2f4bb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26898432->26902527] (req 26898432->26902527) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760767.826729:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973886 is committed 00002000:00010000:1.0:1789760767.827370:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26898432 -> 26902528 00010000:00010000:1.0:1789760767.827388:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2f4bb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26898432->26902527] (req 26898432->26902527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.827400:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2f4bb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26898432->26902527] (req 26898432->26902527) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.827408:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2f4bb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26898432->26902527] (req 26898432->26902527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.827416:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760767.827421:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2f4bb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26898432->26902527] (req 26898432->26902527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.827434:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2f4bb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26898432->26902527] (req 26898432->26902527) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.827454:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2f4bb lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26898432->26902527] (req 26898432->26902527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.827461:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2f4bb lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26898432->26902527] (req 26898432->26902527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760767.827593:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5305500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512703104:12345-192.168.203.56@tcp:4:dd.0 Request processed in 31315us (33261us total) trans 4294973886 rc 0/0 00000100:00100000:1.0:1789760767.827603:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6597 00000100:00100000:1.0:1789760767.827612:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 747307853484 time 747360561425 state 0x2 ref 0 00000100:00100000:0.0:1789760767.836752:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760767.836824:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512703232 00000100:00100000:2.0:1789760767.836835:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512703232 00000100:00100000:2.0:1789760767.836909:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 747307853484 time 747360561425 state 0x0 ref 1 00000100:00100000:2.0:1789760767.836934:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 747360821841 time 747360561425 state 0x1 ref 1 00000100:00100000:2.0:1789760767.836977:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 747360821841 time 747401277112 state 0x0 ref 1 00000100:00100000:2.0:1789760767.836990:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac405bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512703232:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760767.837182:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26902528, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0xd007 00010000:00010000:2.0:1789760767.837192:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2f4c2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.837209:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=747s v=(0 0) 00010000:00010000:2.0:1789760767.837214:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760767.837221:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2f4c2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26902528->26906623] (req 26902528->26906623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.837237:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2f4c2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26902528->26906623] (req 26902528->26906623) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760767.878155:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26902528 -> 26906624 00010000:00010000:1.0:1789760767.878173:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2f4c2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26902528->26906623] (req 26902528->26906623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.878187:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2f4c2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26902528->26906623] (req 26902528->26906623) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.878196:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2f4c2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26902528->26906623] (req 26902528->26906623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.878203:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760767.878208:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2f4c2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26902528->26906623] (req 26902528->26906623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.878217:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2f4c2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26902528->26906623] (req 26902528->26906623) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.878235:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2f4c2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26902528->26906623] (req 26902528->26906623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.878241:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2f4c2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26902528->26906623] (req 26902528->26906623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760767.878333:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac405bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2361:x1876699512703232:12345-192.168.203.56@tcp:4:dd.0 Request processed in 41339us (41585us total) trans 4294973887 rc 0/0 00000100:00100000:1.0:1789760767.878343:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6598 00000100:00100000:1.0:1789760767.878353:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 747360821841 time 747401277112 state 0x2 ref 0 00000001:00080000:0.0:1789760767.878431:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973887 is committed 00000100:00100000:2.0:1789760767.886262:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760767.887191:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512703360 00000100:00100000:1.0:1789760767.887202:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512703360 00000100:00100000:1.0:1789760767.887256:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 747360821841 time 747401277112 state 0x0 ref 1 00000100:00100000:1.0:1789760767.887272:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 747401537528 time 747401277112 state 0x1 ref 1 00000100:00100000:1.0:1789760767.887306:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 747401537528 time 747451605405 state 0x0 ref 1 00000100:00100000:1.0:1789760767.887318:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac531d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512703360:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760767.887368:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26906624, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0xd007 00010000:00010000:1.0:1789760767.887378:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2f4c9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.887395:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=747s v=(0 0) 00010000:00010000:1.0:1789760767.887399:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760767.887406:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2f4c9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26906624->26910719] (req 26906624->26910719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.887420:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2f4c9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26906624->26910719] (req 26906624->26910719) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760767.913725:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973888 is committed 00002000:00010000:1.0:1789760767.913955:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26906624 -> 26910720 00010000:00010000:1.0:1789760767.913970:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2f4c9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26906624->26910719] (req 26906624->26910719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.913983:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2f4c9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26906624->26910719] (req 26906624->26910719) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.913993:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2f4c9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26906624->26910719] (req 26906624->26910719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.914001:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760767.914005:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2f4c9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26906624->26910719] (req 26906624->26910719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.914014:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2f4c9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26906624->26910719] (req 26906624->26910719) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.914042:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2f4c9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26906624->26910719] (req 26906624->26910719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.914050:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2f4c9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26906624->26910719] (req 26906624->26910719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760767.914255:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac531d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512703360:12345-192.168.203.56@tcp:4:dd.0 Request processed in 26934us (27999us total) trans 4294973888 rc 0/0 00000100:00100000:1.0:1789760767.914268:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6599 00000100:00100000:1.0:1789760767.914278:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 747401537528 time 747451605405 state 0x2 ref 0 00000100:00100000:2.0:1789760767.917121:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760767.917233:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512703488 00000100:00100000:1.0:1789760767.917242:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512703488 00000100:00100000:1.0:1789760767.917303:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 747401537528 time 747451605405 state 0x0 ref 1 00000100:00100000:1.0:1789760767.917317:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 747451865821 time 747451605405 state 0x1 ref 1 00000100:00100000:1.0:1789760767.917348:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 747451865821 time 747481647450 state 0x0 ref 1 00000100:00100000:1.0:1789760767.917361:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac3351180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512703488:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760767.917408:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26910720, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0xd007 00010000:00010000:1.0:1789760767.917418:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2f4d0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.917433:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=747s v=(0 0) 00010000:00010000:1.0:1789760767.917438:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760767.917445:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2f4d0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26910720->26914815] (req 26910720->26914815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.917458:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2f4d0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26910720->26914815] (req 26910720->26914815) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760767.961617:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973889 is committed 00002000:00010000:2.0:1789760767.961901:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26910720 -> 26914816 00010000:00010000:2.0:1789760767.961918:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2f4d0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26910720->26914815] (req 26910720->26914815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.961930:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2f4d0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26910720->26914815] (req 26910720->26914815) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.961938:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2f4d0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26910720->26914815] (req 26910720->26914815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.961945:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760767.961950:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2f4d0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26910720->26914815] (req 26910720->26914815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.961959:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2f4d0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26910720->26914815] (req 26910720->26914815) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.961979:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2f4d0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26910720->26914815] (req 26910720->26914815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.961986:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2f4d0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26910720->26914815] (req 26910720->26914815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760767.962118:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac3351180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512703488:12345-192.168.203.56@tcp:4:dd.0 Request processed in 44755us (45002us total) trans 4294973889 rc 0/0 00000100:00100000:2.0:1789760767.962128:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6600 00000100:00100000:2.0:1789760767.962147:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 747451865821 time 747481647450 state 0x2 ref 0 00000100:00100000:0.0:1789760767.966592:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760767.966786:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512703616 00000100:00100000:1.0:1789760767.966796:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512703616 00000100:00100000:1.0:1789760767.966853:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 747451865821 time 747481647450 state 0x0 ref 1 00000100:00100000:1.0:1789760767.966868:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 747481907866 time 747481647450 state 0x1 ref 1 00000100:00100000:1.0:1789760767.966895:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 747481907866 time 747531194636 state 0x0 ref 1 00000100:00100000:1.0:1789760767.966908:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4058380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512703616:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760767.966956:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26914816, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0xd007 00010000:00010000:1.0:1789760767.966965:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2f4d7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.966980:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=747s v=(0 0) 00010000:00010000:1.0:1789760767.966984:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760767.966990:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2f4d7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26914816->26918911] (req 26914816->26918911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.967002:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2f4d7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26914816->26918911] (req 26914816->26918911) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760767.997146:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973890 is committed 00002000:00010000:2.0:1789760767.997414:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26914816 -> 26918912 00010000:00010000:2.0:1789760767.997430:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2f4d7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26914816->26918911] (req 26914816->26918911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.997442:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2f4d7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26914816->26918911] (req 26914816->26918911) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.997453:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2f4d7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26914816->26918911] (req 26914816->26918911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.997461:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760767.997466:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2f4d7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26914816->26918911] (req 26914816->26918911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.997476:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2f4d7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26914816->26918911] (req 26914816->26918911) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.997496:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2f4d7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26914816->26918911] (req 26914816->26918911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.997510:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2f4d7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26914816->26918911] (req 26914816->26918911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760767.997970:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4058380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512703616:12345-192.168.203.56@tcp:4:dd.0 Request processed in 31059us (31383us total) trans 4294973890 rc 0/0 00000100:00100000:2.0:1789760767.997982:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6601 00000100:00100000:2.0:1789760767.997993:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 747481907866 time 747531194636 state 0x2 ref 0 00000100:00100000:3.0:1789760768.001785:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760768.001938:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512703744 00000100:00100000:1.0:1789760768.001948:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512703744 00000100:00100000:1.0:1789760768.002002:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 747481907866 time 747531194636 state 0x0 ref 1 00000100:00100000:1.0:1789760768.002017:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 747531455052 time 747531194636 state 0x1 ref 1 00000100:00100000:1.0:1789760768.002078:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 747531455052 time 747566377207 state 0x0 ref 1 00000100:00100000:1.0:1789760768.002090:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5307b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512703744:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760768.002149:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26918912, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0xd007 00010000:00010000:1.0:1789760768.002159:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2f4de lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.002175:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=747s v=(0 0) 00010000:00010000:1.0:1789760768.002179:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760768.002186:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2f4de lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26918912->26923007] (req 26918912->26923007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.002200:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2f4de lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26918912->26923007] (req 26918912->26923007) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760768.067398:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973891 is committed 00002000:00010000:1.0:1789760768.067518:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26918912 -> 26923008 00010000:00010000:1.0:1789760768.067533:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2f4de lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26918912->26923007] (req 26918912->26923007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.067553:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2f4de lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26918912->26923007] (req 26918912->26923007) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.067563:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2f4de lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26918912->26923007] (req 26918912->26923007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.067570:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760768.067575:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2f4de lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26918912->26923007] (req 26918912->26923007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.067584:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2f4de lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26918912->26923007] (req 26918912->26923007) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.067601:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2f4de lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26918912->26923007] (req 26918912->26923007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.067609:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2f4de lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26918912->26923007] (req 26918912->26923007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760768.067736:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5307b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512703744:12345-192.168.203.56@tcp:4:dd.0 Request processed in 65606us (65919us total) trans 4294973891 rc 0/0 00000100:00100000:1.0:1789760768.067746:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6602 00000100:00100000:1.0:1789760768.067756:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 747531455052 time 747566377207 state 0x2 ref 0 00000100:00100000:3.0:1789760768.071619:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760768.071764:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512703872 00000100:00100000:2.0:1789760768.071774:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512703872 00000100:00100000:2.0:1789760768.071826:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 747531455052 time 747566377207 state 0x0 ref 1 00000100:00100000:2.0:1789760768.071841:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 747566637623 time 747566377207 state 0x1 ref 1 00000100:00100000:2.0:1789760768.071861:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 747566637623 time 747636161664 state 0x0 ref 1 00000100:00100000:2.0:1789760768.071874:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5305f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512703872:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760768.071923:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26923008, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0xd007 00010000:00010000:2.0:1789760768.071932:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2f4e5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.071948:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=747s v=(0 0) 00010000:00010000:2.0:1789760768.071952:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760768.071958:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2f4e5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26923008->26927103] (req 26923008->26927103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.071972:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2f4e5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26923008->26927103] (req 26923008->26927103) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760768.116552:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26923008 -> 26927104 00010000:00010000:1.0:1789760768.116572:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2f4e5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26923008->26927103] (req 26923008->26927103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.116585:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2f4e5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26923008->26927103] (req 26923008->26927103) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.116594:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2f4e5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26923008->26927103] (req 26923008->26927103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.116601:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760768.116606:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2f4e5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26923008->26927103] (req 26923008->26927103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.116616:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2f4e5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26923008->26927103] (req 26923008->26927103) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.116633:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2f4e5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26923008->26927103] (req 26923008->26927103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.116641:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2f4e5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26923008->26927103] (req 26923008->26927103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760768.116757:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5305f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2361:x1876699512703872:12345-192.168.203.56@tcp:4:dd.0 Request processed in 44880us (45145us total) trans 4294973892 rc 0/0 00000100:00100000:1.0:1789760768.116767:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6603 00000100:00100000:1.0:1789760768.116777:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 747566637623 time 747636161664 state 0x2 ref 0 00000001:00080000:0.0:1789760768.118827:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973892 is committed 00000100:00100000:0.0:1789760768.119358:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760768.119450:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512704000 00000100:00100000:2.0:1789760768.119459:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512704000 00000100:00100000:2.0:1789760768.119510:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 747566637623 time 747636161664 state 0x0 ref 1 00000100:00100000:2.0:1789760768.119562:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 747636422080 time 747636161664 state 0x1 ref 1 00000100:00100000:2.0:1789760768.119595:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 747636422080 time 747683894480 state 0x0 ref 1 00000100:00100000:2.0:1789760768.119607:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4058a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512704000:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760768.119651:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26927104, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0xd007 00010000:00010000:2.0:1789760768.119658:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2f4ec lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.119671:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=747s v=(0 0) 00010000:00010000:2.0:1789760768.119674:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760768.119678:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2f4ec lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26927104->26931199] (req 26927104->26931199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.119688:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2f4ec lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26927104->26931199] (req 26927104->26931199) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760768.149782:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973893 is committed 00002000:00010000:2.0:1789760768.150205:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26927104 -> 26931200 00010000:00010000:2.0:1789760768.150222:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2f4ec lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26927104->26931199] (req 26927104->26931199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.150235:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2f4ec lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26927104->26931199] (req 26927104->26931199) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.150317:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2f4ec lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26927104->26931199] (req 26927104->26931199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.150325:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760768.150330:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2f4ec lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26927104->26931199] (req 26927104->26931199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.150339:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2f4ec lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26927104->26931199] (req 26927104->26931199) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.150359:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2f4ec lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26927104->26931199] (req 26927104->26931199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.150367:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2f4ec lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26927104->26931199] (req 26927104->26931199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760768.150457:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4058a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512704000:12345-192.168.203.56@tcp:4:dd.0 Request processed in 30847us (31099us total) trans 4294973893 rc 0/0 00000100:00100000:2.0:1789760768.150467:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6604 00000100:00100000:2.0:1789760768.150477:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 747636422080 time 747683894480 state 0x2 ref 0 00000100:00100000:1.0:1789760768.156644:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760768.157111:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512704128 00000100:00100000:2.0:1789760768.157122:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512704128 00000100:00100000:2.0:1789760768.157187:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 747636422080 time 747683894480 state 0x0 ref 1 00000100:00100000:2.0:1789760768.157202:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 747684154896 time 747683894480 state 0x1 ref 1 00000100:00100000:2.0:1789760768.157236:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 747684154896 time 747721535437 state 0x0 ref 1 00000100:00100000:2.0:1789760768.157322:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4038e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512704128:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760768.157373:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26931200, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0xd007 00010000:00010000:2.0:1789760768.157383:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2f4f3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.157399:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=747s v=(0 0) 00010000:00010000:2.0:1789760768.157403:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760768.157410:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2f4f3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26931200->26935295] (req 26931200->26935295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.157422:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2f4f3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26931200->26935295] (req 26931200->26935295) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760768.203195:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973894 is committed 00002000:00010000:2.0:1789760768.203361:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26931200 -> 26935296 00010000:00010000:2.0:1789760768.203378:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2f4f3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26931200->26935295] (req 26931200->26935295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.203391:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2f4f3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26931200->26935295] (req 26931200->26935295) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.203400:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2f4f3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26931200->26935295] (req 26931200->26935295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.203407:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760768.203412:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2f4f3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26931200->26935295] (req 26931200->26935295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.203421:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2f4f3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26931200->26935295] (req 26931200->26935295) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.203441:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2f4f3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26931200->26935295] (req 26931200->26935295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.203449:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2f4f3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26931200->26935295] (req 26931200->26935295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760768.203550:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4038e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512704128:12345-192.168.203.56@tcp:4:dd.0 Request processed in 46227us (46910us total) trans 4294973894 rc 0/0 00000100:00100000:2.0:1789760768.203566:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6605 00000100:00100000:2.0:1789760768.203576:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 747684154896 time 747721535437 state 0x2 ref 0 00000100:00100000:3.0:1789760768.215585:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760768.215765:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512704256 00000100:00100000:1.0:1789760768.215775:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512704256 00000100:00100000:1.0:1789760768.215831:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 747684154896 time 747721535437 state 0x0 ref 1 00000100:00100000:1.0:1789760768.215848:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 747721795853 time 747721535437 state 0x1 ref 1 00000100:00100000:1.0:1789760768.215885:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 747721795853 time 747780184087 state 0x0 ref 1 00000100:00100000:1.0:1789760768.215898:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5307480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512704256:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760768.215941:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26935296, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0xd007 00010000:00010000:1.0:1789760768.215950:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2f4fa lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.215966:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=747s v=(0 0) 00010000:00010000:1.0:1789760768.215970:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760768.215976:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2f4fa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26935296->26939391] (req 26935296->26939391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.215990:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2f4fa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26935296->26939391] (req 26935296->26939391) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760768.268173:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973895 is committed 00002000:00010000:1.0:1789760768.268281:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26935296 -> 26939392 00010000:00010000:1.0:1789760768.268297:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2f4fa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26935296->26939391] (req 26935296->26939391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.268311:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2f4fa lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26935296->26939391] (req 26935296->26939391) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.268320:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2f4fa lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26935296->26939391] (req 26935296->26939391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.268337:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760768.268342:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2f4fa lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26935296->26939391] (req 26935296->26939391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.268351:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2f4fa lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26935296->26939391] (req 26935296->26939391) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.268371:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2f4fa lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26935296->26939391] (req 26935296->26939391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.268379:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2f4fa lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26935296->26939391] (req 26935296->26939391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760768.268474:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5307480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512704256:12345-192.168.203.56@tcp:4:dd.0 Request processed in 52573us (52892us total) trans 4294973895 rc 0/0 00000100:00100000:1.0:1789760768.268484:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6606 00000100:00100000:1.0:1789760768.268494:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 747721795853 time 747780184087 state 0x2 ref 0 00000100:00100000:0.0:1789760768.271990:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760768.272225:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512704384 00000100:00100000:1.0:1789760768.272236:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512704384 00000100:00100000:1.0:1789760768.272290:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 747721795853 time 747780184087 state 0x0 ref 1 00000100:00100000:1.0:1789760768.272305:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 747780444503 time 747780184087 state 0x1 ref 1 00000100:00100000:1.0:1789760768.272420:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 747780444503 time 747836718488 state 0x0 ref 1 00000100:00100000:1.0:1789760768.272434:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4059500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512704384:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760768.272483:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26939392, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0xd007 00010000:00010000:1.0:1789760768.272491:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2f501 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.272506:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=747s v=(0 0) 00010000:00010000:1.0:1789760768.272509:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760768.272515:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2f501 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26939392->26943487] (req 26939392->26943487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.272540:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2f501 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26939392->26943487] (req 26939392->26943487) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760768.312279:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973896 is committed 00002000:00010000:1.0:1789760768.312625:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26939392 -> 26943488 00010000:00010000:1.0:1789760768.312642:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2f501 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26939392->26943487] (req 26939392->26943487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.312655:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2f501 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26939392->26943487] (req 26939392->26943487) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.312670:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2f501 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26939392->26943487] (req 26939392->26943487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.312677:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760768.312682:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2f501 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26939392->26943487] (req 26939392->26943487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.312691:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2f501 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26939392->26943487] (req 26939392->26943487) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.312744:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2f501 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26939392->26943487] (req 26939392->26943487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.312752:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2f501 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26939392->26943487] (req 26939392->26943487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760768.312835:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4059500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512704384:12345-192.168.203.56@tcp:4:dd.0 Request processed in 40398us (40848us total) trans 4294973896 rc 0/0 00000100:00100000:1.0:1789760768.312845:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6607 00000100:00100000:1.0:1789760768.312855:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 747780444503 time 747836718488 state 0x2 ref 0 00000100:00100000:0.0:1789760768.316676:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760768.316894:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512704512 00000100:00100000:2.0:1789760768.316903:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512704512 00000100:00100000:2.0:1789760768.316955:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 747780444503 time 747836718488 state 0x0 ref 1 00000100:00100000:2.0:1789760768.316971:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 747836978904 time 747836718488 state 0x1 ref 1 00000100:00100000:2.0:1789760768.317044:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 747836978904 time 747881342034 state 0x0 ref 1 00000100:00100000:2.0:1789760768.317058:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac405ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512704512:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760768.317109:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26943488, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0xd007 00010000:00010000:2.0:1789760768.317117:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2f508 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.317168:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=747s v=(0 0) 00010000:00010000:2.0:1789760768.317173:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760768.317179:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2f508 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26943488->26947583] (req 26943488->26947583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.317194:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2f508 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26943488->26947583] (req 26943488->26947583) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760768.360345:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26943488 -> 26947584 00010000:00010000:2.0:1789760768.360366:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2f508 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26943488->26947583] (req 26943488->26947583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.360379:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2f508 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26943488->26947583] (req 26943488->26947583) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.360389:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2f508 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26943488->26947583] (req 26943488->26947583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.360397:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760768.360403:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2f508 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26943488->26947583] (req 26943488->26947583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.360413:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2f508 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26943488->26947583] (req 26943488->26947583) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.360432:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2f508 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26943488->26947583] (req 26943488->26947583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.360440:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2f508 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26943488->26947583] (req 26943488->26947583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760768.360579:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac405ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2360:x1876699512704512:12345-192.168.203.56@tcp:4:dd.0 Request processed in 43518us (43908us total) trans 4294973897 rc 0/0 00000100:00100000:2.0:1789760768.360590:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6608 00000100:00100000:2.0:1789760768.360600:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 747836978904 time 747881342034 state 0x2 ref 0 00000001:00080000:2.0:1789760768.360641:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973897 is committed 00010000:00010000:2.0:1789760768.371342:0:7647:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789760768.375316:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760768.375582:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512704640 00000100:00100000:1.0:1789760768.375592:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512704640 00000100:00100000:1.0:1789760768.375645:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 747836978904 time 747881342034 state 0x0 ref 1 00000100:00100000:1.0:1789760768.375659:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 747881602450 time 747881342034 state 0x1 ref 1 00000100:00100000:1.0:1789760768.375692:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 747881602450 time 747939990996 state 0x0 ref 1 00000100:00100000:2.0:1789760768.377503:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5304700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512704640:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760768.377568:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26947584, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0xd007 00010000:00010000:2.0:1789760768.377580:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2f50f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.377597:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=747s v=(0 0) 00010000:00010000:2.0:1789760768.377603:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760768.377610:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2f50f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26947584->26951679] (req 26947584->26951679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.377625:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2f50f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26947584->26951679] (req 26947584->26951679) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760768.443317:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973898 is committed 00002000:00010000:0.0:1789760768.443430:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26947584 -> 26951680 00010000:00010000:0.0:1789760768.443447:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2f50f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26947584->26951679] (req 26947584->26951679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.443460:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2f50f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26947584->26951679] (req 26947584->26951679) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.443469:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2f50f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26947584->26951679] (req 26947584->26951679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.443482:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760768.443486:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2f50f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26947584->26951679] (req 26947584->26951679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.443495:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2f50f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26947584->26951679] (req 26947584->26951679) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.443516:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2f50f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26947584->26951679] (req 26947584->26951679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.443524:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2f50f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26947584->26951679] (req 26947584->26951679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760768.443621:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5304700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512704640:12345-192.168.203.56@tcp:4:dd.0 Request processed in 67804us (68312us total) trans 4294973898 rc 0/0 00000100:00100000:0.0:1789760768.443632:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6609 00000100:00100000:0.0:1789760768.443642:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 747881602450 time 747939990996 state 0x2 ref 0 00000100:00100000:2.0:1789760768.449710:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760768.450013:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512704768 00000100:00100000:1.0:1789760768.450024:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512704768 00000100:00100000:1.0:1789760768.450078:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 747881602450 time 747939990996 state 0x0 ref 1 00000100:00100000:1.0:1789760768.450093:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 747940251412 time 747939990996 state 0x1 ref 1 00000100:00100000:1.0:1789760768.450126:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 747940251412 time 748014425600 state 0x0 ref 1 00000100:00100000:1.0:1789760768.450150:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac3352a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512704768:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760768.450198:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26951680, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0xd007 00010000:00010000:1.0:1789760768.450207:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2f516 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.450223:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=748s v=(0 0) 00010000:00010000:1.0:1789760768.450228:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760768.450235:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2f516 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26951680->26955775] (req 26951680->26955775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.450250:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2f516 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26951680->26955775] (req 26951680->26955775) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760768.493365:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973899 is committed 00002000:00010000:2.0:1789760768.493432:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26951680 -> 26955776 00010000:00010000:2.0:1789760768.493447:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2f516 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26951680->26955775] (req 26951680->26955775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.493460:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2f516 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26951680->26955775] (req 26951680->26955775) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.493469:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2f516 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26951680->26955775] (req 26951680->26955775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.493477:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760768.493481:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2f516 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26951680->26955775] (req 26951680->26955775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.493491:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2f516 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26951680->26955775] (req 26951680->26955775) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.493510:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2f516 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26951680->26955775] (req 26951680->26955775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.493523:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2f516 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26951680->26955775] (req 26951680->26955775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760768.493616:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac3352a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512704768:12345-192.168.203.56@tcp:4:dd.0 Request processed in 43466us (43925us total) trans 4294973899 rc 0/0 00000100:00100000:2.0:1789760768.493629:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6610 00000100:00100000:2.0:1789760768.493639:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 747940251412 time 748014425600 state 0x2 ref 0 00000100:00100000:1.0:1789760768.501046:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760768.501155:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512704896 00000100:00100000:2.0:1789760768.501165:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512704896 00000100:00100000:2.0:1789760768.501229:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 747940251412 time 748014425600 state 0x0 ref 1 00000100:00100000:2.0:1789760768.501245:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 748014686016 time 748014425600 state 0x1 ref 1 00000100:00100000:2.0:1789760768.501278:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 748014686016 time 748065577474 state 0x0 ref 1 00000100:00100000:2.0:1789760768.501291:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac403b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512704896:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760768.501348:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26955776, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0xd007 00010000:00010000:2.0:1789760768.501358:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2f51d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.501375:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=748s v=(0 0) 00010000:00010000:2.0:1789760768.501379:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760768.501386:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2f51d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26955776->26959871] (req 26955776->26959871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.501404:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2f51d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26955776->26959871] (req 26955776->26959871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760768.539719:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973900 is committed 00002000:00010000:2.0:1789760768.539790:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26955776 -> 26959872 00002000:00010000:2.0:1789760768.539795:0:6710:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760767 -> 1789760768 00002000:00010000:2.0:1789760768.539798:0:6710:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760767 -> 1789760768 00010000:00010000:2.0:1789760768.539820:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2f51d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26955776->26959871] (req 26955776->26959871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.539841:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2f51d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26955776->26959871] (req 26955776->26959871) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.539851:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2f51d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26955776->26959871] (req 26955776->26959871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.539858:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760768.539863:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2f51d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26955776->26959871] (req 26955776->26959871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.539873:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2f51d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26955776->26959871] (req 26955776->26959871) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.539892:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2f51d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26955776->26959871] (req 26955776->26959871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.539910:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2f51d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26955776->26959871] (req 26955776->26959871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760768.539992:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac403b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512704896:12345-192.168.203.56@tcp:4:dd.0 Request processed in 38698us (38950us total) trans 4294973900 rc 0/0 00000100:00100000:2.0:1789760768.540003:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6611 00000100:00100000:2.0:1789760768.540013:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 748014686016 time 748065577474 state 0x2 ref 0 00000100:00100000:3.0:1789760768.547186:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760768.547478:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512705024 00000100:00100000:0.0:1789760768.547489:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512705024 00000100:00100000:0.0:1789760768.547552:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 748014686016 time 748065577474 state 0x0 ref 1 00000100:00100000:0.0:1789760768.547568:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 748065837890 time 748065577474 state 0x1 ref 1 00000100:00100000:0.0:1789760768.547601:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 748065837890 time 748111901323 state 0x0 ref 1 00000100:00100000:0.0:1789760768.547616:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5304380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512705024:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760768.547665:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26959872, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0xd007 00010000:00010000:0.0:1789760768.547675:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2f524 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.547690:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=748s v=(0 0) 00010000:00010000:0.0:1789760768.547736:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760768.547743:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2f524 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26959872->26963967] (req 26959872->26963967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.547756:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2f524 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26959872->26963967] (req 26959872->26963967) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760768.603524:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973901 is committed 00002000:00010000:2.0:1789760768.604119:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26959872 -> 26963968 00010000:00010000:2.0:1789760768.604146:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2f524 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26959872->26963967] (req 26959872->26963967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.604159:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2f524 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26959872->26963967] (req 26959872->26963967) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.604167:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2f524 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26959872->26963967] (req 26959872->26963967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.604175:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760768.604180:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2f524 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26959872->26963967] (req 26959872->26963967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.604189:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2f524 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26959872->26963967] (req 26959872->26963967) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.604213:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2f524 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26959872->26963967] (req 26959872->26963967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.604221:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2f524 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26959872->26963967] (req 26959872->26963967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760768.604314:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5304380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512705024:12345-192.168.203.56@tcp:4:dd.0 Request processed in 56695us (57132us total) trans 4294973901 rc 0/0 00000100:00100000:2.0:1789760768.604323:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6612 00000100:00100000:2.0:1789760768.604333:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 748065837890 time 748111901323 state 0x2 ref 0 00000100:00100000:0.0:1789760768.609827:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760768.610530:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512705152 00000100:00100000:2.0:1789760768.610541:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512705152 00000100:00100000:2.0:1789760768.610692:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 748065837890 time 748111901323 state 0x0 ref 1 00000100:00100000:2.0:1789760768.610788:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 748112161739 time 748111901323 state 0x1 ref 1 00000100:00100000:2.0:1789760768.610855:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 748112161739 time 748175154511 state 0x0 ref 1 00000100:00100000:2.0:1789760768.610868:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4058e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512705152:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760768.610916:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26963968, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0xd007 00010000:00010000:2.0:1789760768.610925:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2f52b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.610941:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=748s v=(0 0) 00010000:00010000:2.0:1789760768.610945:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760768.610951:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2f52b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26963968->26968063] (req 26963968->26968063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.610965:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2f52b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26963968->26968063] (req 26963968->26968063) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760768.646022:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973902 is committed 00002000:00010000:3.0:1789760768.646230:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26963968 -> 26968064 00010000:00010000:3.0:1789760768.646243:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2f52b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26963968->26968063] (req 26963968->26968063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.646338:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2f52b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26963968->26968063] (req 26963968->26968063) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.646345:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2f52b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26963968->26968063] (req 26963968->26968063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.646353:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760768.646358:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2f52b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26963968->26968063] (req 26963968->26968063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.646372:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2f52b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26963968->26968063] (req 26963968->26968063) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.646393:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2f52b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26963968->26968063] (req 26963968->26968063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.646400:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2f52b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26963968->26968063] (req 26963968->26968063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760768.646486:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4058e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512705152:12345-192.168.203.56@tcp:4:dd.0 Request processed in 35615us (36662us total) trans 4294973902 rc 0/0 00000100:00100000:3.0:1789760768.646496:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6613 00000100:00100000:3.0:1789760768.646505:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 748112161739 time 748175154511 state 0x2 ref 0 00000100:00100000:1.0:1789760768.656462:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760768.656611:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512705280 00000100:00100000:2.0:1789760768.656621:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512705280 00000100:00100000:2.0:1789760768.656669:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 748112161739 time 748175154511 state 0x0 ref 1 00000100:00100000:2.0:1789760768.656684:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 748175414927 time 748175154511 state 0x1 ref 1 00000100:00100000:2.0:1789760768.656746:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 748175414927 time 748221045206 state 0x0 ref 1 00000100:00100000:2.0:1789760768.656759:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac403a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512705280:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760768.657396:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26968064, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0xd007 00010000:00010000:2.0:1789760768.657408:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2f532 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.657423:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=748s v=(0 0) 00010000:00010000:2.0:1789760768.657428:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760768.657434:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2f532 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26968064->26972159] (req 26968064->26972159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.657458:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2f532 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26968064->26972159] (req 26968064->26972159) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760768.716677:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26968064 -> 26972160 00010000:00010000:3.0:1789760768.716716:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2f532 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26968064->26972159] (req 26968064->26972159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760768.716720:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973903 is committed 00010000:00010000:3.0:1789760768.716733:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2f532 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26968064->26972159] (req 26968064->26972159) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.716742:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2f532 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26968064->26972159] (req 26968064->26972159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.716750:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760768.716757:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2f532 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26968064->26972159] (req 26968064->26972159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.716766:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2f532 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26968064->26972159] (req 26968064->26972159) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.716794:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2f532 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26968064->26972159] (req 26968064->26972159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.716811:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2f532 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26968064->26972159] (req 26968064->26972159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760768.717014:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac403a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512705280:12345-192.168.203.56@tcp:4:dd.0 Request processed in 60252us (60563us total) trans 4294973903 rc 0/0 00000100:00100000:3.0:1789760768.717026:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6614 00000100:00100000:3.0:1789760768.717035:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 748175414927 time 748221045206 state 0x2 ref 0 00000100:00100000:2.0:1789760768.720585:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760768.720679:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512705408 00000100:00100000:1.0:1789760768.720690:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512705408 00000100:00100000:1.0:1789760768.721225:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 748175414927 time 748221045206 state 0x0 ref 1 00000100:00100000:1.0:1789760768.721240:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 748221305622 time 748221045206 state 0x1 ref 1 00000100:00100000:1.0:1789760768.721262:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 748221305622 time 748285562413 state 0x0 ref 1 00000100:00100000:1.0:1789760768.721273:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac3353480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512705408:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760768.721320:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26972160, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0xd007 00010000:00010000:1.0:1789760768.721328:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2f539 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.721343:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=748s v=(0 0) 00010000:00010000:1.0:1789760768.721348:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760768.721353:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2f539 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26972160->26976255] (req 26972160->26976255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.721441:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2f539 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26972160->26976255] (req 26972160->26976255) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760768.781640:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973904 is committed 00002000:00010000:3.0:1789760768.781752:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26972160 -> 26976256 00010000:00010000:3.0:1789760768.781767:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2f539 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26972160->26976255] (req 26972160->26976255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.781915:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2f539 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26972160->26976255] (req 26972160->26976255) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.781925:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2f539 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26972160->26976255] (req 26972160->26976255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.781937:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760768.781942:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2f539 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26972160->26976255] (req 26972160->26976255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.781951:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2f539 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26972160->26976255] (req 26972160->26976255) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.781970:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2f539 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26972160->26976255] (req 26972160->26976255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.781977:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2f539 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26972160->26976255] (req 26972160->26976255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760768.782103:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac3353480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512705408:12345-192.168.203.56@tcp:4:dd.0 Request processed in 60826us (61579us total) trans 4294973904 rc 0/0 00000100:00100000:3.0:1789760768.782114:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6615 00000100:00100000:3.0:1789760768.782123:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 748221305622 time 748285562413 state 0x2 ref 0 00000100:00100000:0.0:1789760768.784880:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760768.784954:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512705536 00000100:00100000:3.0:1789760768.784964:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512705536 00000100:00100000:3.0:1789760768.785015:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 748221305622 time 748285562413 state 0x0 ref 1 00000100:00100000:3.0:1789760768.785031:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 748285822829 time 748285562413 state 0x1 ref 1 00000100:00100000:3.0:1789760768.785061:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 748285822829 time 748349361111 state 0x0 ref 1 00000100:00100000:3.0:1789760768.785077:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4059180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512705536:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760768.785122:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26976256, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0xd007 00010000:00010000:3.0:1789760768.785139:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2f540 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.785156:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=748s v=(0 0) 00010000:00010000:3.0:1789760768.785160:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760768.785167:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2f540 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26976256->26980351] (req 26976256->26980351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.785179:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2f540 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26976256->26980351] (req 26976256->26980351) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760768.825733:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26976256 -> 26980352 00010000:00010000:3.0:1789760768.825754:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2f540 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26976256->26980351] (req 26976256->26980351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.825767:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2f540 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26976256->26980351] (req 26976256->26980351) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.825872:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2f540 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26976256->26980351] (req 26976256->26980351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.825928:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760768.825933:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2f540 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26976256->26980351] (req 26976256->26980351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.825943:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2f540 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26976256->26980351] (req 26976256->26980351) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.825962:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2f540 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26976256->26980351] (req 26976256->26980351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.825971:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2f540 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26976256->26980351] (req 26976256->26980351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760768.826067:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4059180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2358:x1876699512705536:12345-192.168.203.56@tcp:4:dd.0 Request processed in 40987us (41192us total) trans 4294973905 rc 0/0 00000100:00100000:3.0:1789760768.826078:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6616 00000100:00100000:3.0:1789760768.826087:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 748285822829 time 748349361111 state 0x2 ref 0 00000001:00080000:2.0:1789760768.826173:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973905 is committed 00000100:00100000:1.0:1789760768.828553:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760768.828659:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512705664 00000100:00100000:3.0:1789760768.828668:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512705664 00000100:00100000:3.0:1789760768.828742:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 748285822829 time 748349361111 state 0x0 ref 1 00000100:00100000:3.0:1789760768.828758:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 748349621527 time 748349361111 state 0x1 ref 1 00000100:00100000:0.0:1789760768.828920:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 748349621527 time 748393217438 state 0x0 ref 1 00000100:00100000:0.0:1789760768.828936:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4038700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512705664:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760768.828987:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26980352, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0xd007 00010000:00010000:0.0:1789760768.828996:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2f547 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.829012:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=748s v=(0 0) 00010000:00010000:0.0:1789760768.829017:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760768.829023:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2f547 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26980352->26984447] (req 26980352->26984447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.829036:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2f547 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26980352->26984447] (req 26980352->26984447) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760768.879282:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973906 is committed 00002000:00010000:2.0:1789760768.879664:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26980352 -> 26984448 00010000:00010000:2.0:1789760768.879684:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2f547 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26980352->26984447] (req 26980352->26984447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.879713:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2f547 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26980352->26984447] (req 26980352->26984447) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.879723:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2f547 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26980352->26984447] (req 26980352->26984447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.879729:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760768.879733:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2f547 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26980352->26984447] (req 26980352->26984447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.879743:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2f547 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26980352->26984447] (req 26980352->26984447) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.879763:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2f547 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26980352->26984447] (req 26980352->26984447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.879781:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2f547 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26980352->26984447] (req 26980352->26984447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760768.879890:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4038700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512705664:12345-192.168.203.56@tcp:4:dd.0 Request processed in 50954us (51341us total) trans 4294973906 rc 0/0 00000100:00100000:2.0:1789760768.879900:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6617 00000100:00100000:2.0:1789760768.879910:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 748349621527 time 748393217438 state 0x2 ref 0 00000100:00100000:0.0:1789760768.889605:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760768.889893:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512705792 00000100:00100000:3.0:1789760768.889903:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512705792 00000100:00100000:3.0:1789760768.889962:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 748349621527 time 748393217438 state 0x0 ref 1 00000100:00100000:3.0:1789760768.889977:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 748393477854 time 748393217438 state 0x1 ref 1 00000100:00100000:3.0:1789760768.890012:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 748393477854 time 748454311483 state 0x0 ref 1 00000100:00100000:3.0:1789760768.890024:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febff932a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512705792:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760768.890072:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26984448, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0xd007 00010000:00010000:3.0:1789760768.890081:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2f54e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.890097:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=748s v=(0 0) 00010000:00010000:3.0:1789760768.890101:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760768.890106:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2f54e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26984448->26988543] (req 26984448->26988543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.890120:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2f54e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26984448->26988543] (req 26984448->26988543) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760768.941473:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26984448 -> 26988544 00010000:00010000:3.0:1789760768.941492:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2f54e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26984448->26988543] (req 26984448->26988543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.941513:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2f54e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26984448->26988543] (req 26984448->26988543) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.941525:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2f54e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26984448->26988543] (req 26984448->26988543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.941533:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760768.941538:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2f54e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26984448->26988543] (req 26984448->26988543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.941547:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2f54e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26984448->26988543] (req 26984448->26988543) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760768.941552:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973907 is committed 00010000:00010000:3.0:1789760768.941568:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2f54e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26984448->26988543] (req 26984448->26988543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.941578:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2f54e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26984448->26988543] (req 26984448->26988543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760768.941670:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febff932a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512705792:12345-192.168.203.56@tcp:4:dd.0 Request processed in 51642us (52069us total) trans 4294973907 rc 0/0 00000100:00100000:3.0:1789760768.941681:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6618 00000100:00100000:3.0:1789760768.941691:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 748393477854 time 748454311483 state 0x2 ref 0 00000100:00100000:0.0:1789760768.945240:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760768.945530:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512705920 00000100:00100000:3.0:1789760768.945540:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512705920 00000100:00100000:3.0:1789760768.945595:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 748393477854 time 748454311483 state 0x0 ref 1 00000100:00100000:3.0:1789760768.945610:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 748454571899 time 748454311483 state 0x1 ref 1 00000100:00100000:3.0:1789760768.945645:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 748454571899 time 748509944865 state 0x0 ref 1 00000100:00100000:3.0:1789760768.945658:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febff930e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512705920:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760768.945732:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26988544, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0xd007 00010000:00010000:3.0:1789760768.945742:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2f555 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.945759:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=748s v=(0 0) 00010000:00010000:3.0:1789760768.945764:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760768.945771:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2f555 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26988544->26992639] (req 26988544->26992639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.945786:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2f555 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26988544->26992639] (req 26988544->26992639) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760768.984226:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973908 is committed 00002000:00010000:3.0:1789760768.984319:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26988544 -> 26992640 00010000:00010000:3.0:1789760768.984335:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2f555 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26988544->26992639] (req 26988544->26992639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.984348:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2f555 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26988544->26992639] (req 26988544->26992639) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.984358:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2f555 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26988544->26992639] (req 26988544->26992639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.984365:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760768.984370:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2f555 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26988544->26992639] (req 26988544->26992639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.984379:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2f555 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26988544->26992639] (req 26988544->26992639) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.984398:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2f555 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26988544->26992639] (req 26988544->26992639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.984406:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2f555 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26988544->26992639] (req 26988544->26992639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760768.984499:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febff930e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512705920:12345-192.168.203.56@tcp:4:dd.0 Request processed in 38838us (39263us total) trans 4294973908 rc 0/0 00000100:00100000:3.0:1789760768.984508:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6619 00000100:00100000:3.0:1789760768.984517:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 748454571899 time 748509944865 state 0x2 ref 0 00000100:00100000:3.0:1789760768.987078:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760768.987563:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512706048 00000100:00100000:2.0:1789760768.987574:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512706048 00000100:00100000:2.0:1789760768.987626:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 748454571899 time 748509944865 state 0x0 ref 1 00000100:00100000:2.0:1789760768.987641:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 748510205281 time 748509944865 state 0x1 ref 1 00000100:00100000:2.0:1789760768.987726:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 748510205281 time 748552024959 state 0x0 ref 1 00000100:00100000:2.0:1789760768.987740:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5306680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512706048:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760768.987786:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26992640, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0xd007 00010000:00010000:2.0:1789760768.987795:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2f55c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.987820:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=748s v=(0 0) 00010000:00010000:2.0:1789760768.987825:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760768.987832:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2f55c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26992640->26996735] (req 26992640->26996735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.987845:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2f55c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26992640->26996735] (req 26992640->26996735) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760769.018958:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973909 is committed 00002000:00010000:1.0:1789760769.019033:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26992640 -> 26996736 00010000:00010000:1.0:1789760769.019049:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2f55c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26992640->26996735] (req 26992640->26996735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.019067:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2f55c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26992640->26996735] (req 26992640->26996735) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.019076:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2f55c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26992640->26996735] (req 26992640->26996735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.019083:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760769.019089:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2f55c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26992640->26996735] (req 26992640->26996735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.019098:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2f55c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26992640->26996735] (req 26992640->26996735) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.019118:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2f55c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26992640->26996735] (req 26992640->26996735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.019125:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2f55c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26992640->26996735] (req 26992640->26996735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760769.019257:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5306680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512706048:12345-192.168.203.56@tcp:4:dd.0 Request processed in 31515us (32182us total) trans 4294973909 rc 0/0 00000100:00100000:1.0:1789760769.019268:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6620 00000100:00100000:1.0:1789760769.019278:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 748510205281 time 748552024959 state 0x2 ref 0 00000100:00100000:2.0:1789760769.023556:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760769.025234:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512706176 00000100:00100000:3.0:1789760769.025246:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512706176 00000100:00100000:0.0:1789760769.026771:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 748510205281 time 748552024959 state 0x0 ref 1 00000100:00100000:0.0:1789760769.026790:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 748552285375 time 748552024959 state 0x1 ref 1 00000100:00080000:2.0:1789760769.027115:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1789760769.027184:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 748552285375 time 748591481682 state 0x0 ref 1 00000100:00100000:0.0:1789760769.027199:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac3352300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512706176:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760769.027248:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26996736, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0xd007 00010000:00010000:0.0:1789760769.027256:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f563 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.027271:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=748s v=(0 0) 00010000:00010000:0.0:1789760769.027275:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760769.027281:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f563 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26996736->27000831] (req 26996736->27000831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.027301:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f563 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26996736->27000831] (req 26996736->27000831) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760769.067556:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973910 is committed 00002000:00010000:0.0:1789760769.067913:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26996736 -> 27000832 00010000:00010000:0.0:1789760769.067929:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f563 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26996736->27000831] (req 26996736->27000831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.067942:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f563 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26996736->27000831] (req 26996736->27000831) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.067951:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f563 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26996736->27000831] (req 26996736->27000831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.067959:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760769.067964:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f563 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26996736->27000831] (req 26996736->27000831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.067973:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f563 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26996736->27000831] (req 26996736->27000831) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.067993:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f563 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26996736->27000831] (req 26996736->27000831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.068000:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f563 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26996736->27000831] (req 26996736->27000831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760769.068141:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac3352300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512706176:12345-192.168.203.56@tcp:4:dd.0 Request processed in 40932us (44580us total) trans 4294973910 rc 0/0 00000100:00100000:0.0:1789760769.068151:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6621 00000100:00100000:0.0:1789760769.068161:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 748552285375 time 748591481682 state 0x2 ref 0 00000100:00100000:0.0:1789760769.074341:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760769.074422:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512706304 00000100:00100000:3.0:1789760769.074433:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512706304 00000100:00100000:3.0:1789760769.074484:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 748552285375 time 748591481682 state 0x0 ref 1 00000100:00100000:3.0:1789760769.074500:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 748591742098 time 748591481682 state 0x1 ref 1 00000100:00100000:3.0:1789760769.074539:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 748591742098 time 748638838760 state 0x0 ref 1 00000100:00100000:3.0:1789760769.074550:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febfc0a0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512706304:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760769.074593:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27000832, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0xd007 00010000:00010000:3.0:1789760769.074600:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2f56a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.074615:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=748s v=(0 0) 00010000:00010000:3.0:1789760769.074619:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760769.074625:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2f56a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27000832->27004927] (req 27000832->27004927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.074638:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2f56a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27000832->27004927] (req 27000832->27004927) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760769.155424:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27000832 -> 27004928 00010000:00010000:1.0:1789760769.155574:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2f56a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27000832->27004927] (req 27000832->27004927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.155588:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2f56a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27000832->27004927] (req 27000832->27004927) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.155596:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2f56a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27000832->27004927] (req 27000832->27004927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.155604:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760769.155609:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2f56a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27000832->27004927] (req 27000832->27004927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.155618:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2f56a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27000832->27004927] (req 27000832->27004927) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.155637:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2f56a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27000832->27004927] (req 27000832->27004927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.155644:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2f56a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27000832->27004927] (req 27000832->27004927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760769.155874:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973911 is committed 00000100:00100000:0.0:1789760769.156185:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febfc0a0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512706304:12345-192.168.203.56@tcp:4:dd.0 Request processed in 81631us (81848us total) trans 4294973911 rc 0/0 00000100:00100000:0.0:1789760769.156196:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6622 00000100:00100000:0.0:1789760769.156207:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 748591742098 time 748638838760 state 0x2 ref 0 00000100:00100000:3.0:1789760769.159458:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760769.159533:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512706432 00000100:00100000:1.0:1789760769.159542:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512706432 00000100:00100000:1.0:1789760769.159596:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 748591742098 time 748638838760 state 0x0 ref 1 00000100:00100000:1.0:1789760769.159612:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 748639099176 time 748638838760 state 0x1 ref 1 00000100:00100000:1.0:1789760769.159644:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 748639099176 time 748723943498 state 0x0 ref 1 00000100:00100000:1.0:1789760769.159656:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5307100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512706432:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760769.159722:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27004928, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0xd007 00010000:00010000:1.0:1789760769.159732:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2f571 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.159748:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=748s v=(0 0) 00010000:00010000:1.0:1789760769.159752:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760769.159758:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2f571 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27004928->27009023] (req 27004928->27009023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.159785:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2f571 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27004928->27009023] (req 27004928->27009023) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760769.194637:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27004928 -> 27009024 00010000:00010000:1.0:1789760769.194656:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2f571 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27004928->27009023] (req 27004928->27009023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.194668:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2f571 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27004928->27009023] (req 27004928->27009023) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.194676:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2f571 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27004928->27009023] (req 27004928->27009023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.194684:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760769.194689:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2f571 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27004928->27009023] (req 27004928->27009023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.194730:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2f571 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27004928->27009023] (req 27004928->27009023) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.194749:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2f571 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27004928->27009023] (req 27004928->27009023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.194758:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2f571 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27004928->27009023] (req 27004928->27009023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760769.194848:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5307100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2358:x1876699512706432:12345-192.168.203.56@tcp:4:dd.0 Request processed in 35189us (35394us total) trans 4294973912 rc 0/0 00000100:00100000:1.0:1789760769.194857:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6623 00000100:00100000:1.0:1789760769.194867:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 748639099176 time 748723943498 state 0x2 ref 0 00000001:00080000:3.0:1789760769.195564:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973912 is committed 00000100:00100000:2.0:1789760769.200678:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760769.200985:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512706560 00000100:00100000:0.0:1789760769.200994:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512706560 00000100:00100000:0.0:1789760769.201050:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 748639099176 time 748723943498 state 0x0 ref 1 00000100:00100000:0.0:1789760769.201065:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 748724203914 time 748723943498 state 0x1 ref 1 00000100:00100000:0.0:1789760769.201094:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 748724203914 time 748765393795 state 0x0 ref 1 00000100:00100000:0.0:1789760769.201106:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac3353800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512706560:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760769.201163:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27009024, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0xd007 00010000:00010000:0.0:1789760769.201172:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f578 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.201197:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=748s v=(0 0) 00010000:00010000:0.0:1789760769.201203:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760769.201209:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f578 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27009024->27013119] (req 27009024->27013119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.201222:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f578 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27009024->27013119] (req 27009024->27013119) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760769.244363:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973913 is committed 00002000:00010000:1.0:1789760769.244476:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27009024 -> 27013120 00010000:00010000:1.0:1789760769.244492:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f578 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27009024->27013119] (req 27009024->27013119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.244505:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f578 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27009024->27013119] (req 27009024->27013119) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.244515:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f578 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27009024->27013119] (req 27009024->27013119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.244527:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760769.244532:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f578 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27009024->27013119] (req 27009024->27013119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.244541:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f578 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27009024->27013119] (req 27009024->27013119) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.244560:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f578 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27009024->27013119] (req 27009024->27013119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.244567:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f578 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27009024->27013119] (req 27009024->27013119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760769.244660:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac3353800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512706560:12345-192.168.203.56@tcp:4:dd.0 Request processed in 43550us (43986us total) trans 4294973913 rc 0/0 00000100:00100000:1.0:1789760769.244670:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6624 00000100:00100000:1.0:1789760769.244678:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 748724203914 time 748765393795 state 0x2 ref 0 00000100:00100000:2.0:1789760769.249932:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760769.250026:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512706688 00000100:00100000:0.0:1789760769.250035:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512706688 00000100:00100000:0.0:1789760769.250087:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 748724203914 time 748765393795 state 0x0 ref 1 00000100:00100000:0.0:1789760769.250102:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 748765654211 time 748765393795 state 0x1 ref 1 00000100:00100000:0.0:1789760769.250141:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 748765654211 time 748814441276 state 0x0 ref 1 00000100:00100000:0.0:1789760769.250154:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febfbddca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512706688:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760769.250199:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27013120, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0xd007 00010000:00010000:0.0:1789760769.250208:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2f57f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.250223:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=748s v=(0 0) 00010000:00010000:0.0:1789760769.250227:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760769.250233:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2f57f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27013120->27017215] (req 27013120->27017215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.250245:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2f57f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27013120->27017215] (req 27013120->27017215) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760769.292657:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973914 is committed 00002000:00010000:1.0:1789760769.292764:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27013120 -> 27017216 00010000:00010000:1.0:1789760769.292779:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2f57f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27013120->27017215] (req 27013120->27017215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.292792:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2f57f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27013120->27017215] (req 27013120->27017215) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.292873:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2f57f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27013120->27017215] (req 27013120->27017215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.292880:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760769.292885:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2f57f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27013120->27017215] (req 27013120->27017215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.292894:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2f57f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27013120->27017215] (req 27013120->27017215) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.292914:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2f57f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27013120->27017215] (req 27013120->27017215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.292921:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2f57f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27013120->27017215] (req 27013120->27017215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760769.293029:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febfbddca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512706688:12345-192.168.203.56@tcp:4:dd.0 Request processed in 42870us (43099us total) trans 4294973914 rc 0/0 00000100:00100000:1.0:1789760769.293039:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6625 00000100:00100000:1.0:1789760769.293048:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 748765654211 time 748814441276 state 0x2 ref 0 00000100:00100000:3.0:1789760769.300113:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760769.301766:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512706816 00000100:00100000:1.0:1789760769.301780:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512706816 00000100:00100000:1.0:1789760769.301842:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 748765654211 time 748814441276 state 0x0 ref 1 00000100:00100000:1.0:1789760769.301857:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 748814701692 time 748814441276 state 0x1 ref 1 00000100:00100000:1.0:1789760769.301892:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 748814701692 time 748866191805 state 0x0 ref 1 00000100:00100000:1.0:1789760769.301906:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5304a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512706816:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760769.301953:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27017216, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0xd007 00010000:00010000:1.0:1789760769.301961:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2f586 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.301978:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=748s v=(0 0) 00010000:00010000:1.0:1789760769.301983:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760769.301990:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2f586 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27017216->27021311] (req 27017216->27021311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.302004:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2f586 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27017216->27021311] (req 27017216->27021311) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760769.363454:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973915 is committed 00002000:00010000:1.0:1789760769.364002:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27017216 -> 27021312 00010000:00010000:1.0:1789760769.364020:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2f586 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27017216->27021311] (req 27017216->27021311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.364035:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2f586 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27017216->27021311] (req 27017216->27021311) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.364044:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2f586 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27017216->27021311] (req 27017216->27021311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.364051:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760769.364057:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2f586 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27017216->27021311] (req 27017216->27021311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.364066:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2f586 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27017216->27021311] (req 27017216->27021311) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.364164:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2f586 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27017216->27021311] (req 27017216->27021311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.364175:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2f586 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27017216->27021311] (req 27017216->27021311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760769.364369:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5304a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512706816:12345-192.168.203.56@tcp:4:dd.0 Request processed in 62461us (64260us total) trans 4294973915 rc 0/0 00000100:00100000:1.0:1789760769.364380:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6626 00000100:00100000:1.0:1789760769.364390:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 748814701692 time 748866191805 state 0x2 ref 0 00000100:00100000:0.0:1789760769.370659:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760769.370928:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512706944 00000100:00100000:1.0:1789760769.370938:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512706944 00000100:00100000:1.0:1789760769.370992:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 748814701692 time 748866191805 state 0x0 ref 1 00000100:00100000:1.0:1789760769.371007:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 748866452221 time 748866191805 state 0x1 ref 1 00000100:00100000:1.0:1789760769.371037:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 748866452221 time 748935336311 state 0x0 ref 1 00000100:00100000:1.0:1789760769.371050:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febfc0a3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512706944:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760769.371112:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27021312, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0xd007 00010000:00010000:1.0:1789760769.371122:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2f58d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.371149:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=748s v=(0 0) 00010000:00010000:1.0:1789760769.371154:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760769.371162:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2f58d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27021312->27025407] (req 27021312->27025407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.371176:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2f58d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27021312->27025407] (req 27021312->27025407) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.400529:0:5836:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:2.0:1789760769.413308:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973916 is committed 00002000:00010000:1.0:1789760769.413416:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27021312 -> 27025408 00010000:00010000:1.0:1789760769.413432:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2f58d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27021312->27025407] (req 27021312->27025407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.413456:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2f58d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27021312->27025407] (req 27021312->27025407) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.413466:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2f58d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27021312->27025407] (req 27021312->27025407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.413475:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760769.413480:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2f58d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27021312->27025407] (req 27021312->27025407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.413489:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2f58d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27021312->27025407] (req 27021312->27025407) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.413514:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2f58d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27021312->27025407] (req 27021312->27025407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.413532:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2f58d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27021312->27025407] (req 27021312->27025407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760769.413924:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febfc0a3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512706944:12345-192.168.203.56@tcp:4:dd.0 Request processed in 42870us (43270us total) trans 4294973916 rc 0/0 00000100:00100000:1.0:1789760769.413935:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6627 00000100:00100000:1.0:1789760769.413945:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 748866452221 time 748935336311 state 0x2 ref 0 00000100:00100000:0.0:1789760769.419567:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760769.419683:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512707072 00000100:00100000:1.0:1789760769.419693:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512707072 00000100:00100000:1.0:1789760769.419770:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 748866452221 time 748935336311 state 0x0 ref 1 00000100:00100000:1.0:1789760769.419785:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 748935596727 time 748935336311 state 0x1 ref 1 00000100:00100000:1.0:1789760769.419815:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 748935596727 time 748984114786 state 0x0 ref 1 00000100:00100000:1.0:1789760769.419825:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4360a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512707072:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760769.419876:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27025408, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0xd007 00010000:00010000:1.0:1789760769.419886:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2f594 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.419902:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=748s v=(0 0) 00010000:00010000:1.0:1789760769.419907:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760769.419913:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2f594 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27025408->27029503] (req 27025408->27029503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.419926:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2f594 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27025408->27029503] (req 27025408->27029503) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760769.488180:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973917 is committed 00002000:00010000:1.0:1789760769.488324:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27025408 -> 27029504 00010000:00010000:1.0:1789760769.488339:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2f594 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27025408->27029503] (req 27025408->27029503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.488352:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2f594 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27025408->27029503] (req 27025408->27029503) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.488361:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2f594 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27025408->27029503] (req 27025408->27029503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.488369:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760769.488374:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2f594 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27025408->27029503] (req 27025408->27029503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.488383:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2f594 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27025408->27029503] (req 27025408->27029503) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.488405:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2f594 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27025408->27029503] (req 27025408->27029503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.488412:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2f594 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27025408->27029503] (req 27025408->27029503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760769.488507:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4360a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512707072:12345-192.168.203.56@tcp:4:dd.0 Request processed in 68678us (68943us total) trans 4294973917 rc 0/0 00000100:00100000:1.0:1789760769.489369:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6628 00000100:00100000:1.0:1789760769.489382:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 748935596727 time 748984114786 state 0x2 ref 0 00000100:00100000:0.0:1789760769.492831:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760769.493351:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512707200 00000100:00100000:1.0:1789760769.493361:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512707200 00000100:00100000:1.0:1789760769.493413:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 748935596727 time 748984114786 state 0x0 ref 1 00000100:00100000:1.0:1789760769.493428:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 748984375202 time 748984114786 state 0x1 ref 1 00000100:00100000:1.0:1789760769.493465:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 748984375202 time 749057765319 state 0x0 ref 1 00000100:00100000:1.0:1789760769.493478:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4362a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512707200:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760769.493525:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27029504, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0xd007 00010000:00010000:1.0:1789760769.493535:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2f59b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.493551:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=749s v=(0 0) 00010000:00010000:1.0:1789760769.493556:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760769.493563:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2f59b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27029504->27033599] (req 27029504->27033599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.493576:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2f59b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27029504->27033599] (req 27029504->27033599) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760769.526213:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973918 is committed 00002000:00010000:1.0:1789760769.526375:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27029504 -> 27033600 00002000:00010000:1.0:1789760769.526380:0:13062:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760768 -> 1789760769 00002000:00010000:1.0:1789760769.526383:0:13062:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760768 -> 1789760769 00010000:00010000:1.0:1789760769.526398:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2f59b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27029504->27033599] (req 27029504->27033599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.526409:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2f59b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27029504->27033599] (req 27029504->27033599) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.526417:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2f59b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27029504->27033599] (req 27029504->27033599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.526425:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760769.526430:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2f59b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27029504->27033599] (req 27029504->27033599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.526509:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2f59b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27029504->27033599] (req 27029504->27033599) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.526531:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2f59b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27029504->27033599] (req 27029504->27033599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.526538:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2f59b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27029504->27033599] (req 27029504->27033599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760769.526643:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4362a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512707200:12345-192.168.203.56@tcp:4:dd.0 Request processed in 33161us (33820us total) trans 4294973918 rc 0/0 00000100:00100000:1.0:1789760769.526654:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6629 00000100:00100000:1.0:1789760769.526665:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 748984375202 time 749057765319 state 0x2 ref 0 00000100:00100000:0.0:1789760769.532170:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760769.532343:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512707328 00000100:00100000:1.0:1789760769.532353:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512707328 00000100:00100000:1.0:1789760769.532403:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 748984375202 time 749057765319 state 0x0 ref 1 00000100:00100000:1.0:1789760769.532418:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 749058025735 time 749057765319 state 0x1 ref 1 00000100:00100000:1.0:1789760769.532453:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 749058025735 time 749096742225 state 0x0 ref 1 00000100:00100000:1.0:1789760769.532467:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4363100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512707328:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760769.532520:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27033600, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0xd007 00010000:00010000:1.0:1789760769.532528:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2f5a2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.532545:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=749s v=(0 0) 00010000:00010000:1.0:1789760769.532550:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760769.532556:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2f5a2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27033600->27037695] (req 27033600->27037695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.532570:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2f5a2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27033600->27037695] (req 27033600->27037695) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760769.573119:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973919 is committed 00002000:00010000:2.0:1789760769.573315:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27033600 -> 27037696 00010000:00010000:2.0:1789760769.573331:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2f5a2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27033600->27037695] (req 27033600->27037695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.573343:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2f5a2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27033600->27037695] (req 27033600->27037695) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.573352:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2f5a2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27033600->27037695] (req 27033600->27037695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.573359:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760769.573365:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2f5a2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27033600->27037695] (req 27033600->27037695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.573375:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2f5a2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27033600->27037695] (req 27033600->27037695) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.573396:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2f5a2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27033600->27037695] (req 27033600->27037695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.573411:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2f5a2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27033600->27037695] (req 27033600->27037695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760769.573611:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4363100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512707328:12345-192.168.203.56@tcp:4:dd.0 Request processed in 41142us (41446us total) trans 4294973919 rc 0/0 00000100:00100000:2.0:1789760769.573622:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6630 00000100:00100000:2.0:1789760769.573632:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 749058025735 time 749096742225 state 0x2 ref 0 00000100:00100000:0.0:1789760769.578101:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760769.578199:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512707456 00000100:00100000:1.0:1789760769.578209:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512707456 00000100:00100000:1.0:1789760769.578265:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 749058025735 time 749096742225 state 0x0 ref 1 00000100:00100000:1.0:1789760769.578282:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 749097002641 time 749096742225 state 0x1 ref 1 00000100:00100000:1.0:1789760769.578315:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 749097002641 time 749142614607 state 0x0 ref 1 00000100:00100000:1.0:1789760769.578329:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4362d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512707456:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760769.578389:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27037696, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0xd007 00010000:00010000:1.0:1789760769.578400:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2f5a9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.578421:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=749s v=(0 0) 00010000:00010000:1.0:1789760769.578426:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760769.578433:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2f5a9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27037696->27041791] (req 27037696->27041791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.578460:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2f5a9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27037696->27041791] (req 27037696->27041791) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760769.619645:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27037696 -> 27041792 00010000:00010000:2.0:1789760769.619665:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2f5a9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27037696->27041791] (req 27037696->27041791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.619678:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2f5a9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27037696->27041791] (req 27037696->27041791) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.619687:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2f5a9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27037696->27041791] (req 27037696->27041791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.619712:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760769.619718:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2f5a9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27037696->27041791] (req 27037696->27041791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.619727:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2f5a9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27037696->27041791] (req 27037696->27041791) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.619751:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2f5a9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27037696->27041791] (req 27037696->27041791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.619758:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2f5a9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27037696->27041791] (req 27037696->27041791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760769.619854:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4362d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2358:x1876699512707456:12345-192.168.203.56@tcp:4:dd.0 Request processed in 41522us (41759us total) trans 4294973920 rc 0/0 00000100:00100000:2.0:1789760769.619864:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6631 00000100:00100000:2.0:1789760769.619875:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 749097002641 time 749142614607 state 0x2 ref 0 00000001:00080000:0.0:1789760769.619941:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973920 is committed 00000100:00100000:0.0:1789760769.622618:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760769.622771:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512707584 00000100:00100000:1.0:1789760769.622780:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512707584 00000100:00100000:1.0:1789760769.623038:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 749097002641 time 749142614607 state 0x0 ref 1 00000100:00100000:1.0:1789760769.623052:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 749142875023 time 749142614607 state 0x1 ref 1 00000100:00100000:1.0:1789760769.623122:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 749142875023 time 749187421288 state 0x0 ref 1 00000100:00100000:1.0:1789760769.623143:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febfb897100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512707584:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760769.623186:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27041792, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0xd007 00010000:00010000:1.0:1789760769.623195:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2f5b0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.623209:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=749s v=(0 0) 00010000:00010000:1.0:1789760769.623214:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760769.623220:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2f5b0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27041792->27045887] (req 27041792->27045887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.623232:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2f5b0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27041792->27045887] (req 27041792->27045887) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760769.648294:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27041792 -> 27045888 00010000:00010000:2.0:1789760769.648318:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2f5b0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27041792->27045887] (req 27041792->27045887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.648335:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2f5b0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27041792->27045887] (req 27041792->27045887) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.648344:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2f5b0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27041792->27045887] (req 27041792->27045887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.648353:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760769.648358:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2f5b0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27041792->27045887] (req 27041792->27045887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.648375:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2f5b0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27041792->27045887] (req 27041792->27045887) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.648400:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2f5b0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27041792->27045887] (req 27041792->27045887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.648408:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2f5b0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27041792->27045887] (req 27041792->27045887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760769.648525:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febfb897100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2361:x1876699512707584:12345-192.168.203.56@tcp:4:dd.0 Request processed in 25378us (25910us total) trans 4294973921 rc 0/0 00000100:00100000:2.0:1789760769.648538:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6632 00000100:00100000:2.0:1789760769.648550:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 749142875023 time 749187421288 state 0x2 ref 0 00000001:00080000:3.0:1789760769.648666:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973921 is committed 00000100:00100000:1.0:1789760769.652090:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760769.652525:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512707712 00000100:00100000:2.0:1789760769.652534:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512707712 00000100:00100000:2.0:1789760769.652591:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 749142875023 time 749187421288 state 0x0 ref 1 00000100:00100000:2.0:1789760769.652606:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 749187681704 time 749187421288 state 0x1 ref 1 00000100:00100000:2.0:1789760769.652635:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 749187681704 time 749216934682 state 0x0 ref 1 00000100:00100000:2.0:1789760769.652648:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4039f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512707712:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760769.652759:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27045888, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0xd007 00010000:00010000:2.0:1789760769.652768:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2f5b7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.652786:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=749s v=(0 0) 00010000:00010000:2.0:1789760769.652791:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760769.652805:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2f5b7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27045888->27049983] (req 27045888->27049983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.652820:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2f5b7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27045888->27049983] (req 27045888->27049983) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760769.674054:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973922 is committed 00002000:00010000:0.0:1789760769.674641:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27045888 -> 27049984 00010000:00010000:0.0:1789760769.674657:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2f5b7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27045888->27049983] (req 27045888->27049983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.674669:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2f5b7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27045888->27049983] (req 27045888->27049983) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.674678:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2f5b7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27045888->27049983] (req 27045888->27049983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.674684:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760769.674688:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2f5b7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27045888->27049983] (req 27045888->27049983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.674727:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2f5b7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27045888->27049983] (req 27045888->27049983) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.674747:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2f5b7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27045888->27049983] (req 27045888->27049983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.674755:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2f5b7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27045888->27049983] (req 27045888->27049983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760769.674859:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4039f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512707712:12345-192.168.203.56@tcp:4:dd.0 Request processed in 22209us (22781us total) trans 4294973922 rc 0/0 00000100:00100000:0.0:1789760769.674869:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6633 00000100:00100000:0.0:1789760769.674879:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 749187681704 time 749216934682 state 0x2 ref 0 00000100:00100000:0.0:1789760769.679150:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760769.679551:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512707840 00000100:00100000:2.0:1789760769.679572:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512707840 00000100:00100000:2.0:1789760769.679650:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 749187681704 time 749216934682 state 0x0 ref 1 00000100:00100000:2.0:1789760769.679666:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 749217195098 time 749216934682 state 0x1 ref 1 00000100:00100000:2.0:1789760769.679718:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 749217195098 time 749243996014 state 0x0 ref 1 00000100:00100000:2.0:1789760769.679731:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febfb895880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512707840:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760769.679786:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27049984, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0xd007 00010000:00010000:2.0:1789760769.679797:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2f5be lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.679814:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=749s v=(0 0) 00010000:00010000:2.0:1789760769.679818:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760769.679825:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2f5be lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27049984->27054079] (req 27049984->27054079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.679846:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2f5be lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27049984->27054079] (req 27049984->27054079) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760769.712976:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973923 is committed 00002000:00010000:0.0:1789760769.712998:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27049984 -> 27054080 00010000:00010000:0.0:1789760769.713015:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2f5be lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27049984->27054079] (req 27049984->27054079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.713028:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2f5be lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27049984->27054079] (req 27049984->27054079) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.713038:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2f5be lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27049984->27054079] (req 27049984->27054079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.713046:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760769.713050:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2f5be lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27049984->27054079] (req 27049984->27054079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.713060:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2f5be lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27049984->27054079] (req 27049984->27054079) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.713083:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2f5be lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27049984->27054079] (req 27049984->27054079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.713091:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2f5be lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27049984->27054079] (req 27049984->27054079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760769.713361:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febfb895880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512707840:12345-192.168.203.56@tcp:4:dd.0 Request processed in 33626us (34216us total) trans 4294973923 rc 0/0 00000100:00100000:0.0:1789760769.713372:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6634 00000100:00100000:0.0:1789760769.713383:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 749217195098 time 749243996014 state 0x2 ref 0 00000100:00100000:3.0:1789760769.718102:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760769.718294:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512707968 00000100:00100000:2.0:1789760769.718303:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512707968 00000100:00100000:2.0:1789760769.718349:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 749217195098 time 749243996014 state 0x0 ref 1 00000100:00100000:2.0:1789760769.718363:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 749244256430 time 749243996014 state 0x1 ref 1 00000100:00100000:2.0:1789760769.718391:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 749244256430 time 749282691356 state 0x0 ref 1 00000100:00100000:2.0:1789760769.718401:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5305880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512707968:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760769.718457:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27054080, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0xd007 00010000:00010000:2.0:1789760769.718466:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2f5c5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.718481:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=749s v=(0 0) 00010000:00010000:2.0:1789760769.718486:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760769.718491:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2f5c5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27054080->27058175] (req 27054080->27058175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.718511:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2f5c5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27054080->27058175] (req 27054080->27058175) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760769.755166:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27054080 -> 27058176 00010000:00010000:2.0:1789760769.755185:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2f5c5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27054080->27058175] (req 27054080->27058175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760769.755337:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973924 is committed 00010000:00010000:2.0:1789760769.755391:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2f5c5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27054080->27058175] (req 27054080->27058175) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.755399:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2f5c5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27054080->27058175] (req 27054080->27058175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.755406:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760769.755410:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2f5c5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27054080->27058175] (req 27054080->27058175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.755419:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2f5c5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27054080->27058175] (req 27054080->27058175) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.755439:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2f5c5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27054080->27058175] (req 27054080->27058175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.755446:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2f5c5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27054080->27058175] (req 27054080->27058175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760769.756418:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5305880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512707968:12345-192.168.203.56@tcp:4:dd.0 Request processed in 38010us (38322us total) trans 4294973924 rc 0/0 00000100:00100000:2.0:1789760769.756429:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6635 00000100:00100000:2.0:1789760769.756440:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 749244256430 time 749282691356 state 0x2 ref 0 00000100:00100000:1.0:1789760769.764647:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760769.765086:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512708096 00000100:00100000:0.0:1789760769.765100:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512708096 00000100:00100000:0.0:1789760769.765169:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 749244256430 time 749282691356 state 0x0 ref 1 00000100:00100000:0.0:1789760769.765186:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 749282951772 time 749282691356 state 0x1 ref 1 00000100:00100000:0.0:1789760769.765356:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 749282951772 time 749329654595 state 0x0 ref 1 00000100:00100000:0.0:1789760769.765372:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4038000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512708096:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760769.765425:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27058176, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0xd007 00010000:00010000:0.0:1789760769.765435:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2f5cc lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.765451:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=749s v=(0 0) 00010000:00010000:0.0:1789760769.765457:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760769.765463:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2f5cc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27058176->27062271] (req 27058176->27062271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.765477:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2f5cc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27058176->27062271] (req 27058176->27062271) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760769.810675:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27058176 -> 27062272 00010000:00010000:1.0:1789760769.810724:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2f5cc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27058176->27062271] (req 27058176->27062271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.810739:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2f5cc lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27058176->27062271] (req 27058176->27062271) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.810755:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2f5cc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27058176->27062271] (req 27058176->27062271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.810763:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760769.810768:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2f5cc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27058176->27062271] (req 27058176->27062271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.810776:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2f5cc lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27058176->27062271] (req 27058176->27062271) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.810808:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2f5cc lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27058176->27062271] (req 27058176->27062271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.810816:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2f5cc lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27058176->27062271] (req 27058176->27062271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760769.810910:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4038000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2358:x1876699512708096:12345-192.168.203.56@tcp:4:dd.0 Request processed in 45536us (46266us total) trans 4294973925 rc 0/0 00000100:00100000:1.0:1789760769.810920:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6636 00000100:00100000:1.0:1789760769.810931:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 749282951772 time 749329654595 state 0x2 ref 0 00000001:00080000:3.0:1789760769.811184:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973925 is committed 00000100:00100000:1.0:1789760769.815053:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760769.815142:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512708224 00000100:00100000:0.0:1789760769.815153:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512708224 00000100:00100000:0.0:1789760769.815210:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 749282951772 time 749329654595 state 0x0 ref 1 00000100:00100000:0.0:1789760769.815226:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 749329915011 time 749329654595 state 0x1 ref 1 00000100:00100000:0.0:1789760769.815258:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 749329915011 time 749379558156 state 0x0 ref 1 00000100:00100000:0.0:1789760769.815271:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac403bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512708224:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760769.815320:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27062272, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0xd007 00010000:00010000:0.0:1789760769.815329:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2f5d3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.815346:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=749s v=(0 0) 00010000:00010000:0.0:1789760769.815350:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760769.815356:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2f5d3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27062272->27066367] (req 27062272->27066367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.815370:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2f5d3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27062272->27066367] (req 27062272->27066367) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760769.868312:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973926 is committed 00002000:00010000:1.0:1789760769.868463:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27062272 -> 27066368 00010000:00010000:1.0:1789760769.868478:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2f5d3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27062272->27066367] (req 27062272->27066367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.868495:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2f5d3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27062272->27066367] (req 27062272->27066367) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.868504:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2f5d3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27062272->27066367] (req 27062272->27066367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.868510:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760769.868515:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2f5d3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27062272->27066367] (req 27062272->27066367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.868524:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2f5d3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27062272->27066367] (req 27062272->27066367) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.868543:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2f5d3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27062272->27066367] (req 27062272->27066367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.868550:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2f5d3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27062272->27066367] (req 27062272->27066367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760769.868786:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac403bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512708224:12345-192.168.203.56@tcp:4:dd.0 Request processed in 53512us (53830us total) trans 4294973926 rc 0/0 00000100:00100000:1.0:1789760769.868855:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6637 00000100:00100000:1.0:1789760769.868866:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 749329915011 time 749379558156 state 0x2 ref 0 00000100:00100000:0.0:1789760769.874996:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760769.875340:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512708352 00000100:00100000:2.0:1789760769.877301:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512708352 00000100:00100000:2.0:1789760769.877666:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 749329915011 time 749379558156 state 0x0 ref 1 00000100:00100000:2.0:1789760769.877685:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 749379818572 time 749379558156 state 0x1 ref 1 00000100:00100000:2.0:1789760769.877741:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 749379818572 time 749442040749 state 0x0 ref 1 00000100:00100000:2.0:1789760769.877756:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febea4b7850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512708352:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760769.877801:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27066368, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0xd007 00010000:00010000:2.0:1789760769.877810:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2f5da lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.877827:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=749s v=(0 0) 00010000:00010000:2.0:1789760769.877832:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760769.877838:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2f5da lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27066368->27070463] (req 27066368->27070463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.877851:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2f5da lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27066368->27070463] (req 27066368->27070463) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760769.932193:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973927 is committed 00002000:00010000:3.0:1789760769.932368:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27066368 -> 27070464 00010000:00010000:3.0:1789760769.932382:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2f5da lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27066368->27070463] (req 27066368->27070463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.932395:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2f5da lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27066368->27070463] (req 27066368->27070463) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.932409:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2f5da lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27066368->27070463] (req 27066368->27070463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.932416:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760769.932420:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2f5da lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27066368->27070463] (req 27066368->27070463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.932428:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2f5da lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27066368->27070463] (req 27066368->27070463) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.932446:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2f5da lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27066368->27070463] (req 27066368->27070463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.932453:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2f5da lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27066368->27070463] (req 27066368->27070463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760769.932920:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febea4b7850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512708352:12345-192.168.203.56@tcp:4:dd.0 Request processed in 55161us (57927us total) trans 4294973927 rc 0/0 00000100:00100000:3.0:1789760769.932930:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6638 00000100:00100000:3.0:1789760769.932939:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 749379818572 time 749442040749 state 0x2 ref 0 00000100:00100000:0.0:1789760769.936024:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760769.936111:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512708480 00000100:00100000:1.0:1789760769.936131:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512708480 00000100:00100000:1.0:1789760769.936184:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 749379818572 time 749442040749 state 0x0 ref 1 00000100:00100000:1.0:1789760769.936198:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 749442301165 time 749442040749 state 0x1 ref 1 00000100:00100000:1.0:1789760769.936235:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 749442301165 time 749500534161 state 0x0 ref 1 00000100:00100000:1.0:1789760769.936248:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febfb897100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512708480:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760769.936297:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27070464, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0xd007 00010000:00010000:1.0:1789760769.936306:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2f5e1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.936322:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=749s v=(0 0) 00010000:00010000:1.0:1789760769.936327:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760769.936333:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2f5e1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27070464->27074559] (req 27070464->27074559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.936348:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2f5e1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27070464->27074559] (req 27070464->27074559) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760769.968200:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27070464 -> 27074560 00000001:00080000:3.0:1789760769.968300:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973928 is committed 00010000:00010000:1.0:1789760769.968362:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2f5e1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27070464->27074559] (req 27070464->27074559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.968375:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2f5e1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27070464->27074559] (req 27070464->27074559) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.968390:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2f5e1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27070464->27074559] (req 27070464->27074559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.968398:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760769.968404:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2f5e1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27070464->27074559] (req 27070464->27074559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.968413:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2f5e1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27070464->27074559] (req 27070464->27074559) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.968433:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2f5e1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27070464->27074559] (req 27070464->27074559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.968441:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2f5e1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27070464->27074559] (req 27070464->27074559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760769.968677:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febfb897100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512708480:12345-192.168.203.56@tcp:4:dd.0 Request processed in 32426us (32657us total) trans 4294973928 rc 0/0 00000100:00100000:1.0:1789760769.968688:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6639 00000100:00100000:1.0:1789760769.968738:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 749442301165 time 749500534161 state 0x2 ref 0 00000100:00100000:1.0:1789760769.972659:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760769.972868:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512708608 00000100:00100000:0.0:1789760769.972879:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512708608 00000100:00100000:0.0:1789760769.972933:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 749442301165 time 749500534161 state 0x0 ref 1 00000100:00100000:0.0:1789760769.972947:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 749500794577 time 749500534161 state 0x1 ref 1 00000100:00100000:0.0:1789760769.973064:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 749500794577 time 749537362597 state 0x0 ref 1 00000100:00100000:0.0:1789760769.973078:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac403b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512708608:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760769.973135:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27074560, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0xd007 00010000:00010000:0.0:1789760769.973144:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2f5e8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.973160:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=749s v=(0 0) 00010000:00010000:0.0:1789760769.973164:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760769.973171:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2f5e8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27074560->27078655] (req 27074560->27078655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.973184:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2f5e8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27074560->27078655] (req 27074560->27078655) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760770.011060:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973929 is committed 00002000:00010000:0.0:1789760770.011175:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27074560 -> 27078656 00010000:00010000:0.0:1789760770.011190:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2f5e8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27074560->27078655] (req 27074560->27078655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.011201:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2f5e8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27074560->27078655] (req 27074560->27078655) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.011209:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2f5e8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27074560->27078655] (req 27074560->27078655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.011216:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760770.011220:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2f5e8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27074560->27078655] (req 27074560->27078655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.011230:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2f5e8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27074560->27078655] (req 27074560->27078655) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.011248:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2f5e8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27074560->27078655] (req 27074560->27078655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.011256:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2f5e8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27074560->27078655] (req 27074560->27078655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760770.011357:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac403b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512708608:12345-192.168.203.56@tcp:4:dd.0 Request processed in 38277us (38702us total) trans 4294973929 rc 0/0 00000100:00100000:0.0:1789760770.011367:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6640 00000100:00100000:0.0:1789760770.011377:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 749500794577 time 749537362597 state 0x2 ref 0 00000100:00100000:0.0:1789760770.014311:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760770.014394:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512708736 00000100:00100000:3.0:1789760770.014405:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512708736 00000100:00100000:3.0:1789760770.014460:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 749500794577 time 749537362597 state 0x0 ref 1 00000100:00100000:3.0:1789760770.014475:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 749537623013 time 749537362597 state 0x1 ref 1 00000100:00100000:3.0:1789760770.014595:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 749537623013 time 749578893843 state 0x0 ref 1 00000100:00100000:3.0:1789760770.014609:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febfb895880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512708736:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760770.014655:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27078656, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0xd007 00010000:00010000:3.0:1789760770.014665:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2f5ef lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.014681:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=749s v=(0 0) 00010000:00010000:3.0:1789760770.014686:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760770.014692:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2f5ef lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27078656->27082751] (req 27078656->27082751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.014734:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2f5ef lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27078656->27082751] (req 27078656->27082751) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760770.057592:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973930 is committed 00002000:00010000:3.0:1789760770.058010:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27078656 -> 27082752 00010000:00010000:3.0:1789760770.058025:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2f5ef lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27078656->27082751] (req 27078656->27082751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.058037:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2f5ef lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27078656->27082751] (req 27078656->27082751) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.058047:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2f5ef lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27078656->27082751] (req 27078656->27082751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.058056:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760770.058062:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2f5ef lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27078656->27082751] (req 27078656->27082751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.058071:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2f5ef lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27078656->27082751] (req 27078656->27082751) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.058089:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2f5ef lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27078656->27082751] (req 27078656->27082751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.058096:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2f5ef lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27078656->27082751] (req 27078656->27082751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760770.058230:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febfb895880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512708736:12345-192.168.203.56@tcp:4:dd.0 Request processed in 43618us (43923us total) trans 4294973930 rc 0/0 00000100:00100000:3.0:1789760770.058241:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6641 00000100:00100000:3.0:1789760770.058251:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 749537623013 time 749578893843 state 0x2 ref 0 00000100:00100000:1.0:1789760770.061151:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760770.061966:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512708864 00000100:00100000:0.0:1789760770.061977:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512708864 00000100:00100000:0.0:1789760770.062031:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 749537623013 time 749578893843 state 0x0 ref 1 00000100:00100000:0.0:1789760770.062046:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 749579154259 time 749578893843 state 0x1 ref 1 00000100:00100000:0.0:1789760770.062076:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 749579154259 time 749626375423 state 0x0 ref 1 00000100:00100000:0.0:1789760770.062087:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac403b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512708864:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760770.062146:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27082752, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0xd007 00010000:00010000:0.0:1789760770.062155:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2f5f6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.062171:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=749s v=(0 0) 00010000:00010000:0.0:1789760770.062175:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760770.062180:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2f5f6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27082752->27086847] (req 27082752->27086847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.062194:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2f5f6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27082752->27086847] (req 27082752->27086847) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760770.101113:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27082752 -> 27086848 00010000:00010000:0.0:1789760770.101142:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2f5f6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27082752->27086847] (req 27082752->27086847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.101163:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2f5f6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27082752->27086847] (req 27082752->27086847) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.101172:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2f5f6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27082752->27086847] (req 27082752->27086847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.101179:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760770.101184:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2f5f6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27082752->27086847] (req 27082752->27086847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.101194:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2f5f6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27082752->27086847] (req 27082752->27086847) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.101214:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2f5f6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27082752->27086847] (req 27082752->27086847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.101222:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2f5f6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27082752->27086847] (req 27082752->27086847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760770.101328:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac403b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2358:x1876699512708864:12345-192.168.203.56@tcp:4:dd.0 Request processed in 39229us (40173us total) trans 4294973931 rc 0/0 00000100:00100000:0.0:1789760770.101338:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6642 00000100:00100000:0.0:1789760770.101347:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 749579154259 time 749626375423 state 0x2 ref 0 00000001:00080000:3.0:1789760770.103505:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973931 is committed 00000100:00100000:1.0:1789760770.104577:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760770.104630:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512708992 00000100:00100000:0.0:1789760770.104645:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512708992 00000100:00100000:0.0:1789760770.104719:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 749579154259 time 749626375423 state 0x0 ref 1 00000100:00100000:0.0:1789760770.104736:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 749626635839 time 749626375423 state 0x1 ref 1 00000100:00100000:0.0:1789760770.104765:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 749626635839 time 749669065047 state 0x0 ref 1 00000100:00100000:0.0:1789760770.104776:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4038380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512708992:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760770.104831:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27086848, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0xd007 00010000:00010000:0.0:1789760770.104844:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2f5fd lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.104859:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=749s v=(0 0) 00010000:00010000:0.0:1789760770.104864:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760770.104871:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2f5fd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27086848->27090943] (req 27086848->27090943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.104884:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2f5fd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27086848->27090943] (req 27086848->27090943) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760770.139977:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973932 is committed 00002000:00010000:3.0:1789760770.140131:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27086848 -> 27090944 00010000:00010000:3.0:1789760770.140145:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2f5fd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27086848->27090943] (req 27086848->27090943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.140158:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2f5fd lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27086848->27090943] (req 27086848->27090943) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.140174:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2f5fd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27086848->27090943] (req 27086848->27090943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.140181:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760770.140186:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2f5fd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27086848->27090943] (req 27086848->27090943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.140197:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2f5fd lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27086848->27090943] (req 27086848->27090943) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.140215:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2f5fd lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27086848->27090943] (req 27086848->27090943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.140223:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2f5fd lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27086848->27090943] (req 27086848->27090943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760770.140314:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4038380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512708992:12345-192.168.203.56@tcp:4:dd.0 Request processed in 35534us (35739us total) trans 4294973932 rc 0/0 00000100:00100000:3.0:1789760770.140324:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6643 00000100:00100000:3.0:1789760770.140334:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 749626635839 time 749669065047 state 0x2 ref 0 00000100:00100000:0.0:1789760770.145342:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760770.145731:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512709120 00000100:00100000:3.0:1789760770.145741:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512709120 00000100:00100000:3.0:1789760770.145858:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 749626635839 time 749669065047 state 0x0 ref 1 00000100:00100000:3.0:1789760770.145875:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 749669325463 time 749669065047 state 0x1 ref 1 00000100:00100000:3.0:1789760770.145907:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 749669325463 time 749710206434 state 0x0 ref 1 00000100:00100000:3.0:1789760770.145919:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4076a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512709120:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760770.145962:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27090944, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0xd007 00010000:00010000:3.0:1789760770.145970:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2f604 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.145984:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=749s v=(0 0) 00010000:00010000:3.0:1789760770.145988:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760770.145994:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2f604 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27090944->27095039] (req 27090944->27095039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.146007:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2f604 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27090944->27095039] (req 27090944->27095039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760770.188687:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973933 is committed 00002000:00010000:1.0:1789760770.189171:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27090944 -> 27095040 00010000:00010000:1.0:1789760770.189187:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2f604 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27090944->27095039] (req 27090944->27095039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.189199:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2f604 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27090944->27095039] (req 27090944->27095039) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.189208:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2f604 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27090944->27095039] (req 27090944->27095039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.189216:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760770.189223:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2f604 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27090944->27095039] (req 27090944->27095039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.189233:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2f604 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27090944->27095039] (req 27090944->27095039) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.189252:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2f604 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27090944->27095039] (req 27090944->27095039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.189259:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2f604 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27090944->27095039] (req 27090944->27095039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760770.189402:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4076a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512709120:12345-192.168.203.56@tcp:4:dd.0 Request processed in 43478us (44062us total) trans 4294973933 rc 0/0 00000100:00100000:1.0:1789760770.189414:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6644 00000100:00100000:1.0:1789760770.189423:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 749669325463 time 749710206434 state 0x2 ref 0 00000100:00100000:3.0:1789760770.193892:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760770.194328:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512709248 00000100:00100000:0.0:1789760770.194339:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512709248 00000100:00100000:0.0:1789760770.194398:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 749669325463 time 749710206434 state 0x0 ref 1 00000100:00100000:0.0:1789760770.194414:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 749710466850 time 749710206434 state 0x1 ref 1 00000100:00100000:0.0:1789760770.194453:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 749710466850 time 749758752228 state 0x0 ref 1 00000100:00100000:0.0:1789760770.194467:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5305500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512709248:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760770.194524:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27095040, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0xd007 00010000:00010000:0.0:1789760770.194534:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2f60b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.194551:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=749s v=(0 0) 00010000:00010000:0.0:1789760770.194555:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760770.194562:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2f60b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27095040->27099135] (req 27095040->27099135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.194577:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2f60b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27095040->27099135] (req 27095040->27099135) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760770.260558:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973934 is committed 00002000:00010000:3.0:1789760770.260672:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27095040 -> 27099136 00010000:00010000:3.0:1789760770.260689:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2f60b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27095040->27099135] (req 27095040->27099135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.260723:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2f60b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27095040->27099135] (req 27095040->27099135) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.260732:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2f60b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27095040->27099135] (req 27095040->27099135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.260746:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760770.260751:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2f60b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27095040->27099135] (req 27095040->27099135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.260760:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2f60b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27095040->27099135] (req 27095040->27099135) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.260779:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2f60b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27095040->27099135] (req 27095040->27099135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.260786:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2f60b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27095040->27099135] (req 27095040->27099135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760770.260903:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5305500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512709248:12345-192.168.203.56@tcp:4:dd.0 Request processed in 66434us (67017us total) trans 4294973934 rc 0/0 00000100:00100000:3.0:1789760770.260913:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6645 00000100:00100000:3.0:1789760770.260922:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 749710466850 time 749758752228 state 0x2 ref 0 00000100:00100000:0.0:1789760770.264682:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760770.265219:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512709376 00000100:00100000:3.0:1789760770.265228:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512709376 00000100:00100000:3.0:1789760770.265283:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 749710466850 time 749758752228 state 0x0 ref 1 00000100:00100000:3.0:1789760770.265298:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 749759012644 time 749758752228 state 0x1 ref 1 00000100:00100000:3.0:1789760770.265355:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 749759012644 time 749829654196 state 0x0 ref 1 00000100:00100000:3.0:1789760770.265371:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4074e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512709376:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760770.265417:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27099136, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0xd007 00010000:00010000:3.0:1789760770.265426:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2f612 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.265442:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=749s v=(0 0) 00010000:00010000:3.0:1789760770.265447:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760770.265453:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2f612 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27099136->27103231] (req 27099136->27103231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.265466:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2f612 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27099136->27103231] (req 27099136->27103231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760770.305981:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973935 is committed 00002000:00010000:3.0:1789760770.306086:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27099136 -> 27103232 00010000:00010000:3.0:1789760770.306101:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2f612 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27099136->27103231] (req 27099136->27103231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.306115:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2f612 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27099136->27103231] (req 27099136->27103231) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.306135:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2f612 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27099136->27103231] (req 27099136->27103231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.306143:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760770.306149:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2f612 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27099136->27103231] (req 27099136->27103231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.306159:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2f612 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27099136->27103231] (req 27099136->27103231) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.306179:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2f612 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27099136->27103231] (req 27099136->27103231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.306187:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2f612 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27099136->27103231] (req 27099136->27103231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760770.306368:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4074e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512709376:12345-192.168.203.56@tcp:4:dd.0 Request processed in 40993us (41689us total) trans 4294973935 rc 0/0 00000100:00100000:3.0:1789760770.306379:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6646 00000100:00100000:3.0:1789760770.306389:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 749759012644 time 749829654196 state 0x2 ref 0 00000100:00100000:1.0:1789760770.310189:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760770.310228:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760770.310270:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760770.310293:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760770.310317:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512709504 00000100:00100000:1.0:1789760770.310326:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512709504 00000100:00100000:1.0:1789760770.310372:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 408 00000100:00100000:1.0:1789760770.310381:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febe00c6050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:e6cf7267-9b19-462a-a263-14115df15dab+13:2358:x1876699512709504:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:1.0:1789760770.310445:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febe00c6050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:e6cf7267-9b19-462a-a263-14115df15dab+13:2358:x1876699512709504:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 65us (175us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760770.310453:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 408 00000100:00100000:1.0:1789760770.310487:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512709632 00000100:00100000:1.0:1789760770.310491:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512709632 00000100:00100000:1.0:1789760770.310535:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512709760 00000100:00100000:1.0:1789760770.310538:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512709760 00000100:00100000:1.0:1789760770.310560:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 499 00000100:00100000:1.0:1789760770.310575:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4038a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+8:2358:x1876699512709632:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:1.0:1789760770.310607:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4038a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+8:2358:x1876699512709632:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 33us (379us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760770.310614:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 499 00000100:00100000:1.0:1789760770.310625:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 500 00000100:00100000:1.0:1789760770.310631:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac403ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:d2c40150-e97b-40da-952b-cb9f18bf7e96+5:2358:x1876699512709760:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:1.0:1789760770.310653:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac403ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:d2c40150-e97b-40da-952b-cb9f18bf7e96+5:2358:x1876699512709760:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 22us (360us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760770.310659:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 500 00000100:00080000:1.0:1789760770.310678:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789760770.311472:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512709888 00000100:00100000:3.0:1789760770.311479:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512709888 00000100:00100000:3.0:1789760770.311531:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 749759012644 time 749829654196 state 0x0 ref 1 00000100:00100000:3.0:1789760770.311547:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 749829914612 time 749829654196 state 0x1 ref 1 00000100:00100000:3.0:1789760770.311576:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 749829914612 time 749875875939 state 0x0 ref 1 00000100:00100000:3.0:1789760770.311589:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4039500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512709888:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760770.311634:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27103232, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0xd007 00010000:00010000:3.0:1789760770.311643:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2f619 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.311660:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=749s v=(0 0) 00010000:00010000:3.0:1789760770.311664:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760770.311671:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2f619 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27103232->27107327] (req 27103232->27107327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.311686:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2f619 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27103232->27107327] (req 27103232->27107327) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760770.351207:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27103232 -> 27107328 00010000:00010000:3.0:1789760770.351227:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2f619 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27103232->27107327] (req 27103232->27107327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.351241:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2f619 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27103232->27107327] (req 27103232->27107327) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.351250:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2f619 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27103232->27107327] (req 27103232->27107327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.351257:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760770.351262:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2f619 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27103232->27107327] (req 27103232->27107327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.351271:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2f619 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27103232->27107327] (req 27103232->27107327) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.351291:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2f619 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27103232->27107327] (req 27103232->27107327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.351300:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2f619 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27103232->27107327] (req 27103232->27107327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760770.351406:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4039500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2359:x1876699512709888:12345-192.168.203.56@tcp:4:dd.0 Request processed in 39813us (41220us total) trans 4294973936 rc 0/0 00000100:00100000:3.0:1789760770.351417:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6647 00000100:00100000:3.0:1789760770.351427:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 749829914612 time 749875875939 state 0x2 ref 0 00000001:00080000:1.0:1789760770.352430:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973936 is committed 00000100:00100000:1.0:1789760770.355288:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760770.355361:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512710016 00000100:00100000:3.0:1789760770.355371:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512710016 00000100:00100000:3.0:1789760770.355421:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 749829914612 time 749875875939 state 0x0 ref 1 00000100:00100000:3.0:1789760770.355435:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 749876136355 time 749875875939 state 0x1 ref 1 00000100:00100000:3.0:1789760770.355463:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 749876136355 time 749919763393 state 0x0 ref 1 00000100:00100000:3.0:1789760770.355475:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac403a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512710016:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760770.355519:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27107328, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0xd007 00010000:00010000:3.0:1789760770.355528:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2f620 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.355544:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=749s v=(0 0) 00010000:00010000:3.0:1789760770.355548:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760770.355554:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2f620 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27107328->27111423] (req 27107328->27111423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.355574:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2f620 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27107328->27111423] (req 27107328->27111423) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760770.394583:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973937 is committed 00002000:00010000:0.0:1789760770.394832:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27107328 -> 27111424 00010000:00010000:0.0:1789760770.394848:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2f620 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27107328->27111423] (req 27107328->27111423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.394861:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2f620 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27107328->27111423] (req 27107328->27111423) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.394869:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2f620 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27107328->27111423] (req 27107328->27111423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.394877:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760770.394882:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2f620 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27107328->27111423] (req 27107328->27111423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.394892:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2f620 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27107328->27111423] (req 27107328->27111423) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.394910:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2f620 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27107328->27111423] (req 27107328->27111423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.394917:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2f620 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27107328->27111423] (req 27107328->27111423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760770.395042:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac403a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512710016:12345-192.168.203.56@tcp:4:dd.0 Request processed in 39563us (39757us total) trans 4294973937 rc 0/0 00000100:00100000:0.0:1789760770.395053:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6648 00000100:00100000:0.0:1789760770.395062:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 749876136355 time 749919763393 state 0x2 ref 0 00000100:00100000:0.0:1789760770.398392:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760770.398474:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512710144 00000100:00100000:3.0:1789760770.398482:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512710144 00000100:00100000:3.0:1789760770.398591:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 749876136355 time 749919763393 state 0x0 ref 1 00000100:00100000:3.0:1789760770.398607:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 749920023809 time 749919763393 state 0x1 ref 1 00000100:00100000:3.0:1789760770.398636:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 749920023809 time 749962936561 state 0x0 ref 1 00000100:00100000:3.0:1789760770.398649:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4075500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512710144:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760770.398849:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27111424, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0xd007 00010000:00010000:3.0:1789760770.398859:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2f627 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.398874:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=749s v=(0 0) 00010000:00010000:3.0:1789760770.398878:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760770.398884:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2f627 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27111424->27115519] (req 27111424->27115519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.398899:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2f627 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27111424->27115519] (req 27111424->27115519) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.418851:0:5835:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:2.0:1789760770.428417:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973938 is committed 00002000:00010000:2.0:1789760770.428760:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27111424 -> 27115520 00010000:00010000:2.0:1789760770.428775:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2f627 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27111424->27115519] (req 27111424->27115519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.428788:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2f627 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27111424->27115519] (req 27111424->27115519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.428926:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2f627 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27111424->27115519] (req 27111424->27115519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.428936:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760770.428940:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2f627 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27111424->27115519] (req 27111424->27115519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.428950:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2f627 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27111424->27115519] (req 27111424->27115519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.428970:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2f627 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27111424->27115519] (req 27111424->27115519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.428978:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2f627 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27111424->27115519] (req 27111424->27115519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760770.429103:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4075500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512710144:12345-192.168.203.56@tcp:4:dd.0 Request processed in 30452us (30715us total) trans 4294973938 rc 0/0 00000100:00100000:2.0:1789760770.429114:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6649 00000100:00100000:2.0:1789760770.429134:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 749920023809 time 749962936561 state 0x2 ref 0 00000100:00100000:1.0:1789760770.432946:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760770.433064:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512710272 00000100:00100000:3.0:1789760770.433075:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512710272 00000100:00100000:3.0:1789760770.433140:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 749920023809 time 749962936561 state 0x0 ref 1 00000100:00100000:3.0:1789760770.433154:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 749963196977 time 749962936561 state 0x1 ref 1 00000100:00100000:3.0:1789760770.433211:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 749963196977 time 749997510135 state 0x0 ref 1 00000100:00100000:3.0:1789760770.433224:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4039c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512710272:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760770.433270:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27115520, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0xd007 00010000:00010000:3.0:1789760770.433279:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2f62e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.433294:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=749s v=(0 0) 00010000:00010000:3.0:1789760770.433299:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760770.433305:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2f62e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27115520->27119615] (req 27115520->27119615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.433319:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2f62e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27115520->27119615] (req 27115520->27119615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760770.469249:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27115520 -> 27119616 00010000:00010000:2.0:1789760770.469269:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2f62e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27115520->27119615] (req 27115520->27119615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.469282:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2f62e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27115520->27119615] (req 27115520->27119615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.469292:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2f62e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27115520->27119615] (req 27115520->27119615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.469300:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760770.469305:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2f62e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27115520->27119615] (req 27115520->27119615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.469315:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2f62e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27115520->27119615] (req 27115520->27119615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.469336:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2f62e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27115520->27119615] (req 27115520->27119615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.469344:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2f62e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27115520->27119615] (req 27115520->27119615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760770.469439:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4039c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2358:x1876699512710272:12345-192.168.203.56@tcp:4:dd.0 Request processed in 36212us (36497us total) trans 4294973939 rc 0/0 00000100:00100000:2.0:1789760770.469449:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6650 00000100:00100000:2.0:1789760770.469461:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 749963196977 time 749997510135 state 0x2 ref 0 00000001:00080000:2.0:1789760770.469502:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973939 is committed 00000100:00100000:1.0:1789760770.472316:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760770.472437:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512710400 00000100:00100000:0.0:1789760770.472447:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512710400 00000100:00100000:0.0:1789760770.472499:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 749963196977 time 749997510135 state 0x0 ref 1 00000100:00100000:0.0:1789760770.472582:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 749997770551 time 749997510135 state 0x1 ref 1 00000100:00100000:0.0:1789760770.472646:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 749997770551 time 750036944627 state 0x0 ref 1 00000100:00100000:0.0:1789760770.472659:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4038e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512710400:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760770.472733:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27119616, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0xd007 00010000:00010000:0.0:1789760770.472743:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f635 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.472758:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=750s v=(0 0) 00010000:00010000:0.0:1789760770.472763:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760770.472769:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f635 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27119616->27123711] (req 27119616->27123711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.472784:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f635 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27119616->27123711] (req 27119616->27123711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760770.504918:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973940 is committed 00002000:00010000:2.0:1789760770.504966:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27119616 -> 27123712 00010000:00010000:2.0:1789760770.504981:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f635 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27119616->27123711] (req 27119616->27123711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.504991:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f635 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27119616->27123711] (req 27119616->27123711) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.505001:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f635 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27119616->27123711] (req 27119616->27123711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.505008:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760770.505013:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f635 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27119616->27123711] (req 27119616->27123711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.505022:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f635 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27119616->27123711] (req 27119616->27123711) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.505042:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f635 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27119616->27123711] (req 27119616->27123711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.505047:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f635 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27119616->27123711] (req 27119616->27123711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760770.505127:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4038e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512710400:12345-192.168.203.56@tcp:4:dd.0 Request processed in 32459us (32808us total) trans 4294973940 rc 0/0 00000100:00100000:2.0:1789760770.505141:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6651 00000100:00100000:2.0:1789760770.505150:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 749997770551 time 750036944627 state 0x2 ref 0 00000100:00100000:0.0:1789760770.510789:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760770.510871:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512710528 00000100:00100000:2.0:1789760770.510882:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512710528 00000100:00100000:2.0:1789760770.510931:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 749997770551 time 750036944627 state 0x0 ref 1 00000100:00100000:2.0:1789760770.510948:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 750037205043 time 750036944627 state 0x1 ref 1 00000100:00100000:2.0:1789760770.510969:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 750037205043 time 750075268591 state 0x0 ref 1 00000100:00100000:2.0:1789760770.510981:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4076300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512710528:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760770.511028:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27123712, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0xd007 00010000:00010000:2.0:1789760770.511036:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2f63c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.511052:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=750s v=(0 0) 00010000:00010000:2.0:1789760770.511056:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760770.511063:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2f63c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27123712->27127807] (req 27123712->27127807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.511076:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2f63c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27123712->27127807] (req 27123712->27127807) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760770.540641:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973941 is committed 00002000:00010000:2.0:1789760770.540910:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27123712 -> 27127808 00002000:00010000:2.0:1789760770.540915:0:6708:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760769 -> 1789760770 00002000:00010000:2.0:1789760770.540918:0:6708:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760769 -> 1789760770 00010000:00010000:2.0:1789760770.540932:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2f63c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27123712->27127807] (req 27123712->27127807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.540943:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2f63c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27123712->27127807] (req 27123712->27127807) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.540951:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2f63c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27123712->27127807] (req 27123712->27127807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.540957:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760770.540961:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2f63c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27123712->27127807] (req 27123712->27127807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.540970:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2f63c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27123712->27127807] (req 27123712->27127807) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.540987:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2f63c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27123712->27127807] (req 27123712->27127807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.540995:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2f63c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27123712->27127807] (req 27123712->27127807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760770.541128:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4076300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512710528:12345-192.168.203.56@tcp:4:dd.0 Request processed in 30137us (30336us total) trans 4294973941 rc 0/0 00000100:00100000:2.0:1789760770.541139:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6652 00000100:00100000:2.0:1789760770.541148:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 750037205043 time 750075268591 state 0x2 ref 0 00000100:00100000:1.0:1789760770.545228:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760770.545312:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512710656 00000100:00100000:2.0:1789760770.545321:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512710656 00000100:00100000:2.0:1789760770.545377:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 750037205043 time 750075268591 state 0x0 ref 1 00000100:00100000:2.0:1789760770.545392:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 750075529007 time 750075268591 state 0x1 ref 1 00000100:00100000:2.0:1789760770.545554:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 750075529007 time 750109851942 state 0x0 ref 1 00000100:00100000:2.0:1789760770.545575:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac403b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512710656:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760770.545627:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27127808, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0xd007 00010000:00010000:2.0:1789760770.545636:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2f643 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.545652:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=750s v=(0 0) 00010000:00010000:2.0:1789760770.545656:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760770.545663:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2f643 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27127808->27131903] (req 27127808->27131903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.546074:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2f643 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27127808->27131903] (req 27127808->27131903) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760770.597322:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973942 is committed 00002000:00010000:1.0:1789760770.597409:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27127808 -> 27131904 00010000:00010000:1.0:1789760770.597425:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2f643 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27127808->27131903] (req 27127808->27131903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.597438:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2f643 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27127808->27131903] (req 27127808->27131903) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.597447:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2f643 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27127808->27131903] (req 27127808->27131903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.597455:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760770.597460:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2f643 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27127808->27131903] (req 27127808->27131903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.597468:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2f643 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27127808->27131903] (req 27127808->27131903) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.597487:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2f643 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27127808->27131903] (req 27127808->27131903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.597500:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2f643 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27127808->27131903] (req 27127808->27131903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760770.597604:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac403b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512710656:12345-192.168.203.56@tcp:4:dd.0 Request processed in 52027us (52381us total) trans 4294973942 rc 0/0 00000100:00100000:1.0:1789760770.597614:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6653 00000100:00100000:1.0:1789760770.597624:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 750075529007 time 750109851942 state 0x2 ref 0 00000100:00100000:1.0:1789760770.604054:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760770.604238:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512710784 00000100:00100000:2.0:1789760770.604248:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512710784 00000100:00100000:2.0:1789760770.604307:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 750075529007 time 750109851942 state 0x0 ref 1 00000100:00100000:2.0:1789760770.604324:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 750110112358 time 750109851942 state 0x1 ref 1 00000100:00100000:2.0:1789760770.604359:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 750110112358 time 750168659321 state 0x0 ref 1 00000100:00100000:2.0:1789760770.604371:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac403a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512710784:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760770.604420:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27131904, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0xd007 00010000:00010000:2.0:1789760770.604429:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2f64a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.604445:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=750s v=(0 0) 00010000:00010000:2.0:1789760770.604449:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760770.604456:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2f64a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27131904->27135999] (req 27131904->27135999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.604471:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2f64a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27131904->27135999] (req 27131904->27135999) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760770.637474:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973943 is committed 00002000:00010000:0.0:1789760770.637574:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27131904 -> 27136000 00010000:00010000:0.0:1789760770.637588:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2f64a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27131904->27135999] (req 27131904->27135999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.637601:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2f64a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27131904->27135999] (req 27131904->27135999) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.637611:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2f64a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27131904->27135999] (req 27131904->27135999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.637619:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760770.637624:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2f64a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27131904->27135999] (req 27131904->27135999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.637633:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2f64a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27131904->27135999] (req 27131904->27135999) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.637652:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2f64a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27131904->27135999] (req 27131904->27135999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.637659:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2f64a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27131904->27135999] (req 27131904->27135999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760770.637921:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac403a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512710784:12345-192.168.203.56@tcp:4:dd.0 Request processed in 33547us (33873us total) trans 4294973943 rc 0/0 00000100:00100000:0.0:1789760770.637932:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6654 00000100:00100000:0.0:1789760770.637941:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 750110112358 time 750168659321 state 0x2 ref 0 00000100:00100000:1.0:1789760770.642429:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760770.642529:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512710912 00000100:00100000:3.0:1789760770.642539:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512710912 00000100:00100000:3.0:1789760770.642593:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 750110112358 time 750168659321 state 0x0 ref 1 00000100:00100000:3.0:1789760770.642609:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 750168919737 time 750168659321 state 0x1 ref 1 00000100:00100000:3.0:1789760770.642645:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 750168919737 time 750206943585 state 0x0 ref 1 00000100:00100000:3.0:1789760770.642658:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4038700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512710912:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760770.642735:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27136000, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0xd007 00010000:00010000:3.0:1789760770.642756:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2f651 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.642773:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=750s v=(0 0) 00010000:00010000:3.0:1789760770.642778:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760770.642785:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2f651 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27136000->27140095] (req 27136000->27140095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.642999:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2f651 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27136000->27140095] (req 27136000->27140095) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760770.683074:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973944 is committed 00002000:00010000:3.0:1789760770.683301:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27136000 -> 27140096 00010000:00010000:3.0:1789760770.683318:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2f651 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27136000->27140095] (req 27136000->27140095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.683331:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2f651 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27136000->27140095] (req 27136000->27140095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.683341:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2f651 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27136000->27140095] (req 27136000->27140095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.683354:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760770.683360:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2f651 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27136000->27140095] (req 27136000->27140095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.683369:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2f651 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27136000->27140095] (req 27136000->27140095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.683387:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2f651 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27136000->27140095] (req 27136000->27140095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.683395:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2f651 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27136000->27140095] (req 27136000->27140095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760770.683958:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4038700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512710912:12345-192.168.203.56@tcp:4:dd.0 Request processed in 41297us (41532us total) trans 4294973944 rc 0/0 00000100:00100000:3.0:1789760770.683970:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6655 00000100:00100000:3.0:1789760770.683979:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 750168919737 time 750206943585 state 0x2 ref 0 00000100:00100000:3.0:1789760770.686555:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760770.686685:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512711040 00000100:00100000:1.0:1789760770.686727:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512711040 00000100:00100000:1.0:1789760770.686776:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 750168919737 time 750206943585 state 0x0 ref 1 00000100:00100000:1.0:1789760770.686791:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 750207204001 time 750206943585 state 0x1 ref 1 00000100:00100000:1.0:1789760770.686816:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 750207204001 time 750251116114 state 0x0 ref 1 00000100:00100000:1.0:1789760770.686828:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5307b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512711040:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760770.686872:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27140096, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0xd007 00010000:00010000:1.0:1789760770.686881:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2f658 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.686895:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=750s v=(0 0) 00010000:00010000:1.0:1789760770.686900:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760770.686907:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2f658 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27140096->27144191] (req 27140096->27144191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.686920:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2f658 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27140096->27144191] (req 27140096->27144191) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760770.718352:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973945 is committed 00002000:00010000:1.0:1789760770.718481:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27140096 -> 27144192 00010000:00010000:1.0:1789760770.718497:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2f658 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27140096->27144191] (req 27140096->27144191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.718576:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2f658 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27140096->27144191] (req 27140096->27144191) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.718587:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2f658 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27140096->27144191] (req 27140096->27144191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.718594:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760770.718599:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2f658 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27140096->27144191] (req 27140096->27144191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.718609:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2f658 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27140096->27144191] (req 27140096->27144191) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.718627:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2f658 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27140096->27144191] (req 27140096->27144191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.718640:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2f658 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27140096->27144191] (req 27140096->27144191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760770.718864:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5307b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512711040:12345-192.168.203.56@tcp:4:dd.0 Request processed in 32032us (32313us total) trans 4294973945 rc 0/0 00000100:00100000:1.0:1789760770.718874:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6656 00000100:00100000:1.0:1789760770.718883:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 750207204001 time 750251116114 state 0x2 ref 0 00000100:00100000:3.0:1789760770.721874:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760770.721935:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512711168 00000100:00100000:0.0:1789760770.721944:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512711168 00000100:00100000:0.0:1789760770.721993:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 750207204001 time 750251116114 state 0x0 ref 1 00000100:00100000:0.0:1789760770.722007:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 750251376530 time 750251116114 state 0x1 ref 1 00000100:00100000:0.0:1789760770.722037:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 750251376530 time 750286337514 state 0x0 ref 1 00000100:00100000:0.0:1789760770.722050:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5305f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512711168:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760770.722094:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27144192, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0xd007 00010000:00010000:0.0:1789760770.722103:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f65f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.722134:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=750s v=(0 0) 00010000:00010000:0.0:1789760770.722146:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760770.722153:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f65f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27144192->27148287] (req 27144192->27148287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.722167:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f65f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27144192->27148287] (req 27144192->27148287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760770.748513:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973946 is committed 00002000:00010000:3.0:1789760770.748640:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27144192 -> 27148288 00010000:00010000:3.0:1789760770.748656:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f65f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27144192->27148287] (req 27144192->27148287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.748676:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f65f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27144192->27148287] (req 27144192->27148287) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.748685:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f65f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27144192->27148287] (req 27144192->27148287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.748693:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760770.748721:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f65f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27144192->27148287] (req 27144192->27148287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.748733:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f65f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27144192->27148287] (req 27144192->27148287) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.748753:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f65f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27144192->27148287] (req 27144192->27148287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.748761:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f65f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27144192->27148287] (req 27144192->27148287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760770.748892:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5305f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512711168:12345-192.168.203.56@tcp:4:dd.0 Request processed in 26839us (27021us total) trans 4294973946 rc 0/0 00000100:00100000:3.0:1789760770.748903:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6657 00000100:00100000:3.0:1789760770.748913:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 750251376530 time 750286337514 state 0x2 ref 0 00000100:00100000:1.0:1789760770.751587:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760770.751676:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512711296 00000100:00100000:3.0:1789760770.751686:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512711296 00000100:00100000:3.0:1789760770.751756:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 750251376530 time 750286337514 state 0x0 ref 1 00000100:00100000:3.0:1789760770.751772:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 750286597930 time 750286337514 state 0x1 ref 1 00000100:00100000:3.0:1789760770.751878:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 750286597930 time 750316177098 state 0x0 ref 1 00000100:00100000:3.0:1789760770.751891:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4039f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512711296:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760770.751935:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27148288, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0xd007 00010000:00010000:3.0:1789760770.751943:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2f666 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.751958:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=750s v=(0 0) 00010000:00010000:3.0:1789760770.751962:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760770.751968:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2f666 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27148288->27152383] (req 27148288->27152383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.751982:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2f666 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27148288->27152383] (req 27148288->27152383) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760770.789512:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973947 is committed 00002000:00010000:2.0:1789760770.789769:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27148288 -> 27152384 00010000:00010000:2.0:1789760770.789785:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2f666 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27148288->27152383] (req 27148288->27152383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.789863:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2f666 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27148288->27152383] (req 27148288->27152383) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.789873:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2f666 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27148288->27152383] (req 27148288->27152383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.789886:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760770.789891:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2f666 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27148288->27152383] (req 27148288->27152383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.789899:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2f666 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27148288->27152383] (req 27148288->27152383) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.789917:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2f666 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27148288->27152383] (req 27148288->27152383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.789925:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2f666 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27148288->27152383] (req 27148288->27152383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760770.790018:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4039f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512711296:12345-192.168.203.56@tcp:4:dd.0 Request processed in 38123us (38434us total) trans 4294973947 rc 0/0 00000100:00100000:2.0:1789760770.790029:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6658 00000100:00100000:2.0:1789760770.790039:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 750286597930 time 750316177098 state 0x2 ref 0 00000100:00100000:0.0:1789760770.794737:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760770.795062:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512711424 00000100:00100000:1.0:1789760770.795072:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512711424 00000100:00100000:1.0:1789760770.795129:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 750286597930 time 750316177098 state 0x0 ref 1 00000100:00100000:1.0:1789760770.795204:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 750316437514 time 750316177098 state 0x1 ref 1 00000100:00100000:1.0:1789760770.795235:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 750316437514 time 750359534626 state 0x0 ref 1 00000100:00100000:1.0:1789760770.795247:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4075180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512711424:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760770.795294:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27152384, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0xd007 00010000:00010000:1.0:1789760770.795303:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2f66d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.795318:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=750s v=(0 0) 00010000:00010000:1.0:1789760770.795322:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760770.795329:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2f66d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27152384->27156479] (req 27152384->27156479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.795344:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2f66d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27152384->27156479] (req 27152384->27156479) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760770.832634:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973948 is committed 00002000:00010000:2.0:1789760770.832797:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27152384 -> 27156480 00010000:00010000:2.0:1789760770.832813:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2f66d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27152384->27156479] (req 27152384->27156479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.832827:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2f66d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27152384->27156479] (req 27152384->27156479) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.832842:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2f66d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27152384->27156479] (req 27152384->27156479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.832850:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760770.832855:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2f66d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27152384->27156479] (req 27152384->27156479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.832864:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2f66d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27152384->27156479] (req 27152384->27156479) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.832883:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2f66d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27152384->27156479] (req 27152384->27156479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.832891:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2f66d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27152384->27156479] (req 27152384->27156479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760770.832979:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4075180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512711424:12345-192.168.203.56@tcp:4:dd.0 Request processed in 37730us (38246us total) trans 4294973948 rc 0/0 00000100:00100000:2.0:1789760770.832988:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6659 00000100:00100000:2.0:1789760770.832997:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 750316437514 time 750359534626 state 0x2 ref 0 00000100:00100000:2.0:1789760770.836062:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760770.836236:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512711552 00000100:00100000:3.0:1789760770.836246:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512711552 00000100:00100000:3.0:1789760770.836295:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 750316437514 time 750359534626 state 0x0 ref 1 00000100:00100000:3.0:1789760770.836310:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 750359795042 time 750359534626 state 0x1 ref 1 00000100:00100000:3.0:1789760770.836348:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 750359795042 time 750400647658 state 0x0 ref 1 00000100:00100000:3.0:1789760770.836359:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febfbddc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512711552:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760770.836404:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27156480, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0xd007 00010000:00010000:3.0:1789760770.836412:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2f674 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.836427:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=750s v=(0 0) 00010000:00010000:3.0:1789760770.836431:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760770.836437:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2f674 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27156480->27160575] (req 27156480->27160575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.836451:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2f674 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27156480->27160575] (req 27156480->27160575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760770.866035:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973949 is committed 00002000:00010000:3.0:1789760770.866158:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27156480 -> 27160576 00010000:00010000:3.0:1789760770.866173:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2f674 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27156480->27160575] (req 27156480->27160575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.866186:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2f674 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27156480->27160575] (req 27156480->27160575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.866196:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2f674 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27156480->27160575] (req 27156480->27160575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.866204:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760770.866210:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2f674 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27156480->27160575] (req 27156480->27160575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.866219:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2f674 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27156480->27160575] (req 27156480->27160575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.866237:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2f674 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27156480->27160575] (req 27156480->27160575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.866244:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2f674 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27156480->27160575] (req 27156480->27160575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760770.866326:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febfbddc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512711552:12345-192.168.203.56@tcp:4:dd.0 Request processed in 29963us (30268us total) trans 4294973949 rc 0/0 00000100:00100000:3.0:1789760770.866335:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6660 00000100:00100000:3.0:1789760770.866345:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 750359795042 time 750400647658 state 0x2 ref 0 00000100:00100000:0.0:1789760770.873790:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760770.874078:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512711680 00000100:00100000:2.0:1789760770.874088:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512711680 00000100:00100000:2.0:1789760770.874165:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 750359795042 time 750400647658 state 0x0 ref 1 00000100:00100000:2.0:1789760770.874184:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 750400908074 time 750400647658 state 0x1 ref 1 00000100:00100000:2.0:1789760770.874214:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 750400908074 time 750438514519 state 0x0 ref 1 00000100:00100000:2.0:1789760770.874225:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4074a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512711680:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760770.874272:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27160576, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0xd007 00010000:00010000:2.0:1789760770.874281:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2f67b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.874296:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=750s v=(0 0) 00010000:00010000:2.0:1789760770.874301:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760770.874307:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2f67b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27160576->27164671] (req 27160576->27164671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.874321:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2f67b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27160576->27164671] (req 27160576->27164671) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760770.903358:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27160576 -> 27164672 00010000:00010000:0.0:1789760770.903378:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2f67b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27160576->27164671] (req 27160576->27164671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.903390:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2f67b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27160576->27164671] (req 27160576->27164671) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.903399:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2f67b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27160576->27164671] (req 27160576->27164671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.903406:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760770.903411:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2f67b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27160576->27164671] (req 27160576->27164671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.903420:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2f67b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27160576->27164671] (req 27160576->27164671) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.903438:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2f67b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27160576->27164671] (req 27160576->27164671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.903447:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2f67b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27160576->27164671] (req 27160576->27164671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760770.903545:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4074a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2358:x1876699512711680:12345-192.168.203.56@tcp:4:dd.0 Request processed in 29315us (29757us total) trans 4294973950 rc 0/0 00000100:00100000:0.0:1789760770.903555:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6661 00000100:00100000:0.0:1789760770.903563:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 750400908074 time 750438514519 state 0x2 ref 0 00000001:00080000:1.0:1789760770.906066:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973950 is committed 00000100:00100000:0.0:1789760770.906784:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760770.906927:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512711808 00000100:00100000:2.0:1789760770.906937:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512711808 00000100:00100000:2.0:1789760770.906988:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 750400908074 time 750438514519 state 0x0 ref 1 00000100:00100000:2.0:1789760770.907004:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 750438774935 time 750438514519 state 0x1 ref 1 00000100:00100000:2.0:1789760770.907031:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 750438774935 time 750471330998 state 0x0 ref 1 00000100:00100000:2.0:1789760770.907042:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4074700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512711808:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760770.907089:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27164672, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0xd007 00010000:00010000:2.0:1789760770.907099:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2f682 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.907123:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=750s v=(0 0) 00010000:00010000:2.0:1789760770.907127:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760770.907134:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2f682 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27164672->27168767] (req 27164672->27168767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.907222:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2f682 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27164672->27168767] (req 27164672->27168767) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760770.941666:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973951 is committed 00002000:00010000:3.0:1789760770.941817:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27164672 -> 27168768 00010000:00010000:3.0:1789760770.941834:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2f682 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27164672->27168767] (req 27164672->27168767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.941847:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2f682 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27164672->27168767] (req 27164672->27168767) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.941857:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2f682 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27164672->27168767] (req 27164672->27168767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.941870:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760770.941876:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2f682 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27164672->27168767] (req 27164672->27168767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.941886:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2f682 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27164672->27168767] (req 27164672->27168767) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.941906:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2f682 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27164672->27168767] (req 27164672->27168767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.941914:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2f682 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27164672->27168767] (req 27164672->27168767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760770.942015:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4074700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512711808:12345-192.168.203.56@tcp:4:dd.0 Request processed in 34967us (35229us total) trans 4294973951 rc 0/0 00000100:00100000:3.0:1789760770.942025:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6662 00000100:00100000:3.0:1789760770.942034:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 750438774935 time 750471330998 state 0x2 ref 0 00000100:00100000:3.0:1789760770.945354:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760770.945489:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512711936 00000100:00100000:0.0:1789760770.945500:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512711936 00000100:00100000:0.0:1789760770.945562:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 750438774935 time 750471330998 state 0x0 ref 1 00000100:00100000:0.0:1789760770.945578:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 750471591414 time 750471330998 state 0x1 ref 1 00000100:00100000:0.0:1789760770.945608:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 750471591414 time 750509907643 state 0x0 ref 1 00000100:00100000:0.0:1789760770.945621:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5307480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512711936:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760770.945671:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27168768, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0xd007 00010000:00010000:0.0:1789760770.945681:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2f689 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.945733:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=750s v=(0 0) 00010000:00010000:0.0:1789760770.945737:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760770.945743:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2f689 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27168768->27172863] (req 27168768->27172863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.945757:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2f689 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27168768->27172863] (req 27168768->27172863) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760770.979838:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27168768 -> 27172864 00010000:00010000:0.0:1789760770.979861:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2f689 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27168768->27172863] (req 27168768->27172863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.979873:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2f689 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27168768->27172863] (req 27168768->27172863) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.979883:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2f689 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27168768->27172863] (req 27168768->27172863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.979891:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760770.979896:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2f689 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27168768->27172863] (req 27168768->27172863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.979905:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2f689 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27168768->27172863] (req 27168768->27172863) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.979925:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2f689 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27168768->27172863] (req 27168768->27172863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.979933:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2f689 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27168768->27172863] (req 27168768->27172863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760770.980029:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5307480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2358:x1876699512711936:12345-192.168.203.56@tcp:4:dd.0 Request processed in 34404us (34679us total) trans 4294973952 rc 0/0 00000100:00100000:0.0:1789760770.980040:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6663 00000100:00100000:0.0:1789760770.980049:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 750471591414 time 750509907643 state 0x2 ref 0 00000001:00080000:1.0:1789760770.980242:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973952 is committed 00000100:00100000:0.0:1789760770.983214:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760770.983314:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512712064 00000100:00100000:1.0:1789760770.983324:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512712064 00000100:00100000:1.0:1789760770.983393:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 750471591414 time 750509907643 state 0x0 ref 1 00000100:00100000:1.0:1789760770.983411:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 750510168059 time 750509907643 state 0x1 ref 1 00000100:00100000:1.0:1789760770.983443:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 750510168059 time 750547741944 state 0x0 ref 1 00000100:00100000:1.0:1789760770.983455:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4215500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512712064:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760770.983520:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27172864, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0xd007 00010000:00010000:1.0:1789760770.983529:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2f690 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.983545:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=750s v=(0 0) 00010000:00010000:1.0:1789760770.983550:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760770.983556:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2f690 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27172864->27176959] (req 27172864->27176959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.983571:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2f690 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27172864->27176959] (req 27172864->27176959) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760771.019582:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27172864 -> 27176960 00010000:00010000:1.0:1789760771.019602:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2f690 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27172864->27176959] (req 27172864->27176959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.019615:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2f690 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27172864->27176959] (req 27172864->27176959) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.019623:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2f690 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27172864->27176959] (req 27172864->27176959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.019631:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760771.019637:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2f690 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27172864->27176959] (req 27172864->27176959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.019647:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2f690 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27172864->27176959] (req 27172864->27176959) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.019667:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2f690 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27172864->27176959] (req 27172864->27176959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.019675:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2f690 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27172864->27176959] (req 27172864->27176959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760771.019870:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4215500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2359:x1876699512712064:12345-192.168.203.56@tcp:4:dd.0 Request processed in 36409us (36724us total) trans 4294973953 rc 0/0 00000100:00100000:1.0:1789760771.019883:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6664 00000100:00100000:1.0:1789760771.019893:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 750510168059 time 750547741944 state 0x2 ref 0 00000001:00080000:0.0:1789760771.019975:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973953 is committed 00000100:00100000:3.0:1789760771.023840:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760771.024282:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512712192 00000100:00100000:2.0:1789760771.024292:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512712192 00000100:00100000:2.0:1789760771.024371:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 750510168059 time 750547741944 state 0x0 ref 1 00000100:00100000:2.0:1789760771.024386:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 750548002360 time 750547741944 state 0x1 ref 1 00000100:00100000:2.0:1789760771.024421:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 750548002360 time 750588720256 state 0x0 ref 1 00000100:00100000:2.0:1789760771.024434:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5304700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512712192:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760771.024482:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27176960, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0xd007 00010000:00010000:2.0:1789760771.024491:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2f697 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.024506:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=750s v=(0 0) 00000100:00080000:3.0:1789760771.024927:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:2.0:1789760771.025110:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760771.025130:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2f697 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27176960->27181055] (req 27176960->27181055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.025218:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2f697 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27176960->27181055] (req 27176960->27181055) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760771.060977:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27176960 -> 27181056 00010000:00010000:3.0:1789760771.060997:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2f697 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27176960->27181055] (req 27176960->27181055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.061011:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2f697 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27176960->27181055] (req 27176960->27181055) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.061022:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2f697 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27176960->27181055] (req 27176960->27181055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.061030:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760771.061035:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2f697 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27176960->27181055] (req 27176960->27181055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.061045:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2f697 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27176960->27181055] (req 27176960->27181055) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.061064:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2f697 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27176960->27181055] (req 27176960->27181055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.061073:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2f697 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27176960->27181055] (req 27176960->27181055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760771.061182:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5304700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2358:x1876699512712192:12345-192.168.203.56@tcp:4:dd.0 Request processed in 36745us (37347us total) trans 4294973954 rc 0/0 00000100:00100000:3.0:1789760771.061193:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6665 00000100:00100000:3.0:1789760771.061203:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 750548002360 time 750588720256 state 0x2 ref 0 00000001:00080000:1.0:1789760771.062322:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973954 is committed 00000100:00100000:3.0:1789760771.064989:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760771.065081:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512712320 00000100:00100000:0.0:1789760771.065091:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512712320 00000100:00100000:0.0:1789760771.065295:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 750548002360 time 750588720256 state 0x0 ref 1 00000100:00100000:0.0:1789760771.065311:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 750588980672 time 750588720256 state 0x1 ref 1 00000100:00100000:0.0:1789760771.065347:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 750588980672 time 750629645963 state 0x0 ref 1 00000100:00100000:0.0:1789760771.065361:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5304380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512712320:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760771.065409:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27181056, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0xd007 00010000:00010000:0.0:1789760771.065423:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2f69e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.065439:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=750s v=(0 0) 00010000:00010000:0.0:1789760771.065444:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760771.065451:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2f69e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27181056->27185151] (req 27181056->27185151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.065464:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2f69e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27181056->27185151] (req 27181056->27185151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760771.104992:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973955 is committed 00002000:00010000:0.0:1789760771.105305:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27181056 -> 27185152 00010000:00010000:0.0:1789760771.105322:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2f69e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27181056->27185151] (req 27181056->27185151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.105336:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2f69e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27181056->27185151] (req 27181056->27185151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.105345:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2f69e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27181056->27185151] (req 27181056->27185151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.105353:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760771.105358:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2f69e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27181056->27185151] (req 27181056->27185151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.105368:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2f69e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27181056->27185151] (req 27181056->27185151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.105388:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2f69e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27181056->27185151] (req 27181056->27185151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.105395:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2f69e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27181056->27185151] (req 27181056->27185151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760771.105487:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5304380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512712320:12345-192.168.203.56@tcp:4:dd.0 Request processed in 40124us (40502us total) trans 4294973955 rc 0/0 00000100:00100000:0.0:1789760771.105498:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6666 00000100:00100000:0.0:1789760771.105508:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 750588980672 time 750629645963 state 0x2 ref 0 00000100:00100000:0.0:1789760771.111961:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760771.112532:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512712448 00000100:00100000:1.0:1789760771.112545:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512712448 00000100:00100000:1.0:1789760771.112611:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 750588980672 time 750629645963 state 0x0 ref 1 00000100:00100000:1.0:1789760771.112632:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 750629906379 time 750629645963 state 0x1 ref 1 00000100:00100000:1.0:1789760771.112671:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 750629906379 time 750676969944 state 0x0 ref 1 00000100:00100000:1.0:1789760771.112691:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4217100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512712448:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760771.113094:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27185152, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0xd007 00010000:00010000:1.0:1789760771.113107:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2f6a5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.113133:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=750s v=(0 0) 00010000:00010000:1.0:1789760771.113138:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760771.113145:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2f6a5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27185152->27189247] (req 27185152->27189247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.113166:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2f6a5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27185152->27189247] (req 27185152->27189247) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760771.157359:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27185152 -> 27189248 00010000:00010000:0.0:1789760771.157380:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2f6a5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27185152->27189247] (req 27185152->27189247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.157398:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2f6a5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27185152->27189247] (req 27185152->27189247) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.157408:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2f6a5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27185152->27189247] (req 27185152->27189247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.157415:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760771.157420:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2f6a5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27185152->27189247] (req 27185152->27189247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.157429:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2f6a5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27185152->27189247] (req 27185152->27189247) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.157448:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2f6a5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27185152->27189247] (req 27185152->27189247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.157461:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2f6a5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27185152->27189247] (req 27185152->27189247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760771.157555:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4217100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2359:x1876699512712448:12345-192.168.203.56@tcp:4:dd.0 Request processed in 44866us (45599us total) trans 4294973956 rc 0/0 00000100:00100000:0.0:1789760771.157565:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6667 00000100:00100000:0.0:1789760771.157574:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 750629906379 time 750676969944 state 0x2 ref 0 00000001:00080000:3.0:1789760771.157903:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973956 is committed 00000100:00100000:3.0:1789760771.161004:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760771.161360:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512712576 00000100:00100000:2.0:1789760771.161370:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512712576 00000100:00100000:2.0:1789760771.161422:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 750629906379 time 750676969944 state 0x0 ref 1 00000100:00100000:2.0:1789760771.161439:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 750677230360 time 750676969944 state 0x1 ref 1 00000100:00100000:2.0:1789760771.161472:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 750677230360 time 750725772497 state 0x0 ref 1 00000100:00100000:2.0:1789760771.161484:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5306680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512712576:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760771.161533:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27189248, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0xd007 00010000:00010000:2.0:1789760771.161543:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2f6ac lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.161573:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=750s v=(0 0) 00010000:00010000:2.0:1789760771.161578:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760771.161585:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2f6ac lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27189248->27193343] (req 27189248->27193343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.161601:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2f6ac lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27189248->27193343] (req 27189248->27193343) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760771.187461:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973957 is committed 00002000:00010000:2.0:1789760771.187662:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27189248 -> 27193344 00010000:00010000:2.0:1789760771.187678:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2f6ac lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27189248->27193343] (req 27189248->27193343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.187691:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2f6ac lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27189248->27193343] (req 27189248->27193343) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.187728:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2f6ac lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27189248->27193343] (req 27189248->27193343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.187735:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760771.187740:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2f6ac lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27189248->27193343] (req 27189248->27193343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.187748:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2f6ac lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27189248->27193343] (req 27189248->27193343) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.187767:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2f6ac lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27189248->27193343] (req 27189248->27193343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.187774:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2f6ac lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27189248->27193343] (req 27189248->27193343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760771.187856:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5306680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512712576:12345-192.168.203.56@tcp:4:dd.0 Request processed in 26369us (26856us total) trans 4294973957 rc 0/0 00000100:00100000:2.0:1789760771.187869:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6668 00000100:00100000:2.0:1789760771.187877:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 750677230360 time 750725772497 state 0x2 ref 0 00000100:00100000:1.0:1789760771.190771:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760771.190849:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512712704 00000100:00100000:0.0:1789760771.190860:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512712704 00000100:00100000:0.0:1789760771.190916:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 750677230360 time 750725772497 state 0x0 ref 1 00000100:00100000:0.0:1789760771.190932:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 750726032913 time 750725772497 state 0x1 ref 1 00000100:00100000:0.0:1789760771.190964:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 750726032913 time 750755263891 state 0x0 ref 1 00000100:00100000:0.0:1789760771.190976:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4038000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512712704:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760771.191018:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27193344, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0xd007 00010000:00010000:0.0:1789760771.191027:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2f6b3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.191042:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=750s v=(0 0) 00010000:00010000:0.0:1789760771.191046:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760771.191052:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2f6b3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27193344->27197439] (req 27193344->27197439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.191071:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2f6b3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27193344->27197439] (req 27193344->27197439) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760771.218565:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973958 is committed 00002000:00010000:0.0:1789760771.218766:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27193344 -> 27197440 00010000:00010000:0.0:1789760771.218781:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2f6b3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27193344->27197439] (req 27193344->27197439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.218803:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2f6b3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27193344->27197439] (req 27193344->27197439) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.218813:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2f6b3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27193344->27197439] (req 27193344->27197439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.218820:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760771.218825:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2f6b3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27193344->27197439] (req 27193344->27197439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.218834:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2f6b3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27193344->27197439] (req 27193344->27197439) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.218853:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2f6b3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27193344->27197439] (req 27193344->27197439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.218862:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2f6b3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27193344->27197439] (req 27193344->27197439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760771.218959:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4038000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512712704:12345-192.168.203.56@tcp:4:dd.0 Request processed in 27978us (28190us total) trans 4294973958 rc 0/0 00000100:00100000:0.0:1789760771.218970:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6669 00000100:00100000:0.0:1789760771.218982:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 750726032913 time 750755263891 state 0x2 ref 0 00000100:00100000:0.0:1789760771.222359:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760771.222464:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512712832 00000100:00100000:2.0:1789760771.222474:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512712832 00000100:00100000:2.0:1789760771.222598:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 750726032913 time 750755263891 state 0x0 ref 1 00000100:00100000:2.0:1789760771.222614:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 750755524307 time 750755263891 state 0x1 ref 1 00000100:00100000:2.0:1789760771.222681:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 750755524307 time 750786979230 state 0x0 ref 1 00000100:00100000:2.0:1789760771.222694:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac287a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512712832:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760771.222781:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27197440, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0xd007 00010000:00010000:2.0:1789760771.222892:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2f6ba lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.222909:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=750s v=(0 0) 00010000:00010000:2.0:1789760771.222913:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760771.222920:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2f6ba lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27197440->27201535] (req 27197440->27201535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.222934:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2f6ba lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27197440->27201535] (req 27197440->27201535) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760771.264912:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27197440 -> 27201536 00010000:00010000:2.0:1789760771.264937:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2f6ba lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27197440->27201535] (req 27197440->27201535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.264951:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2f6ba lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27197440->27201535] (req 27197440->27201535) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.264961:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2f6ba lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27197440->27201535] (req 27197440->27201535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.264969:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760771.264974:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2f6ba lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27197440->27201535] (req 27197440->27201535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.264984:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2f6ba lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27197440->27201535] (req 27197440->27201535) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.265004:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2f6ba lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27197440->27201535] (req 27197440->27201535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.265014:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2f6ba lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27197440->27201535] (req 27197440->27201535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760771.265125:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac287a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2358:x1876699512712832:12345-192.168.203.56@tcp:4:dd.0 Request processed in 42422us (42766us total) trans 4294973959 rc 0/0 00000100:00100000:2.0:1789760771.265135:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6670 00000100:00100000:2.0:1789760771.265146:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 750755524307 time 750786979230 state 0x2 ref 0 00000001:00080000:1.0:1789760771.267312:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973959 is committed 00000100:00100000:2.0:1789760771.269003:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760771.269062:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512712960 00000100:00100000:0.0:1789760771.269153:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512712960 00000100:00100000:0.0:1789760771.269207:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 750755524307 time 750786979230 state 0x0 ref 1 00000100:00100000:0.0:1789760771.269221:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 750787239646 time 750786979230 state 0x1 ref 1 00000100:00100000:0.0:1789760771.269295:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 750787239646 time 750833593973 state 0x0 ref 1 00000100:00100000:0.0:1789760771.269308:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febfbddca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512712960:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760771.269363:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27201536, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0xd007 00010000:00010000:0.0:1789760771.269371:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2f6c1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.269388:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=750s v=(0 0) 00010000:00010000:0.0:1789760771.269393:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760771.269400:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2f6c1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27201536->27205631] (req 27201536->27205631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.269413:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2f6c1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27201536->27205631] (req 27201536->27205631) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760771.304252:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27201536 -> 27205632 00010000:00010000:0.0:1789760771.304272:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2f6c1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27201536->27205631] (req 27201536->27205631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.304284:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2f6c1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27201536->27205631] (req 27201536->27205631) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.304293:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2f6c1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27201536->27205631] (req 27201536->27205631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.304299:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760771.304304:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2f6c1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27201536->27205631] (req 27201536->27205631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.304313:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2f6c1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27201536->27205631] (req 27201536->27205631) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.304331:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2f6c1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27201536->27205631] (req 27201536->27205631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.304339:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2f6c1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27201536->27205631] (req 27201536->27205631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760771.304438:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febfbddca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2358:x1876699512712960:12345-192.168.203.56@tcp:4:dd.0 Request processed in 35128us (35439us total) trans 4294973960 rc 0/0 00000100:00100000:0.0:1789760771.304453:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6671 00000100:00100000:0.0:1789760771.304463:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 750787239646 time 750833593973 state 0x2 ref 0 00000001:00080000:1.0:1789760771.306195:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973960 is committed 00000100:00100000:0.0:1789760771.307304:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760771.307969:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512713088 00000100:00100000:3.0:1789760771.307980:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512713088 00000100:00100000:3.0:1789760771.308043:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 750787239646 time 750833593973 state 0x0 ref 1 00000100:00100000:3.0:1789760771.308061:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 750833854389 time 750833593973 state 0x1 ref 1 00000100:00100000:3.0:1789760771.308215:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 750833854389 time 750872513605 state 0x0 ref 1 00000100:00100000:3.0:1789760771.308229:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2879c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512713088:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760771.308278:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27205632, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0xd007 00010000:00010000:3.0:1789760771.308286:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2f6c8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.308302:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=750s v=(0 0) 00010000:00010000:3.0:1789760771.308307:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760771.308313:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2f6c8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27205632->27209727] (req 27205632->27209727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.308327:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2f6c8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27205632->27209727] (req 27205632->27209727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760771.336759:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973961 is committed 00002000:00010000:3.0:1789760771.336917:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27205632 -> 27209728 00010000:00010000:3.0:1789760771.336934:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2f6c8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27205632->27209727] (req 27205632->27209727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.336948:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2f6c8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27205632->27209727] (req 27205632->27209727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.336958:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2f6c8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27205632->27209727] (req 27205632->27209727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.336965:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760771.336970:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2f6c8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27205632->27209727] (req 27205632->27209727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.336979:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2f6c8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27205632->27209727] (req 27205632->27209727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.336997:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2f6c8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27205632->27209727] (req 27205632->27209727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.337004:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2f6c8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27205632->27209727] (req 27205632->27209727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760771.337097:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2879c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512713088:12345-192.168.203.56@tcp:4:dd.0 Request processed in 28865us (29796us total) trans 4294973961 rc 0/0 00000100:00100000:3.0:1789760771.337107:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6672 00000100:00100000:3.0:1789760771.337128:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 750833854389 time 750872513605 state 0x2 ref 0 00000100:00100000:3.0:1789760771.339536:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760771.339659:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512713216 00000100:00100000:2.0:1789760771.339668:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512713216 00000100:00100000:2.0:1789760771.339749:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 750833854389 time 750872513605 state 0x0 ref 1 00000100:00100000:2.0:1789760771.339764:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 750872774021 time 750872513605 state 0x1 ref 1 00000100:00100000:1.0:1789760771.339910:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 750872774021 time 750904207818 state 0x0 ref 1 00000100:00100000:1.0:1789760771.339925:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5307100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512713216:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760771.339972:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27209728, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0xd007 00010000:00010000:1.0:1789760771.339981:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2f6cf lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.339997:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=750s v=(0 0) 00010000:00010000:1.0:1789760771.340001:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760771.340007:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2f6cf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27209728->27213823] (req 27209728->27213823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.340021:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2f6cf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27209728->27213823] (req 27209728->27213823) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760771.381365:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973962 is committed 00002000:00010000:0.0:1789760771.381606:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27209728 -> 27213824 00010000:00010000:0.0:1789760771.381621:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2f6cf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27209728->27213823] (req 27209728->27213823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.381634:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2f6cf lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27209728->27213823] (req 27209728->27213823) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.381642:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2f6cf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27209728->27213823] (req 27209728->27213823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.381650:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760771.381655:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2f6cf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27209728->27213823] (req 27209728->27213823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.381663:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2f6cf lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27209728->27213823] (req 27209728->27213823) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.381684:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2f6cf lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27209728->27213823] (req 27209728->27213823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.381691:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2f6cf lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27209728->27213823] (req 27209728->27213823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760771.381815:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5307100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512713216:12345-192.168.203.56@tcp:4:dd.0 Request processed in 41889us (42282us total) trans 4294973962 rc 0/0 00000100:00100000:0.0:1789760771.381831:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6673 00000100:00100000:0.0:1789760771.381841:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 750872774021 time 750904207818 state 0x2 ref 0 00000100:00100000:0.0:1789760771.391959:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760771.392069:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512713344 00000100:00100000:2.0:1789760771.392079:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512713344 00000100:00100000:2.0:1789760771.392442:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 750872774021 time 750904207818 state 0x0 ref 1 00000100:00100000:2.0:1789760771.392458:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 750904468234 time 750904207818 state 0x1 ref 1 00000100:00100000:2.0:1789760771.392480:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 750904468234 time 750956780078 state 0x0 ref 1 00000100:00100000:2.0:1789760771.392494:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2878e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512713344:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760771.392549:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27213824, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0xd007 00010000:00010000:2.0:1789760771.392559:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2f6d6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.392576:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=750s v=(0 0) 00010000:00010000:2.0:1789760771.392581:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760771.392588:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2f6d6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27213824->27217919] (req 27213824->27217919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.392602:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2f6d6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27213824->27217919] (req 27213824->27217919) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760771.423973:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27213824 -> 27217920 00010000:00010000:3.0:1789760771.423995:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2f6d6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27213824->27217919] (req 27213824->27217919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.424008:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2f6d6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27213824->27217919] (req 27213824->27217919) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.424018:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2f6d6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27213824->27217919] (req 27213824->27217919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.424026:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760771.424030:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2f6d6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27213824->27217919] (req 27213824->27217919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.424040:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2f6d6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27213824->27217919] (req 27213824->27217919) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.424063:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2f6d6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27213824->27217919] (req 27213824->27217919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.424071:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2f6d6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27213824->27217919] (req 27213824->27217919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760771.424173:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2878e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2361:x1876699512713344:12345-192.168.203.56@tcp:4:dd.0 Request processed in 31677us (32220us total) trans 4294973963 rc 0/0 00000100:00100000:3.0:1789760771.424183:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6674 00000100:00100000:3.0:1789760771.424192:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 750904468234 time 750956780078 state 0x2 ref 0 00000001:00080000:1.0:1789760771.424272:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973963 is committed 00000100:00100000:3.0:1789760771.428093:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760771.428622:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512713472 00000100:00100000:0.0:1789760771.428633:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512713472 00000100:00100000:0.0:1789760771.428688:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 750904468234 time 750956780078 state 0x0 ref 1 00000100:00100000:0.0:1789760771.428961:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 750957040494 time 750956780078 state 0x1 ref 1 00000100:00100000:0.0:1789760771.429042:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 750957040494 time 750993341182 state 0x0 ref 1 00000100:00100000:0.0:1789760771.429062:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5304a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512713472:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760771.429126:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27217920, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0xd007 00010000:00010000:0.0:1789760771.429136:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2f6dd lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.429152:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=750s v=(0 0) 00010000:00010000:0.0:1789760771.429156:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760771.429163:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2f6dd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27217920->27222015] (req 27217920->27222015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.429177:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2f6dd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27217920->27222015] (req 27217920->27222015) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.442878:0:7647:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:1.0:1789760771.458748:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973964 is committed 00002000:00010000:0.0:1789760771.458953:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27217920 -> 27222016 00010000:00010000:0.0:1789760771.458968:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2f6dd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27217920->27222015] (req 27217920->27222015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.458982:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2f6dd lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27217920->27222015] (req 27217920->27222015) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.458991:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2f6dd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27217920->27222015] (req 27217920->27222015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.458999:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760771.459005:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2f6dd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27217920->27222015] (req 27217920->27222015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.459015:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2f6dd lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27217920->27222015] (req 27217920->27222015) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.459039:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2f6dd lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27217920->27222015] (req 27217920->27222015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.459048:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2f6dd lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27217920->27222015] (req 27217920->27222015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760771.459160:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5304a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512713472:12345-192.168.203.56@tcp:4:dd.0 Request processed in 30101us (31071us total) trans 4294973964 rc 0/0 00000100:00100000:0.0:1789760771.459173:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6675 00000100:00100000:0.0:1789760771.459182:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 750957040494 time 750993341182 state 0x2 ref 0 00000100:00100000:0.0:1789760771.461863:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760771.461979:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512713600 00000100:00100000:1.0:1789760771.461988:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512713600 00000100:00100000:1.0:1789760771.462043:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 750957040494 time 750993341182 state 0x0 ref 1 00000100:00100000:1.0:1789760771.462059:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 750993601598 time 750993341182 state 0x1 ref 1 00000100:00100000:1.0:1789760771.462091:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 750993601598 time 751026390187 state 0x0 ref 1 00000100:00100000:1.0:1789760771.462106:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac287b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512713600:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760771.462162:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27222016, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0xd007 00010000:00010000:1.0:1789760771.462171:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2f6e4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.462187:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=751s v=(0 0) 00010000:00010000:1.0:1789760771.462192:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760771.462199:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2f6e4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27222016->27226111] (req 27222016->27226111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.462213:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2f6e4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27222016->27226111] (req 27222016->27226111) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760771.497175:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973965 is committed 00002000:00010000:0.0:1789760771.497279:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27222016 -> 27226112 00010000:00010000:0.0:1789760771.497294:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2f6e4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27222016->27226111] (req 27222016->27226111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.497307:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2f6e4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27222016->27226111] (req 27222016->27226111) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.497316:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2f6e4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27222016->27226111] (req 27222016->27226111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.497323:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760771.497329:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2f6e4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27222016->27226111] (req 27222016->27226111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.497339:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2f6e4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27222016->27226111] (req 27222016->27226111) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.497356:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2f6e4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27222016->27226111] (req 27222016->27226111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.497364:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2f6e4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27222016->27226111] (req 27222016->27226111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760771.497468:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac287b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512713600:12345-192.168.203.56@tcp:4:dd.0 Request processed in 35358us (35608us total) trans 4294973965 rc 0/0 00000100:00100000:0.0:1789760771.497477:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6676 00000100:00100000:0.0:1789760771.497487:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 750993601598 time 751026390187 state 0x2 ref 0 00000100:00100000:3.0:1789760771.500085:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760771.500905:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512713728 00000100:00100000:0.0:1789760771.500919:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512713728 00000100:00100000:1.0:1789760771.501053:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 750993601598 time 751026390187 state 0x0 ref 1 00000100:00100000:1.0:1789760771.501068:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 751026650603 time 751026390187 state 0x1 ref 1 00000100:00100000:1.0:1789760771.501138:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 751026650603 time 751065437823 state 0x0 ref 1 00000100:00100000:1.0:1789760771.501151:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5305880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512713728:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760771.501198:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27226112, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0xd007 00010000:00010000:1.0:1789760771.501207:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2f6eb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.501223:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=751s v=(0 0) 00010000:00010000:1.0:1789760771.501228:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760771.501235:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2f6eb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27226112->27230207] (req 27226112->27230207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.501247:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2f6eb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27226112->27230207] (req 27226112->27230207) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760771.527458:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973966 is committed 00002000:00010000:1.0:1789760771.527751:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27226112 -> 27230208 00002000:00010000:1.0:1789760771.527756:0:13062:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760770 -> 1789760771 00002000:00010000:1.0:1789760771.527759:0:13062:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760770 -> 1789760771 00010000:00010000:1.0:1789760771.527772:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2f6eb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27226112->27230207] (req 27226112->27230207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.527840:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2f6eb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27226112->27230207] (req 27226112->27230207) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.527850:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2f6eb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27226112->27230207] (req 27226112->27230207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.527857:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760771.527862:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2f6eb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27226112->27230207] (req 27226112->27230207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.527872:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2f6eb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27226112->27230207] (req 27226112->27230207) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.527892:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2f6eb lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27226112->27230207] (req 27226112->27230207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.527900:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2f6eb lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27226112->27230207] (req 27226112->27230207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760771.528027:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5305880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512713728:12345-192.168.203.56@tcp:4:dd.0 Request processed in 26874us (27946us total) trans 4294973966 rc 0/0 00000100:00100000:1.0:1789760771.528038:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6677 00000100:00100000:1.0:1789760771.528047:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 751026650603 time 751065437823 state 0x2 ref 0 00000100:00100000:3.0:1789760771.531131:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760771.531225:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512713856 00000100:00100000:0.0:1789760771.531235:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512713856 00000100:00100000:0.0:1789760771.531289:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 751026650603 time 751065437823 state 0x0 ref 1 00000100:00100000:0.0:1789760771.531305:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 751065698239 time 751065437823 state 0x1 ref 1 00000100:00100000:0.0:1789760771.531364:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 751065698239 time 751095663057 state 0x0 ref 1 00000100:00100000:0.0:1789760771.531377:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5306300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512713856:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760771.531425:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27230208, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0xd007 00010000:00010000:0.0:1789760771.531433:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2f6f2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.531447:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=751s v=(0 0) 00010000:00010000:0.0:1789760771.531452:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760771.531459:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2f6f2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27230208->27234303] (req 27230208->27234303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.531472:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2f6f2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27230208->27234303] (req 27230208->27234303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760771.563276:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27230208 -> 27234304 00010000:00010000:3.0:1789760771.563296:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2f6f2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27230208->27234303] (req 27230208->27234303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.563308:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2f6f2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27230208->27234303] (req 27230208->27234303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.563318:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2f6f2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27230208->27234303] (req 27230208->27234303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.563325:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760771.563329:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2f6f2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27230208->27234303] (req 27230208->27234303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.563338:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2f6f2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27230208->27234303] (req 27230208->27234303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.563358:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2f6f2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27230208->27234303] (req 27230208->27234303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.563364:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2f6f2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27230208->27234303] (req 27230208->27234303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760771.563459:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5306300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2359:x1876699512713856:12345-192.168.203.56@tcp:4:dd.0 Request processed in 32079us (32341us total) trans 4294973967 rc 0/0 00000100:00100000:3.0:1789760771.563471:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6678 00000100:00100000:3.0:1789760771.563480:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 751065698239 time 751095663057 state 0x2 ref 0 00000001:00080000:1.0:1789760771.565897:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973967 is committed 00000100:00100000:1.0:1789760771.566482:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760771.566616:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512713984 00000100:00100000:2.0:1789760771.566626:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512713984 00000100:00100000:2.0:1789760771.566679:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 751065698239 time 751095663057 state 0x0 ref 1 00000100:00100000:2.0:1789760771.566730:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 751095923473 time 751095663057 state 0x1 ref 1 00000100:00100000:2.0:1789760771.566756:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 751095923473 time 751131056348 state 0x0 ref 1 00000100:00100000:2.0:1789760771.566768:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac403bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512713984:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760771.566873:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27234304, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0xd007 00010000:00010000:2.0:1789760771.566882:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2f6f9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.566901:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=751s v=(0 0) 00010000:00010000:2.0:1789760771.566905:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760771.566912:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2f6f9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27234304->27238399] (req 27234304->27238399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.566926:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2f6f9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27234304->27238399] (req 27234304->27238399) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760771.607259:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973968 is committed 00002000:00010000:3.0:1789760771.607367:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27234304 -> 27238400 00010000:00010000:3.0:1789760771.607382:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2f6f9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27234304->27238399] (req 27234304->27238399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.607395:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2f6f9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27234304->27238399] (req 27234304->27238399) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.607403:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2f6f9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27234304->27238399] (req 27234304->27238399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.607409:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760771.607414:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2f6f9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27234304->27238399] (req 27234304->27238399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.607422:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2f6f9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27234304->27238399] (req 27234304->27238399) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.607441:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2f6f9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27234304->27238399] (req 27234304->27238399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.607449:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2f6f9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27234304->27238399] (req 27234304->27238399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760771.607744:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac403bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512713984:12345-192.168.203.56@tcp:4:dd.0 Request processed in 40972us (41261us total) trans 4294973968 rc 0/0 00000100:00100000:3.0:1789760771.607755:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6679 00000100:00100000:3.0:1789760771.607765:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 751095923473 time 751131056348 state 0x2 ref 0 00000100:00100000:2.0:1789760771.612245:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760771.612329:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512714112 00000100:00100000:3.0:1789760771.612339:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512714112 00000100:00100000:3.0:1789760771.612393:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 751095923473 time 751131056348 state 0x0 ref 1 00000100:00100000:3.0:1789760771.612407:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 751131316764 time 751131056348 state 0x1 ref 1 00000100:00100000:3.0:1789760771.612439:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 751131316764 time 751176738704 state 0x0 ref 1 00000100:00100000:3.0:1789760771.612452:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febea89b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512714112:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760771.612498:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27238400, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0xd007 00010000:00010000:3.0:1789760771.612615:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2f700 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.612633:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=751s v=(0 0) 00010000:00010000:3.0:1789760771.612638:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760771.612644:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2f700 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27238400->27242495] (req 27238400->27242495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.612658:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2f700 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27238400->27242495] (req 27238400->27242495) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760771.674591:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27238400 -> 27242496 00000001:00080000:1.0:1789760771.674607:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973969 is committed 00010000:00010000:2.0:1789760771.674611:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2f700 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27238400->27242495] (req 27238400->27242495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.674626:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2f700 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27238400->27242495] (req 27238400->27242495) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.674636:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2f700 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27238400->27242495] (req 27238400->27242495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.674643:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760771.674648:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2f700 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27238400->27242495] (req 27238400->27242495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.674657:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2f700 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27238400->27242495] (req 27238400->27242495) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.674675:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2f700 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27238400->27242495] (req 27238400->27242495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.674683:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2f700 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27238400->27242495] (req 27238400->27242495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760771.674807:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febea89b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512714112:12345-192.168.203.56@tcp:4:dd.0 Request processed in 62353us (62570us total) trans 4294973969 rc 0/0 00000100:00100000:2.0:1789760771.674818:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6680 00000100:00100000:2.0:1789760771.674827:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 751131316764 time 751176738704 state 0x2 ref 0 00000100:00100000:2.0:1789760771.678246:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760771.678302:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512714240 00000100:00100000:3.0:1789760771.678311:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512714240 00000100:00100000:3.0:1789760771.678368:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 751131316764 time 751176738704 state 0x0 ref 1 00000100:00100000:3.0:1789760771.678383:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 751176999120 time 751176738704 state 0x1 ref 1 00000100:00100000:3.0:1789760771.678414:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 751176999120 time 751242714224 state 0x0 ref 1 00000100:00100000:3.0:1789760771.678428:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2fba680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512714240:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760771.678477:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27242496, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0xd007 00010000:00010000:3.0:1789760771.678486:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2f707 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.678502:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=751s v=(0 0) 00010000:00010000:3.0:1789760771.678506:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760771.678512:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2f707 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27242496->27246591] (req 27242496->27246591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.678524:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2f707 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27242496->27246591] (req 27242496->27246591) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760771.725470:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973970 is committed 00002000:00010000:3.0:1789760771.725628:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27242496 -> 27246592 00010000:00010000:3.0:1789760771.725645:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2f707 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27242496->27246591] (req 27242496->27246591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.725662:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2f707 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27242496->27246591] (req 27242496->27246591) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.725671:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2f707 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27242496->27246591] (req 27242496->27246591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.725678:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760771.725683:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2f707 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27242496->27246591] (req 27242496->27246591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.725847:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2f707 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27242496->27246591] (req 27242496->27246591) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.725868:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2f707 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27242496->27246591] (req 27242496->27246591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.725876:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2f707 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27242496->27246591] (req 27242496->27246591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760771.725965:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2fba680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512714240:12345-192.168.203.56@tcp:4:dd.0 Request processed in 47535us (47723us total) trans 4294973970 rc 0/0 00000100:00100000:3.0:1789760771.725974:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6681 00000100:00100000:3.0:1789760771.725984:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 751176999120 time 751242714224 state 0x2 ref 0 00000100:00100000:0.0:1789760771.738005:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760771.739232:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512714368 00000100:00100000:3.0:1789760771.739250:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512714368 00000100:00100000:3.0:1789760771.739338:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 751176999120 time 751242714224 state 0x0 ref 1 00000100:00100000:3.0:1789760771.739360:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 751242974640 time 751242714224 state 0x1 ref 1 00000100:00100000:3.0:1789760771.739658:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 751242974640 time 751303955520 state 0x0 ref 1 00000100:00100000:3.0:1789760771.739676:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2878000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512714368:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760771.739769:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27246592, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0xd007 00010000:00010000:3.0:1789760771.739791:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2f70e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.739813:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=751s v=(0 0) 00010000:00010000:3.0:1789760771.739817:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760771.739826:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2f70e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27246592->27250687] (req 27246592->27250687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.739842:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2f70e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27246592->27250687] (req 27246592->27250687) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760771.802603:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973971 is committed 00002000:00010000:3.0:1789760771.802847:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27246592 -> 27250688 00010000:00010000:3.0:1789760771.802868:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2f70e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27246592->27250687] (req 27246592->27250687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.802884:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2f70e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27246592->27250687] (req 27246592->27250687) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.802901:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2f70e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27246592->27250687] (req 27246592->27250687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.802910:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760771.802915:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2f70e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27246592->27250687] (req 27246592->27250687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.802925:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2f70e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27246592->27250687] (req 27246592->27250687) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.802950:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2f70e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27246592->27250687] (req 27246592->27250687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.802957:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2f70e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27246592->27250687] (req 27246592->27250687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760771.803076:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2878000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512714368:12345-192.168.203.56@tcp:4:dd.0 Request processed in 63398us (65295us total) trans 4294973971 rc 0/0 00000100:00100000:3.0:1789760771.803087:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6682 00000100:00100000:3.0:1789760771.803099:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 751242974640 time 751303955520 state 0x2 ref 0 00000100:00100000:3.0:1789760771.808516:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760771.808747:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512714496 00000100:00100000:0.0:1789760771.808762:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512714496 00000100:00100000:0.0:1789760771.808841:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 751242974640 time 751303955520 state 0x0 ref 1 00000100:00100000:0.0:1789760771.808860:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 751304215936 time 751303955520 state 0x1 ref 1 00000100:00100000:0.0:1789760771.808893:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 751304215936 time 751373192388 state 0x0 ref 1 00000100:00100000:0.0:1789760771.808906:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5307800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512714496:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760771.808970:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27250688, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0xd007 00010000:00010000:0.0:1789760771.808981:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2f715 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.809000:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=751s v=(0 0) 00010000:00010000:0.0:1789760771.809006:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760771.809014:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2f715 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27250688->27254783] (req 27250688->27254783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.809030:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2f715 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27250688->27254783] (req 27250688->27254783) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760771.878561:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973972 is committed 00002000:00010000:0.0:1789760771.878970:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27250688 -> 27254784 00010000:00010000:0.0:1789760771.878990:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2f715 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27250688->27254783] (req 27250688->27254783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.879012:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2f715 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27250688->27254783] (req 27250688->27254783) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.879021:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2f715 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27250688->27254783] (req 27250688->27254783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.879028:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760771.879034:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2f715 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27250688->27254783] (req 27250688->27254783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.879043:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2f715 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27250688->27254783] (req 27250688->27254783) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.879069:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2f715 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27250688->27254783] (req 27250688->27254783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.879077:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2f715 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27250688->27254783] (req 27250688->27254783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760771.879204:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5307800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512714496:12345-192.168.203.56@tcp:4:dd.0 Request processed in 70291us (70707us total) trans 4294973972 rc 0/0 00000100:00100000:0.0:1789760771.879217:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6683 00000100:00100000:0.0:1789760771.879230:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 751304215936 time 751373192388 state 0x2 ref 0 00000100:00100000:2.0:1789760771.888342:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760771.890030:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512714624 00000100:00100000:3.0:1789760771.890052:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512714624 00000100:00100000:3.0:1789760771.890153:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 751304215936 time 751373192388 state 0x0 ref 1 00000100:00100000:3.0:1789760771.890177:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 751373452804 time 751373192388 state 0x1 ref 1 00000100:00100000:3.0:1789760771.890221:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 751373452804 time 751454520448 state 0x0 ref 1 00000100:00100000:3.0:1789760771.890238:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2fb8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512714624:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760771.890307:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27254784, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0xd007 00010000:00010000:3.0:1789760771.890318:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2f71c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.890341:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=751s v=(0 0) 00010000:00010000:3.0:1789760771.890346:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760771.890354:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2f71c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27254784->27258879] (req 27254784->27258879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.890370:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2f71c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27254784->27258879] (req 27254784->27258879) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760771.982891:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973973 is committed 00002000:00010000:3.0:1789760771.983071:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27254784 -> 27258880 00010000:00010000:3.0:1789760771.983095:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2f71c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27254784->27258879] (req 27254784->27258879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.983123:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2f71c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27254784->27258879] (req 27254784->27258879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.983135:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2f71c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27254784->27258879] (req 27254784->27258879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.983260:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760771.983267:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2f71c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27254784->27258879] (req 27254784->27258879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.983279:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2f71c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27254784->27258879] (req 27254784->27258879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.983307:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2f71c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27254784->27258879] (req 27254784->27258879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.983315:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2f71c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27254784->27258879] (req 27254784->27258879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760771.983608:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2fb8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512714624:12345-192.168.203.56@tcp:4:dd.0 Request processed in 93362us (95271us total) trans 4294973973 rc 0/0 00000100:00100000:3.0:1789760771.983624:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6684 00000100:00100000:3.0:1789760771.983637:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 751373452804 time 751454520448 state 0x2 ref 0 00000100:00100000:1.0:1789760771.989259:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760771.989553:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512714752 00000100:00100000:3.0:1789760771.989566:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512714752 00000100:00100000:3.0:1789760771.989665:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 751373452804 time 751454520448 state 0x0 ref 1 00000100:00100000:3.0:1789760771.989688:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 751454780864 time 751454520448 state 0x1 ref 1 00000100:00100000:3.0:1789760771.989968:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 751454780864 time 751554264948 state 0x0 ref 1 00000100:00100000:3.0:1789760771.989990:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febe7906300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512714752:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760771.990060:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27258880, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0xd007 00010000:00010000:3.0:1789760771.990071:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2f723 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.990092:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=751s v=(0 0) 00010000:00010000:3.0:1789760771.990098:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760771.990117:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2f723 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27258880->27262975] (req 27258880->27262975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.990135:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2f723 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27258880->27262975] (req 27258880->27262975) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760772.020736:0:3309:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8febfb374380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3309:1876699540229632:0@lo:400:kworker.0 00000100:00100000:0.0:1789760772.020850:0:3309:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760772.021046:0:3309:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8feac38b2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3309:1876699540230144:0@lo:400:kworker.0 00000100:00100000:0.0:1789760772.021081:0:3309:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760772.021132:0:3311:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8feac287b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3311:1876699540229376:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1789760772.021135:0:3309:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1789760772.021140:0:6705:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540230144 00000100:00100000:0.0:1789760772.021141:0:3309:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8feac38b0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3309:1876699540229760:0@lo:400:kworker.0 00000100:00100000:3.0:1789760772.021152:0:6705:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540230144 00000100:00100000:0.0:1789760772.021173:0:3309:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760772.021227:0:6705:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 190 00000100:00100000:3.0:1789760772.021241:0:6705:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac287aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3309:x1876699540230144:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760772.021311:0:6705:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac287aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3309:x1876699540230144:12345-0@lo:400:kworker.0 Request processed in 75us (229us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760772.021319:0:6705:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 190 00000100:00080000:3.0:1789760772.021344:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1789760772.021383:0:3309:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8feac38b2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3309:1876699540230144:0@lo:400:kworker.0 00000100:00100000:0.0:1789760772.021390:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540229632 00000100:00100000:0.0:1789760772.021396:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540229632 00000100:00100000:0.0:1789760772.021432:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540229760 00000100:00100000:0.0:1789760772.021435:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540229760 00000100:00100000:3.0:1789760772.021448:0:8229:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 501 00000100:00100000:3.0:1789760772.021459:0:8229:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2878700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3309:x1876699540229632:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760772.021468:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 502 00000100:00100000:0.0:1789760772.021478:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2879880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3309:x1876699540229760:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760772.021501:0:8229:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2878700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3309:x1876699540229632:12345-0@lo:400:kworker.0 Request processed in 44us (652us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760772.021509:0:8229:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 501 00000100:00080000:3.0:1789760772.021525:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789760772.021663:0:3312:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8feac2fb8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:6df98d54-eb52-4c5c-8fc9-ee9d74f90f98:3312:1876699540229504:0@lo:400:kworker.0 00000100:00100000:3.0:1789760772.022177:0:3309:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8febfb374380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3309:1876699540229632:0@lo:400:kworker.0 00000100:00100000:3.0:1789760772.024684:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760772.026376:0:6707:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540229376 00000100:00100000:1.0:1789760772.026396:0:6707:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540229376 00000100:00100000:1.0:1789760772.026560:0:6707:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 208 00000100:00100000:1.0:1789760772.026575:0:6707:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5306a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3311:x1876699540229376:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1789760772.026610:0:6707:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 527761408 pending 1703936 free 3831496704 avail 3793747968 00000100:00100000:3.0:1789760772.026627:0:3311:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8feac38b2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3311:1876699540229888:0@lo:400:kworker.0 00000100:00100000:3.0:1789760772.026678:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760772.026685:0:6707:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5306a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3311:x1876699540229376:12345-0@lo:13:osp-pre-0-0.0 Request processed in 115us (2009us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760772.026693:0:6707:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 208 00000100:00100000:3.0:1789760772.026763:0:3311:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8feac287b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3311:1876699540229376:0@lo:13:osp-pre-0-0.0 00000100:00080000:1.0:1789760772.026817:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1789760772.031873:0:10102:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540229888 00000100:00100000:3.0:1789760772.031894:0:10102:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540229888 00000100:00100000:3.0:1789760772.031988:0:10102:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 191 00000100:00100000:3.0:1789760772.032003:0:10102:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5305180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3311:x1876699540229888:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760772.032083:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760772.032136:0:5842:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540229504 00000100:00100000:2.0:1789760772.032140:0:5842:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540229504 00000100:00100000:2.0:1789760772.032174:0:5842:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 409 00000100:00100000:2.0:1789760772.032181:0:5842:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febe00c7450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:6df98d54-eb52-4c5c-8fc9-ee9d74f90f98+19:3312:x1876699540229504:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760772.032230:0:5842:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febe00c7450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:6df98d54-eb52-4c5c-8fc9-ee9d74f90f98+19:3312:x1876699540229504:12345-0@lo:400:kworker.0 Request processed in 49us (148us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760772.032238:0:5842:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 409 00000100:00100000:2.0:1789760772.032310:0:3312:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8feac2fb8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:6df98d54-eb52-4c5c-8fc9-ee9d74f90f98:3312:1876699540229504:0@lo:400:kworker.0 00000100:00100000:2.0:1789760772.032324:0:3312:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8feac38b1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3312:1876699540230016:0@lo:400:kworker.0 00000100:00100000:2.0:1789760772.032357:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760772.032384:0:8229:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540230016 00000100:00100000:2.0:1789760772.032387:0:8229:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540230016 00000100:00100000:2.0:1789760772.032428:0:8229:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 503 00000100:00100000:2.0:1789760772.032436:0:8229:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac38b2a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3312:x1876699540230016:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760772.032468:0:10102:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5305180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3311:x1876699540229888:12345-0@lo:400:kworker.0 Request processed in 468us (5787us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760772.032477:0:10102:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 191 00000100:00100000:2.0:1789760772.032482:0:8229:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac38b2a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3312:x1876699540230016:12345-0@lo:400:kworker.0 Request processed in 46us (122us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760772.032489:0:8229:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 503 00000100:00080000:2.0:1789760772.032514:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789760772.032535:0:3312:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8feac38b1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3312:1876699540230016:0@lo:400:kworker.0 00000100:00100000:3.0:1789760772.035330:0:3311:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8feac38b2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3311:1876699540229888:0@lo:400:kworker.0 00000100:00100000:0.0:1789760772.038195:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2879880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3309:x1876699540229760:12345-0@lo:400:kworker.0 Request processed in 16705us (17009us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760772.038213:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 502 00000100:00100000:3.0:1789760772.039052:0:3309:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8feac38b0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3309:1876699540229760:0@lo:400:kworker.0 00002000:00010000:0.0:1789760772.045039:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27258880 -> 27262976 00010000:00010000:0.0:1789760772.045066:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2f723 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27258880->27262975] (req 27258880->27262975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.045082:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2f723 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27258880->27262975] (req 27258880->27262975) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.045092:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2f723 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27258880->27262975] (req 27258880->27262975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.045102:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760772.045562:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2f723 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27258880->27262975] (req 27258880->27262975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.045574:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2f723 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27258880->27262975] (req 27258880->27262975) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.045599:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2f723 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27258880->27262975] (req 27258880->27262975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.045608:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2f723 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27258880->27262975] (req 27258880->27262975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760772.045770:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973974 is committed 00000100:00100000:0.0:1789760772.046325:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febe7906300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512714752:12345-192.168.203.56@tcp:4:dd.0 Request processed in 56329us (57072us total) trans 4294973974 rc 0/0 00000100:00100000:0.0:1789760772.046340:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6685 00000100:00100000:0.0:1789760772.046353:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 751454780864 time 751554264948 state 0x2 ref 0 00000100:00100000:1.0:1789760772.052003:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760772.052118:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512714880 00000100:00100000:0.0:1789760772.052132:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512714880 00000100:00100000:0.0:1789760772.052551:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 751454780864 time 751554264948 state 0x0 ref 1 00000100:00100000:0.0:1789760772.052578:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 751554525364 time 751554264948 state 0x1 ref 1 00000100:00100000:0.0:1789760772.052669:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 751554525364 time 751616966713 state 0x0 ref 1 00000100:00100000:0.0:1789760772.052686:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febe7904e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512714880:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760772.052796:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27262976, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0xd007 00010000:00010000:0.0:1789760772.052808:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2f72a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.052828:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=751s v=(0 0) 00010000:00010000:0.0:1789760772.052834:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760772.052842:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2f72a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27262976->27267071] (req 27262976->27267071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.052859:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2f72a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27262976->27267071] (req 27262976->27267071) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760772.093997:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973975 is committed 00002000:00010000:0.0:1789760772.094153:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27262976 -> 27267072 00002000:00010000:0.0:1789760772.094159:0:13062:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb blocks from disk: 53255 -> 55303 00010000:00010000:0.0:1789760772.094177:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2f72a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27262976->27267071] (req 27262976->27267071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.094204:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2f72a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27262976->27267071] (req 27262976->27267071) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.094216:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2f72a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27262976->27267071] (req 27262976->27267071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.094224:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760772.094231:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2f72a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27262976->27267071] (req 27262976->27267071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.094241:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2f72a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27262976->27267071] (req 27262976->27267071) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.094267:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2f72a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27262976->27267071] (req 27262976->27267071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.094275:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2f72a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27262976->27267071] (req 27262976->27267071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760772.094400:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febe7904e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512714880:12345-192.168.203.56@tcp:4:dd.0 Request processed in 41712us (42407us total) trans 4294973975 rc 0/0 00000100:00100000:0.0:1789760772.094412:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6686 00000100:00100000:0.0:1789760772.094426:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 751554525364 time 751616966713 state 0x2 ref 0 00000100:00100000:2.0:1789760772.099940:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760772.100075:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512715008 00000100:00100000:3.0:1789760772.100089:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512715008 00000100:00100000:3.0:1789760772.101005:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 751554525364 time 751616966713 state 0x0 ref 1 00000100:00100000:3.0:1789760772.101027:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 751617227129 time 751616966713 state 0x1 ref 1 00000100:00100000:3.0:1789760772.101130:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 751617227129 time 751665428135 state 0x0 ref 1 00000100:00100000:3.0:1789760772.101164:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac38b1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512715008:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760772.101247:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27267072, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0xd807 00010000:00010000:3.0:1789760772.101259:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2f731 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.101284:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=751s v=(0 0) 00010000:00010000:3.0:1789760772.101290:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760772.101298:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2f731 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27267072->27271167] (req 27267072->27271167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.101317:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2f731 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27267072->27271167] (req 27267072->27271167) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760772.124598:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27267072 -> 27271168 00010000:00010000:1.0:1789760772.124620:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2f731 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27267072->27271167] (req 27267072->27271167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760772.124634:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973976 is committed 00010000:00010000:1.0:1789760772.124634:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2f731 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27267072->27271167] (req 27267072->27271167) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.124642:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2f731 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27267072->27271167] (req 27267072->27271167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.124650:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760772.124655:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2f731 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27267072->27271167] (req 27267072->27271167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.125031:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2f731 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27267072->27271167] (req 27267072->27271167) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.125051:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2f731 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27267072->27271167] (req 27267072->27271167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.125056:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2f731 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27267072->27271167] (req 27267072->27271167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760772.125234:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac38b1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512715008:12345-192.168.203.56@tcp:4:dd.0 Request processed in 24075us (25305us total) trans 4294973976 rc 0/0 00000100:00100000:1.0:1789760772.125244:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6687 00000100:00100000:1.0:1789760772.125254:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 751617227129 time 751665428135 state 0x2 ref 0 00000100:00100000:2.0:1789760772.131035:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760772.131178:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512715136 00000100:00100000:0.0:1789760772.131188:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512715136 00000100:00100000:0.0:1789760772.131241:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 751617227129 time 751665428135 state 0x0 ref 1 00000100:00100000:0.0:1789760772.131256:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 751665688551 time 751665428135 state 0x1 ref 1 00000100:00100000:0.0:1789760772.131321:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 751665688551 time 751695619485 state 0x0 ref 1 00000100:00100000:0.0:1789760772.131333:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac38b0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512715136:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760772.131380:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27271168, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0xd807 00010000:00010000:0.0:1789760772.131389:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2f738 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.131410:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=751s v=(0 0) 00010000:00010000:0.0:1789760772.131414:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760772.131420:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2f738 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27271168->27275263] (req 27271168->27275263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.131434:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2f738 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27271168->27275263] (req 27271168->27275263) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760772.156166:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973977 is committed 00002000:00010000:1.0:1789760772.156213:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27271168 -> 27275264 00010000:00010000:1.0:1789760772.156230:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2f738 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27271168->27275263] (req 27271168->27275263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.156245:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2f738 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27271168->27275263] (req 27271168->27275263) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.156255:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2f738 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27271168->27275263] (req 27271168->27275263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.156263:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760772.156268:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2f738 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27271168->27275263] (req 27271168->27275263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.156277:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2f738 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27271168->27275263] (req 27271168->27275263) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.156297:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2f738 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27271168->27275263] (req 27271168->27275263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.156305:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2f738 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27271168->27275263] (req 27271168->27275263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760772.156392:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac38b0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512715136:12345-192.168.203.56@tcp:4:dd.0 Request processed in 25056us (25360us total) trans 4294973977 rc 0/0 00000100:00100000:1.0:1789760772.156402:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6688 00000100:00100000:1.0:1789760772.156420:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 751665688551 time 751695619485 state 0x2 ref 0 00000100:00100000:3.0:1789760772.160339:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760772.160787:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512715264 00000100:00100000:0.0:1789760772.160797:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512715264 00000100:00100000:0.0:1789760772.160851:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 751665688551 time 751695619485 state 0x0 ref 1 00000100:00100000:0.0:1789760772.160866:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 751695879901 time 751695619485 state 0x1 ref 1 00000100:00100000:0.0:1789760772.160897:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 751695879901 time 751725196703 state 0x0 ref 1 00000100:00100000:0.0:1789760772.160910:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febfb374380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512715264:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760772.160955:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27275264, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0xd807 00010000:00010000:0.0:1789760772.160964:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2f73f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.160980:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=751s v=(0 0) 00010000:00010000:0.0:1789760772.160984:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760772.160990:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2f73f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27275264->27279359] (req 27275264->27279359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.161010:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2f73f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27275264->27279359] (req 27275264->27279359) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760772.192257:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27275264 -> 27279360 00000001:00080000:2.0:1789760772.192270:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973978 is committed 00010000:00010000:0.0:1789760772.192276:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2f73f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27275264->27279359] (req 27275264->27279359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.192300:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2f73f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27275264->27279359] (req 27275264->27279359) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.192310:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2f73f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27275264->27279359] (req 27275264->27279359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.192317:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760772.192322:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2f73f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27275264->27279359] (req 27275264->27279359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.192331:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2f73f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27275264->27279359] (req 27275264->27279359) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.192352:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2f73f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27275264->27279359] (req 27275264->27279359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.192359:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2f73f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27275264->27279359] (req 27275264->27279359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760772.192769:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febfb374380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512715264:12345-192.168.203.56@tcp:4:dd.0 Request processed in 31855us (32432us total) trans 4294973978 rc 0/0 00000100:00100000:0.0:1789760772.192786:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6689 00000100:00100000:0.0:1789760772.192796:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 751695879901 time 751725196703 state 0x2 ref 0 00000100:00100000:2.0:1789760772.200233:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760772.200925:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512715392 00000100:00100000:1.0:1789760772.200937:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512715392 00000100:00100000:2.0:1789760772.201171:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 751695879901 time 751725196703 state 0x0 ref 1 00000100:00100000:2.0:1789760772.201185:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 751725457119 time 751725196703 state 0x1 ref 1 00000100:00100000:2.0:1789760772.201215:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 751725457119 time 751765514427 state 0x0 ref 1 00000100:00100000:2.0:1789760772.201230:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac38b2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512715392:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760772.201285:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27279360, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0xd807 00010000:00010000:2.0:1789760772.201293:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2f746 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.201308:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=751s v=(0 0) 00010000:00010000:2.0:1789760772.201312:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760772.201318:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2f746 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27279360->27283455] (req 27279360->27283455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.201332:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2f746 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27279360->27283455] (req 27279360->27283455) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760772.249677:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27279360 -> 27283456 00000001:00080000:3.0:1789760772.249967:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973979 is committed 00010000:00010000:2.0:1789760772.250020:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2f746 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27279360->27283455] (req 27279360->27283455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.250034:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2f746 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27279360->27283455] (req 27279360->27283455) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.250044:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2f746 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27279360->27283455] (req 27279360->27283455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.250052:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760772.250057:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2f746 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27279360->27283455] (req 27279360->27283455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.250067:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2f746 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27279360->27283455] (req 27279360->27283455) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.250085:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2f746 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27279360->27283455] (req 27279360->27283455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.250092:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2f746 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27279360->27283455] (req 27279360->27283455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760772.250229:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac38b2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512715392:12345-192.168.203.56@tcp:4:dd.0 Request processed in 48998us (50003us total) trans 4294973979 rc 0/0 00000100:00100000:2.0:1789760772.250239:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6690 00000100:00100000:2.0:1789760772.250248:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 751725457119 time 751765514427 state 0x2 ref 0 00000100:00100000:3.0:1789760772.253101:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760772.253218:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512715520 00000100:00100000:1.0:1789760772.253228:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512715520 00000100:00100000:1.0:1789760772.253359:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 751725457119 time 751765514427 state 0x0 ref 1 00000100:00100000:1.0:1789760772.253384:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 751765774843 time 751765514427 state 0x1 ref 1 00000100:00100000:1.0:1789760772.253445:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 751765774843 time 751817743771 state 0x0 ref 1 00000100:00100000:1.0:1789760772.253457:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac531ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512715520:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760772.253516:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27283456, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0xd807 00010000:00010000:1.0:1789760772.253525:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2f74d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.253541:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=751s v=(0 0) 00010000:00010000:1.0:1789760772.253545:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760772.253551:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2f74d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27283456->27287551] (req 27283456->27287551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.253565:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2f74d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27283456->27287551] (req 27283456->27287551) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760772.291899:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973980 is committed 00002000:00010000:1.0:1789760772.292025:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27283456 -> 27287552 00010000:00010000:1.0:1789760772.292043:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2f74d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27283456->27287551] (req 27283456->27287551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.292063:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2f74d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27283456->27287551] (req 27283456->27287551) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.292071:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2f74d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27283456->27287551] (req 27283456->27287551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.292079:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760772.292084:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2f74d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27283456->27287551] (req 27283456->27287551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.292093:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2f74d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27283456->27287551] (req 27283456->27287551) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.292120:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2f74d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27283456->27287551] (req 27283456->27287551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.292128:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2f74d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27283456->27287551] (req 27283456->27287551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760772.292256:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac531ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512715520:12345-192.168.203.56@tcp:4:dd.0 Request processed in 38796us (39158us total) trans 4294973980 rc 0/0 00000100:00100000:1.0:1789760772.292266:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6691 00000100:00100000:1.0:1789760772.292276:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 751765774843 time 751817743771 state 0x2 ref 0 00000100:00100000:0.0:1789760772.297279:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760772.297361:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512715648 00000100:00100000:2.0:1789760772.297371:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512715648 00000100:00100000:2.0:1789760772.297424:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 751765774843 time 751817743771 state 0x0 ref 1 00000100:00100000:2.0:1789760772.297452:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 751818004187 time 751817743771 state 0x1 ref 1 00000100:00100000:2.0:1789760772.297491:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 751818004187 time 751861791010 state 0x0 ref 1 00000100:00100000:2.0:1789760772.297512:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2879180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512715648:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760772.297557:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27287552, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0xd807 00010000:00010000:2.0:1789760772.297565:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2f754 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.297580:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=751s v=(0 0) 00010000:00010000:2.0:1789760772.297584:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760772.297591:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2f754 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27287552->27291647] (req 27287552->27291647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.297605:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2f754 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27287552->27291647] (req 27287552->27291647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760772.339625:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27287552 -> 27291648 00010000:00010000:2.0:1789760772.339647:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2f754 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27287552->27291647] (req 27287552->27291647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.339667:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2f754 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27287552->27291647] (req 27287552->27291647) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.339678:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2f754 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27287552->27291647] (req 27287552->27291647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.339685:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760772.339691:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2f754 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27287552->27291647] (req 27287552->27291647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.339723:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2f754 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27287552->27291647] (req 27287552->27291647) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.339742:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2f754 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27287552->27291647] (req 27287552->27291647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.339750:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2f754 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27287552->27291647] (req 27287552->27291647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760772.339864:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2879180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2358:x1876699512715648:12345-192.168.203.56@tcp:4:dd.0 Request processed in 42349us (42589us total) trans 4294973981 rc 0/0 00000100:00100000:2.0:1789760772.339874:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6692 00000100:00100000:2.0:1789760772.339883:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 751818004187 time 751861791010 state 0x2 ref 0 00000001:00080000:2.0:1789760772.339923:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973981 is committed 00000100:00100000:2.0:1789760772.345002:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760772.345322:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512715776 00000100:00100000:3.0:1789760772.345332:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512715776 00000100:00100000:3.0:1789760772.345392:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 751818004187 time 751861791010 state 0x0 ref 1 00000100:00100000:3.0:1789760772.345407:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 751862051426 time 751861791010 state 0x1 ref 1 00000100:00100000:3.0:1789760772.345440:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 751862051426 time 751909739244 state 0x0 ref 1 00000100:00100000:3.0:1789760772.345454:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2fb8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512715776:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760772.345517:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27291648, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0xd807 00010000:00010000:3.0:1789760772.345526:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2f75b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.345545:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=751s v=(0 0) 00010000:00010000:3.0:1789760772.345549:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760772.345557:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2f75b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27291648->27295743] (req 27291648->27295743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.345571:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2f75b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27291648->27295743] (req 27291648->27295743) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760772.379565:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27291648 -> 27295744 00010000:00010000:3.0:1789760772.379586:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2f75b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27291648->27295743] (req 27291648->27295743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760772.379595:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973982 is committed 00010000:00010000:3.0:1789760772.379598:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2f75b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27291648->27295743] (req 27291648->27295743) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.379606:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2f75b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27291648->27295743] (req 27291648->27295743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.379613:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760772.379618:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2f75b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27291648->27295743] (req 27291648->27295743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.379628:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2f75b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27291648->27295743] (req 27291648->27295743) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.379648:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2f75b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27291648->27295743] (req 27291648->27295743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.379655:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2f75b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27291648->27295743] (req 27291648->27295743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760772.379802:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2fb8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512715776:12345-192.168.203.56@tcp:4:dd.0 Request processed in 34345us (34803us total) trans 4294973982 rc 0/0 00000100:00100000:3.0:1789760772.379812:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6693 00000100:00100000:3.0:1789760772.379822:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 751862051426 time 751909739244 state 0x2 ref 0 00000100:00100000:0.0:1789760772.384203:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760772.384421:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512715904 00000100:00100000:1.0:1789760772.384431:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512715904 00000100:00100000:1.0:1789760772.384495:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 751862051426 time 751909739244 state 0x0 ref 1 00000100:00100000:1.0:1789760772.384511:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 751909999660 time 751909739244 state 0x1 ref 1 00000100:00100000:1.0:1789760772.384541:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 751909999660 time 751948840230 state 0x0 ref 1 00000100:00100000:1.0:1789760772.384553:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2879500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512715904:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760772.384600:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27295744, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0xd807 00010000:00010000:1.0:1789760772.384609:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2f762 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.384624:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=751s v=(0 0) 00010000:00010000:1.0:1789760772.384629:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760772.384635:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2f762 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27295744->27299839] (req 27295744->27299839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.384652:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2f762 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27295744->27299839] (req 27295744->27299839) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760772.417743:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973983 is committed 00002000:00010000:0.0:1789760772.417868:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27295744 -> 27299840 00010000:00010000:0.0:1789760772.417884:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2f762 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27295744->27299839] (req 27295744->27299839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.417897:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2f762 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27295744->27299839] (req 27295744->27299839) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.417905:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2f762 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27295744->27299839] (req 27295744->27299839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.417912:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760772.417917:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2f762 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27295744->27299839] (req 27295744->27299839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.417939:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2f762 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27295744->27299839] (req 27295744->27299839) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.417959:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2f762 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27295744->27299839] (req 27295744->27299839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.417967:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2f762 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27295744->27299839] (req 27295744->27299839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760772.418062:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2879500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512715904:12345-192.168.203.56@tcp:4:dd.0 Request processed in 33507us (33865us total) trans 4294973983 rc 0/0 00000100:00100000:0.0:1789760772.418072:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6694 00000100:00100000:0.0:1789760772.418082:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 751909999660 time 751948840230 state 0x2 ref 0 00000100:00100000:0.0:1789760772.422052:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760772.422172:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512716032 00000100:00100000:2.0:1789760772.422183:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512716032 00000100:00100000:2.0:1789760772.422233:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 751909999660 time 751948840230 state 0x0 ref 1 00000100:00100000:2.0:1789760772.422248:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 751949100646 time 751948840230 state 0x1 ref 1 00000100:00100000:2.0:1789760772.422283:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 751949100646 time 751986582422 state 0x0 ref 1 00000100:00100000:2.0:1789760772.422297:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febea4b5850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512716032:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760772.422347:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27299840, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0xd807 00010000:00010000:2.0:1789760772.422356:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2f769 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.422372:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=751s v=(0 0) 00010000:00010000:2.0:1789760772.422377:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760772.422384:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2f769 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27299840->27303935] (req 27299840->27303935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.422397:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2f769 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27299840->27303935] (req 27299840->27303935) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.466960:0:5836:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:0.0:1789760772.473756:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973984 is committed 00002000:00010000:3.0:1789760772.473882:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27299840 -> 27303936 00010000:00010000:3.0:1789760772.473895:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2f769 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27299840->27303935] (req 27299840->27303935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.473905:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2f769 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27299840->27303935] (req 27299840->27303935) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.473911:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2f769 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27299840->27303935] (req 27299840->27303935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.473916:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760772.473919:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2f769 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27299840->27303935] (req 27299840->27303935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.473926:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2f769 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27299840->27303935] (req 27299840->27303935) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.473944:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2f769 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27299840->27303935] (req 27299840->27303935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.473949:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2f769 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27299840->27303935] (req 27299840->27303935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760772.474018:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febea4b5850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512716032:12345-192.168.203.56@tcp:4:dd.0 Request processed in 51721us (51972us total) trans 4294973984 rc 0/0 00000100:00100000:3.0:1789760772.474025:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6695 00000100:00100000:3.0:1789760772.474032:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 751949100646 time 751986582422 state 0x2 ref 0 00000100:00100000:2.0:1789760772.477020:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760772.477060:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512716160 00000100:00100000:3.0:1789760772.477069:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512716160 00000100:00100000:3.0:1789760772.477196:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 751949100646 time 751986582422 state 0x0 ref 1 00000100:00100000:3.0:1789760772.477214:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 751986842838 time 751986582422 state 0x1 ref 1 00000100:00100000:3.0:1789760772.477237:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 751986842838 time 752041536751 state 0x0 ref 1 00000100:00100000:3.0:1789760772.477250:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2fba680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512716160:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760772.477299:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27303936, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0xd807 00010000:00010000:3.0:1789760772.477308:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2f770 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.477323:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=752s v=(0 0) 00010000:00010000:3.0:1789760772.477329:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760772.477336:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2f770 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27303936->27308031] (req 27303936->27308031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.477350:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2f770 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27303936->27308031] (req 27303936->27308031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760772.504265:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27303936 -> 27308032 00010000:00010000:2.0:1789760772.504286:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2f770 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27303936->27308031] (req 27303936->27308031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.504299:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2f770 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27303936->27308031] (req 27303936->27308031) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.504309:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2f770 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27303936->27308031] (req 27303936->27308031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.504316:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760772.504322:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2f770 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27303936->27308031] (req 27303936->27308031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.504331:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2f770 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27303936->27308031] (req 27303936->27308031) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.504350:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2f770 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27303936->27308031] (req 27303936->27308031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.504356:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2f770 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27303936->27308031] (req 27303936->27308031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760772.504446:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2fba680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2358:x1876699512716160:12345-192.168.203.56@tcp:4:dd.0 Request processed in 27194us (27429us total) trans 4294973985 rc 0/0 00000100:00100000:2.0:1789760772.504456:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6696 00000100:00100000:2.0:1789760772.504465:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 751986842838 time 752041536751 state 0x2 ref 0 00000001:00080000:0.0:1789760772.504533:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973985 is committed 00000100:00100000:1.0:1789760772.508612:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760772.510053:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512716288 00000100:00100000:0.0:1789760772.510067:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512716288 00000100:00100000:0.0:1789760772.510137:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 751986842838 time 752041536751 state 0x0 ref 1 00000100:00100000:0.0:1789760772.510155:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 752041797167 time 752041536751 state 0x1 ref 1 00000100:00100000:0.0:1789760772.510238:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 752041797167 time 752074537428 state 0x0 ref 1 00000100:00100000:0.0:1789760772.510252:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febe7905f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512716288:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760772.510304:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27308032, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0xd807 00010000:00010000:0.0:1789760772.510313:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2f777 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.510328:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=752s v=(0 0) 00010000:00010000:0.0:1789760772.510333:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760772.510339:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2f777 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27308032->27312127] (req 27308032->27312127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.510353:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2f777 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27308032->27312127] (req 27308032->27312127) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760772.536233:0:3309:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1789760772.536255:0:3309:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8feac287bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3309:1876699540230272:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1789760772.536319:0:3309:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760772.536745:0:6707:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540230272 00000100:00100000:2.0:1789760772.536752:0:6707:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540230272 00000100:00100000:2.0:1789760772.536810:0:6707:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 209 00000100:00100000:2.0:1789760772.536818:0:6707:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2879f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3309:x1876699540230272:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1789760772.536850:0:6707:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3859808256 avail 3857711104 00000100:00100000:2.0:1789760772.536903:0:6707:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2879f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3309:x1876699540230272:12345-0@lo:13:osp-pre-1-0.0 Request processed in 86us (584us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760772.536911:0:6707:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 209 00000100:00100000:0.0:1789760772.536950:0:3309:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8feac287bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3309:1876699540230272:0@lo:13:osp-pre-1-0.0 00002000:00010000:0.0:1789760772.551815:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27308032 -> 27312128 00002000:00010000:0.0:1789760772.551824:0:6710:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760771 -> 1789760772 00002000:00010000:0.0:1789760772.551827:0:6710:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760771 -> 1789760772 00010000:00010000:0.0:1789760772.551840:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2f777 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27308032->27312127] (req 27308032->27312127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.551853:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2f777 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27308032->27312127] (req 27308032->27312127) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.551862:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2f777 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27308032->27312127] (req 27308032->27312127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.551868:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760772.551873:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2f777 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27308032->27312127] (req 27308032->27312127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.551888:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2f777 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27308032->27312127] (req 27308032->27312127) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.551906:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2f777 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27308032->27312127] (req 27308032->27312127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.551914:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2f777 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27308032->27312127] (req 27308032->27312127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760772.552012:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febe7905f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2359:x1876699512716288:12345-192.168.203.56@tcp:4:dd.0 Request processed in 41759us (43407us total) trans 4294973986 rc 0/0 00000100:00100000:0.0:1789760772.552029:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6697 00000100:00100000:0.0:1789760772.552038:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 752041797167 time 752074537428 state 0x2 ref 0 00000001:00080000:0.0:1789760772.552078:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973986 is committed 00000100:00100000:3.0:1789760772.556019:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760772.556100:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512716416 00000100:00100000:2.0:1789760772.556117:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512716416 00000100:00100000:2.0:1789760772.556171:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 752041797167 time 752074537428 state 0x0 ref 1 00000100:00100000:2.0:1789760772.556186:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 752074797844 time 752074537428 state 0x1 ref 1 00000100:00100000:2.0:1789760772.556247:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 752074797844 time 752120546428 state 0x0 ref 1 00000100:00100000:2.0:1789760772.556261:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac531ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512716416:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760772.556308:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27312128, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0xd807 00010000:00010000:2.0:1789760772.556317:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2f77e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.556333:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=752s v=(0 0) 00010000:00010000:2.0:1789760772.556338:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760772.556345:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2f77e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27312128->27316223] (req 27312128->27316223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.556359:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2f77e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27312128->27316223] (req 27312128->27316223) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760772.601272:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973987 is committed 00002000:00010000:2.0:1789760772.601566:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27312128 -> 27316224 00010000:00010000:2.0:1789760772.601590:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2f77e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27312128->27316223] (req 27312128->27316223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.601603:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2f77e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27312128->27316223] (req 27312128->27316223) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.601612:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2f77e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27312128->27316223] (req 27312128->27316223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.601619:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760772.601624:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2f77e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27312128->27316223] (req 27312128->27316223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.601632:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2f77e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27312128->27316223] (req 27312128->27316223) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.601652:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2f77e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27312128->27316223] (req 27312128->27316223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.601659:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2f77e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27312128->27316223] (req 27312128->27316223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760772.601796:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac531ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512716416:12345-192.168.203.56@tcp:4:dd.0 Request processed in 45532us (45779us total) trans 4294973987 rc 0/0 00000100:00100000:2.0:1789760772.601812:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6698 00000100:00100000:2.0:1789760772.601821:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 752074797844 time 752120546428 state 0x2 ref 0 00000100:00100000:2.0:1789760772.615431:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760772.615578:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512716544 00000100:00100000:3.0:1789760772.615588:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512716544 00000100:00100000:3.0:1789760772.615639:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 752074797844 time 752120546428 state 0x0 ref 1 00000100:00100000:3.0:1789760772.615654:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 752120806844 time 752120546428 state 0x1 ref 1 00000100:00100000:3.0:1789760772.615684:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 752120806844 time 752179984406 state 0x0 ref 1 00000100:00100000:3.0:1789760772.615740:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2fb8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512716544:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760772.615787:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27316224, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0xd807 00010000:00010000:3.0:1789760772.615796:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2f785 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.615812:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=752s v=(0 0) 00010000:00010000:3.0:1789760772.615816:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760772.615823:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2f785 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27316224->27320319] (req 27316224->27320319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.615836:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2f785 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27316224->27320319] (req 27316224->27320319) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760772.660529:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973988 is committed 00002000:00010000:2.0:1789760772.660793:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27316224 -> 27320320 00010000:00010000:2.0:1789760772.660809:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2f785 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27316224->27320319] (req 27316224->27320319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.660822:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2f785 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27316224->27320319] (req 27316224->27320319) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.660831:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2f785 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27316224->27320319] (req 27316224->27320319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.660836:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760772.660840:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2f785 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27316224->27320319] (req 27316224->27320319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.660846:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2f785 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27316224->27320319] (req 27316224->27320319) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.660860:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2f785 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27316224->27320319] (req 27316224->27320319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.660865:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2f785 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27316224->27320319] (req 27316224->27320319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760772.660947:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2fb8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512716544:12345-192.168.203.56@tcp:4:dd.0 Request processed in 45248us (45520us total) trans 4294973988 rc 0/0 00000100:00100000:2.0:1789760772.660958:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6699 00000100:00100000:2.0:1789760772.660969:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 752120806844 time 752179984406 state 0x2 ref 0 00000100:00100000:1.0:1789760772.663647:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760772.664012:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512716672 00000100:00100000:0.0:1789760772.664022:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512716672 00000100:00100000:0.0:1789760772.664074:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 752120806844 time 752179984406 state 0x0 ref 1 00000100:00100000:0.0:1789760772.664090:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 752180244822 time 752179984406 state 0x1 ref 1 00000100:00100000:0.0:1789760772.664161:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 752180244822 time 752228460129 state 0x0 ref 1 00000100:00100000:0.0:1789760772.664174:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febe7905880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512716672:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760772.664228:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27320320, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0xd807 00010000:00010000:0.0:1789760772.664237:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2f78c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.664253:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=752s v=(0 0) 00010000:00010000:0.0:1789760772.664258:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760772.664264:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2f78c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27320320->27324415] (req 27320320->27324415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.664277:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2f78c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27320320->27324415] (req 27320320->27324415) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760772.696400:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973989 is committed 00002000:00010000:2.0:1789760772.696628:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27320320 -> 27324416 00010000:00010000:2.0:1789760772.696646:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2f78c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27320320->27324415] (req 27320320->27324415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.696660:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2f78c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27320320->27324415] (req 27320320->27324415) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.696673:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2f78c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27320320->27324415] (req 27320320->27324415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.696680:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760772.696685:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2f78c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27320320->27324415] (req 27320320->27324415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.696711:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2f78c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27320320->27324415] (req 27320320->27324415) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.696733:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2f78c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27320320->27324415] (req 27320320->27324415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.696755:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2f78c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27320320->27324415] (req 27320320->27324415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760772.696992:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febe7905880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512716672:12345-192.168.203.56@tcp:4:dd.0 Request processed in 32814us (33347us total) trans 4294973989 rc 0/0 00000100:00100000:2.0:1789760772.697003:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6700 00000100:00100000:2.0:1789760772.697013:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 752180244822 time 752228460129 state 0x2 ref 0 00000100:00100000:0.0:1789760772.702396:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760772.702485:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512716800 00000100:00100000:2.0:1789760772.702495:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512716800 00000100:00100000:2.0:1789760772.702568:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 752180244822 time 752228460129 state 0x0 ref 1 00000100:00100000:2.0:1789760772.702584:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 752228720545 time 752228460129 state 0x1 ref 1 00000100:00100000:2.0:1789760772.702611:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 752228720545 time 752266911433 state 0x0 ref 1 00000100:00100000:2.0:1789760772.702622:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac287bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512716800:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760772.702678:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27324416, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0xd807 00010000:00010000:2.0:1789760772.702686:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2f793 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.703051:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=752s v=(0 0) 00010000:00010000:2.0:1789760772.703056:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760772.703063:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2f793 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27324416->27328511] (req 27324416->27328511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.703083:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2f793 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27324416->27328511] (req 27324416->27328511) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760772.728937:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973990 is committed 00002000:00010000:0.0:1789760772.729048:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27324416 -> 27328512 00010000:00010000:0.0:1789760772.729064:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2f793 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27324416->27328511] (req 27324416->27328511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.729076:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2f793 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27324416->27328511] (req 27324416->27328511) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.729085:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2f793 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27324416->27328511] (req 27324416->27328511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.729093:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760772.729098:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2f793 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27324416->27328511] (req 27324416->27328511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.729123:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2f793 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27324416->27328511] (req 27324416->27328511) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.729142:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2f793 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27324416->27328511] (req 27324416->27328511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.729151:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2f793 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27324416->27328511] (req 27324416->27328511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760772.729241:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac287bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512716800:12345-192.168.203.56@tcp:4:dd.0 Request processed in 26615us (26851us total) trans 4294973990 rc 0/0 00000100:00100000:0.0:1789760772.729251:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6701 00000100:00100000:0.0:1789760772.729260:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 752228720545 time 752266911433 state 0x2 ref 0 00000100:00100000:3.0:1789760772.733318:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760772.733383:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512716928 00000100:00100000:2.0:1789760772.733392:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512716928 00000100:00100000:2.0:1789760772.733446:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 752228720545 time 752266911433 state 0x0 ref 1 00000100:00100000:2.0:1789760772.733462:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 752267171849 time 752266911433 state 0x1 ref 1 00000100:00100000:2.0:1789760772.733494:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 752267171849 time 752297793894 state 0x0 ref 1 00000100:00100000:2.0:1789760772.733507:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac531f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512716928:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760772.733554:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27328512, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0xd807 00010000:00010000:2.0:1789760772.733563:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2f79a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.733580:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=752s v=(0 0) 00010000:00010000:2.0:1789760772.733585:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760772.733591:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2f79a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27328512->27332607] (req 27328512->27332607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.733606:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2f79a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27328512->27332607] (req 27328512->27332607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760772.770796:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27328512 -> 27332608 00010000:00010000:0.0:1789760772.770825:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2f79a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27328512->27332607] (req 27328512->27332607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.770838:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2f79a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27328512->27332607] (req 27328512->27332607) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.770847:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2f79a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27328512->27332607] (req 27328512->27332607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.770855:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760772.770861:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2f79a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27328512->27332607] (req 27328512->27332607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.770870:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2f79a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27328512->27332607] (req 27328512->27332607) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.770887:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2f79a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27328512->27332607] (req 27328512->27332607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.770894:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2f79a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27328512->27332607] (req 27328512->27332607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760772.770986:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac531f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2359:x1876699512716928:12345-192.168.203.56@tcp:4:dd.0 Request processed in 37475us (37674us total) trans 4294973991 rc 0/0 00000100:00100000:0.0:1789760772.772847:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6702 00000100:00100000:0.0:1789760772.772868:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 752267171849 time 752297793894 state 0x2 ref 0 00000001:00080000:2.0:1789760772.772989:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973991 is committed 00000100:00100000:2.0:1789760772.775612:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760772.775760:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512717056 00000100:00100000:0.0:1789760772.775770:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512717056 00000100:00100000:0.0:1789760772.775833:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 752267171849 time 752297793894 state 0x0 ref 1 00000100:00100000:0.0:1789760772.775847:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 752298054310 time 752297793894 state 0x1 ref 1 00000100:00100000:0.0:1789760772.775877:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 752298054310 time 752340176747 state 0x0 ref 1 00000100:00100000:0.0:1789760772.775889:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2fb8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512717056:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760772.775937:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27332608, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0xd807 00010000:00010000:0.0:1789760772.775945:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2f7a1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.775961:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=752s v=(0 0) 00010000:00010000:0.0:1789760772.775966:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760772.775972:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2f7a1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27332608->27336703] (req 27332608->27336703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.775986:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2f7a1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27332608->27336703] (req 27332608->27336703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760772.809028:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27332608 -> 27336704 00010000:00010000:0.0:1789760772.809047:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2f7a1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27332608->27336703] (req 27332608->27336703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.809059:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2f7a1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27332608->27336703] (req 27332608->27336703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.809068:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2f7a1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27332608->27336703] (req 27332608->27336703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.809075:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760772.809080:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2f7a1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27332608->27336703] (req 27332608->27336703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.809089:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2f7a1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27332608->27336703] (req 27332608->27336703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.809121:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2f7a1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27332608->27336703] (req 27332608->27336703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.809130:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2f7a1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27332608->27336703] (req 27332608->27336703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760772.809226:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2fb8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2360:x1876699512717056:12345-192.168.203.56@tcp:4:dd.0 Request processed in 33334us (33617us total) trans 4294973992 rc 0/0 00000100:00100000:0.0:1789760772.809235:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6703 00000100:00100000:0.0:1789760772.809244:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 752298054310 time 752340176747 state 0x2 ref 0 00000001:00080000:3.0:1789760772.809316:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973992 is committed 00000100:00100000:3.0:1789760772.812445:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760772.813162:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512717184 00000100:00100000:0.0:1789760772.813172:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512717184 00000100:00100000:0.0:1789760772.813225:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 752298054310 time 752340176747 state 0x0 ref 1 00000100:00100000:0.0:1789760772.813240:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 752340437163 time 752340176747 state 0x1 ref 1 00000100:00100000:0.0:1789760772.813307:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 752340437163 time 752377605972 state 0x0 ref 1 00000100:00100000:0.0:1789760772.813320:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac531df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512717184:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760772.813367:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27336704, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0xd807 00010000:00010000:0.0:1789760772.813375:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f7a8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.813390:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=752s v=(0 0) 00010000:00010000:0.0:1789760772.813395:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760772.813401:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f7a8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27336704->27340799] (req 27336704->27340799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.813413:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f7a8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27336704->27340799] (req 27336704->27340799) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760772.852178:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973993 is committed 00002000:00010000:0.0:1789760772.852356:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27336704 -> 27340800 00010000:00010000:0.0:1789760772.852371:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f7a8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27336704->27340799] (req 27336704->27340799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.852383:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f7a8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27336704->27340799] (req 27336704->27340799) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.852392:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f7a8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27336704->27340799] (req 27336704->27340799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.852399:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760772.852404:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f7a8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27336704->27340799] (req 27336704->27340799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.852414:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f7a8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27336704->27340799] (req 27336704->27340799) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.852434:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f7a8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27336704->27340799] (req 27336704->27340799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.852447:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f7a8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27336704->27340799] (req 27336704->27340799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760772.852547:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac531df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512717184:12345-192.168.203.56@tcp:4:dd.0 Request processed in 39225us (40109us total) trans 4294973993 rc 0/0 00000100:00100000:0.0:1789760772.852568:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6704 00000100:00100000:0.0:1789760772.852579:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 752340437163 time 752377605972 state 0x2 ref 0 00000100:00100000:2.0:1789760772.857935:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760772.858413:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512717312 00000100:00100000:0.0:1789760772.858423:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512717312 00000100:00100000:0.0:1789760772.858639:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 752340437163 time 752377605972 state 0x0 ref 1 00000100:00100000:0.0:1789760772.858653:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 752377866388 time 752377605972 state 0x1 ref 1 00000100:00100000:0.0:1789760772.858685:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 752377866388 time 752422984729 state 0x0 ref 1 00000100:00100000:0.0:1789760772.858779:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac28a2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512717312:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760772.858831:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27340800, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0xd807 00010000:00010000:0.0:1789760772.858839:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f7af lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.858855:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=752s v=(0 0) 00010000:00010000:0.0:1789760772.858859:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760772.858866:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f7af lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27340800->27344895] (req 27340800->27344895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.858884:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f7af lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27340800->27344895] (req 27340800->27344895) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760772.901206:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973994 is committed 00002000:00010000:0.0:1789760772.901669:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27340800 -> 27344896 00010000:00010000:0.0:1789760772.901684:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f7af lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27340800->27344895] (req 27340800->27344895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.901716:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f7af lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27340800->27344895] (req 27340800->27344895) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.901726:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f7af lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27340800->27344895] (req 27340800->27344895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.901734:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760772.901738:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f7af lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27340800->27344895] (req 27340800->27344895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.901746:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f7af lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27340800->27344895] (req 27340800->27344895) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.901766:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f7af lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27340800->27344895] (req 27340800->27344895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.901835:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f7af lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27340800->27344895] (req 27340800->27344895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760772.901983:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac28a2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512717312:12345-192.168.203.56@tcp:4:dd.0 Request processed in 43283us (44053us total) trans 4294973994 rc 0/0 00000100:00100000:0.0:1789760772.901995:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6705 00000100:00100000:0.0:1789760772.902003:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 752377866388 time 752422984729 state 0x2 ref 0 00000100:00100000:2.0:1789760772.908989:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760772.909061:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512717440 00000100:00100000:0.0:1789760772.909070:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512717440 00000100:00100000:0.0:1789760772.909132:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 752377866388 time 752422984729 state 0x0 ref 1 00000100:00100000:0.0:1789760772.909147:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 752423245145 time 752422984729 state 0x1 ref 1 00000100:00100000:0.0:1789760772.909235:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 752423245145 time 752473534421 state 0x0 ref 1 00000100:00100000:0.0:1789760772.909249:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac28a0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512717440:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760772.909310:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27344896, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0xd807 00010000:00010000:0.0:1789760772.909320:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f7b6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.909335:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=752s v=(0 0) 00010000:00010000:0.0:1789760772.909339:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760772.909345:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f7b6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27344896->27348991] (req 27344896->27348991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.909362:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f7b6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27344896->27348991] (req 27344896->27348991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760772.944342:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27344896 -> 27348992 00010000:00010000:0.0:1789760772.944362:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f7b6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27344896->27348991] (req 27344896->27348991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.944374:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f7b6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27344896->27348991] (req 27344896->27348991) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760772.944375:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973995 is committed 00010000:00010000:0.0:1789760772.944384:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f7b6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27344896->27348991] (req 27344896->27348991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.944390:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760772.944395:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f7b6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27344896->27348991] (req 27344896->27348991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.944404:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f7b6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27344896->27348991] (req 27344896->27348991) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.944423:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f7b6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27344896->27348991] (req 27344896->27348991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.944431:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f7b6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27344896->27348991] (req 27344896->27348991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760772.944514:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac28a0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512717440:12345-192.168.203.56@tcp:4:dd.0 Request processed in 35263us (35527us total) trans 4294973995 rc 0/0 00000100:00100000:0.0:1789760772.944528:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6706 00000100:00100000:0.0:1789760772.944537:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 752423245145 time 752473534421 state 0x2 ref 0 00000100:00100000:3.0:1789760772.948798:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760772.949211:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512717568 00000100:00100000:0.0:1789760772.949221:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512717568 00000100:00100000:0.0:1789760772.949274:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 752423245145 time 752473534421 state 0x0 ref 1 00000100:00100000:0.0:1789760772.949289:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 752473794837 time 752473534421 state 0x1 ref 1 00000100:00100000:0.0:1789760772.949356:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 752473794837 time 752513654336 state 0x0 ref 1 00000100:00100000:0.0:1789760772.949370:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac531c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512717568:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760772.949420:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27348992, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0xd807 00010000:00010000:0.0:1789760772.949429:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2f7bd lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.949445:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=752s v=(0 0) 00010000:00010000:0.0:1789760772.949449:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760772.949455:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2f7bd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27348992->27353087] (req 27348992->27353087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.949468:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2f7bd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27348992->27353087] (req 27348992->27353087) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760772.984079:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973996 is committed 00002000:00010000:1.0:1789760772.984366:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27348992 -> 27353088 00010000:00010000:1.0:1789760772.984383:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2f7bd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27348992->27353087] (req 27348992->27353087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.984397:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2f7bd lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27348992->27353087] (req 27348992->27353087) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.984407:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2f7bd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27348992->27353087] (req 27348992->27353087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.984416:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760772.984420:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2f7bd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27348992->27353087] (req 27348992->27353087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.984430:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2f7bd lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27348992->27353087] (req 27348992->27353087) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.984452:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2f7bd lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27348992->27353087] (req 27348992->27353087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.984460:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2f7bd lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27348992->27353087] (req 27348992->27353087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760772.984605:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac531c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512717568:12345-192.168.203.56@tcp:4:dd.0 Request processed in 35232us (35809us total) trans 4294973996 rc 0/0 00000100:00100000:1.0:1789760772.984615:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6707 00000100:00100000:1.0:1789760772.984625:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 752473794837 time 752513654336 state 0x2 ref 0 00000100:00100000:2.0:1789760772.990942:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760772.991020:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512717696 00000100:00100000:0.0:1789760772.991029:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512717696 00000100:00100000:0.0:1789760772.991083:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 752473794837 time 752513654336 state 0x0 ref 1 00000100:00100000:0.0:1789760772.991116:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 752513914752 time 752513654336 state 0x1 ref 1 00000100:00100000:0.0:1789760772.991146:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 752513914752 time 752555445554 state 0x0 ref 1 00000100:00100000:0.0:1789760772.991158:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac28a3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512717696:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760772.991214:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27353088, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0xd807 00010000:00010000:0.0:1789760772.991224:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f7c4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.991245:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=752s v=(0 0) 00010000:00010000:0.0:1789760772.991250:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760772.991257:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f7c4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27353088->27357183] (req 27353088->27357183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.991271:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f7c4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27353088->27357183] (req 27353088->27357183) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760773.030204:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973997 is committed 00002000:00010000:0.0:1789760773.030366:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27353088 -> 27357184 00010000:00010000:0.0:1789760773.030382:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f7c4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27353088->27357183] (req 27353088->27357183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.030395:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f7c4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27353088->27357183] (req 27353088->27357183) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.030405:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f7c4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27353088->27357183] (req 27353088->27357183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.030414:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760773.030419:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f7c4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27353088->27357183] (req 27353088->27357183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.030429:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f7c4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27353088->27357183] (req 27353088->27357183) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.030448:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f7c4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27353088->27357183] (req 27353088->27357183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.030456:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f7c4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27353088->27357183] (req 27353088->27357183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760773.030545:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac28a3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512717696:12345-192.168.203.56@tcp:4:dd.0 Request processed in 39383us (39608us total) trans 4294973997 rc 0/0 00000100:00100000:0.0:1789760773.030556:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6708 00000100:00100000:0.0:1789760773.030566:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 752513914752 time 752555445554 state 0x2 ref 0 00000100:00100000:2.0:1789760773.035351:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760773.035464:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512717824 00000100:00100000:0.0:1789760773.035473:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512717824 00000100:00100000:0.0:1789760773.035595:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 752513914752 time 752555445554 state 0x0 ref 1 00000100:00100000:0.0:1789760773.035618:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 752555705970 time 752555445554 state 0x1 ref 1 00000100:00100000:0.0:1789760773.035648:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 752555705970 time 752599947864 state 0x0 ref 1 00000100:00100000:0.0:1789760773.035664:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febffb9e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512717824:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760773.035754:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27357184, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0xd807 00010000:00010000:0.0:1789760773.035764:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f7cb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.035781:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=752s v=(0 0) 00010000:00010000:0.0:1789760773.035785:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760773.035791:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f7cb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27357184->27361279] (req 27357184->27361279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.035805:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f7cb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27357184->27361279] (req 27357184->27361279) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1789760773.036546:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:2.0:1789760773.066863:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973998 is committed 00002000:00010000:3.0:1789760773.067020:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27357184 -> 27361280 00010000:00010000:3.0:1789760773.067036:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f7cb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27357184->27361279] (req 27357184->27361279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.067049:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f7cb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27357184->27361279] (req 27357184->27361279) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.067058:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f7cb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27357184->27361279] (req 27357184->27361279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.067065:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760773.067069:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f7cb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27357184->27361279] (req 27357184->27361279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.067078:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f7cb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27357184->27361279] (req 27357184->27361279) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.067219:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f7cb lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27357184->27361279] (req 27357184->27361279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.067229:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f7cb lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27357184->27361279] (req 27357184->27361279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760773.067350:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febffb9e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512717824:12345-192.168.203.56@tcp:4:dd.0 Request processed in 31684us (32003us total) trans 4294973998 rc 0/0 00000100:00100000:3.0:1789760773.067361:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6709 00000100:00100000:3.0:1789760773.067371:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 752555705970 time 752599947864 state 0x2 ref 0 00000100:00100000:2.0:1789760773.075073:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760773.076089:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512717952 00000100:00100000:1.0:1789760773.076108:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512717952 00000100:00100000:1.0:1789760773.076161:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 752555705970 time 752599947864 state 0x0 ref 1 00000100:00100000:1.0:1789760773.076176:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 752600208280 time 752599947864 state 0x1 ref 1 00000100:00100000:1.0:1789760773.076262:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 752600208280 time 752640560184 state 0x0 ref 1 00000100:00100000:1.0:1789760773.076283:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febffb9f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512717952:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760773.076327:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27361280, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0xd807 00010000:00010000:1.0:1789760773.076336:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2f7d2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.076351:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=752s v=(0 0) 00010000:00010000:1.0:1789760773.076355:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760773.076360:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2f7d2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27361280->27365375] (req 27361280->27365375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.076371:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2f7d2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27361280->27365375] (req 27361280->27365375) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760773.115612:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27361280 -> 27365376 00010000:00010000:0.0:1789760773.115634:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2f7d2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27361280->27365375] (req 27361280->27365375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.115653:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2f7d2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27361280->27365375] (req 27361280->27365375) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.115661:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2f7d2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27361280->27365375] (req 27361280->27365375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.115669:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760773.115673:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2f7d2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27361280->27365375] (req 27361280->27365375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.115684:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2f7d2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27361280->27365375] (req 27361280->27365375) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760773.115717:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973999 is committed 00010000:00010000:0.0:1789760773.115721:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2f7d2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27361280->27365375] (req 27361280->27365375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.115729:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2f7d2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27361280->27365375] (req 27361280->27365375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760773.115922:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febffb9f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512717952:12345-192.168.203.56@tcp:4:dd.0 Request processed in 39644us (40853us total) trans 4294973999 rc 0/0 00000100:00100000:0.0:1789760773.115933:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6710 00000100:00100000:0.0:1789760773.115943:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 752600208280 time 752640560184 state 0x2 ref 0 00000100:00100000:2.0:1789760773.118979:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760773.119493:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512718080 00000100:00100000:0.0:1789760773.119503:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512718080 00000100:00100000:0.0:1789760773.119557:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 752600208280 time 752640560184 state 0x0 ref 1 00000100:00100000:0.0:1789760773.119573:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 752640820600 time 752640560184 state 0x1 ref 1 00000100:00100000:0.0:1789760773.119611:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 752640820600 time 752683902562 state 0x0 ref 1 00000100:00100000:0.0:1789760773.119623:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febffb9ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512718080:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760773.119672:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27365376, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0xd807 00010000:00010000:0.0:1789760773.119681:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f7d9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.119715:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=752s v=(0 0) 00010000:00010000:0.0:1789760773.119720:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760773.119727:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f7d9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27365376->27369471] (req 27365376->27369471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.119741:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f7d9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27365376->27369471] (req 27365376->27369471) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760773.157946:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974000 is committed 00002000:00010000:3.0:1789760773.158480:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27365376 -> 27369472 00010000:00010000:3.0:1789760773.158496:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f7d9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27365376->27369471] (req 27365376->27369471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.158510:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f7d9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27365376->27369471] (req 27365376->27369471) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.158518:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f7d9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27365376->27369471] (req 27365376->27369471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.158525:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760773.158530:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f7d9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27365376->27369471] (req 27365376->27369471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.158538:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f7d9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27365376->27369471] (req 27365376->27369471) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.158561:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f7d9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27365376->27369471] (req 27365376->27369471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.158570:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f7d9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27365376->27369471] (req 27365376->27369471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760773.158663:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febffb9ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512718080:12345-192.168.203.56@tcp:4:dd.0 Request processed in 39037us (39688us total) trans 4294974000 rc 0/0 00000100:00100000:3.0:1789760773.158674:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6711 00000100:00100000:3.0:1789760773.158684:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 752640820600 time 752683902562 state 0x2 ref 0 00000100:00100000:2.0:1789760773.168064:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760773.168793:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512718208 00000100:00100000:3.0:1789760773.168804:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512718208 00000100:00100000:3.0:1789760773.168873:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 752640820600 time 752683902562 state 0x0 ref 1 00000100:00100000:3.0:1789760773.168889:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 752684162978 time 752683902562 state 0x1 ref 1 00000100:00100000:3.0:1789760773.168982:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 752684162978 time 752733280209 state 0x0 ref 1 00000100:00100000:3.0:1789760773.168996:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febffb9c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512718208:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760773.169047:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27369472, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0xd807 00010000:00010000:3.0:1789760773.169057:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2f7e0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.169072:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=752s v=(0 0) 00010000:00010000:3.0:1789760773.169078:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760773.169084:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2f7e0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27369472->27373567] (req 27369472->27373567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.169110:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2f7e0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27369472->27373567] (req 27369472->27373567) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760773.219086:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974001 is committed 00002000:00010000:3.0:1789760773.219358:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27369472 -> 27373568 00010000:00010000:3.0:1789760773.219375:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2f7e0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27369472->27373567] (req 27369472->27373567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.219389:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2f7e0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27369472->27373567] (req 27369472->27373567) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.219398:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2f7e0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27369472->27373567] (req 27369472->27373567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.219407:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760773.219412:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2f7e0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27369472->27373567] (req 27369472->27373567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.219422:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2f7e0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27369472->27373567] (req 27369472->27373567) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.219442:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2f7e0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27369472->27373567] (req 27369472->27373567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.219450:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2f7e0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27369472->27373567] (req 27369472->27373567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760773.219582:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febffb9c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512718208:12345-192.168.203.56@tcp:4:dd.0 Request processed in 50584us (51523us total) trans 4294974001 rc 0/0 00000100:00100000:3.0:1789760773.219593:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6712 00000100:00100000:3.0:1789760773.219602:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 752684162978 time 752733280209 state 0x2 ref 0 00000100:00100000:2.0:1789760773.226825:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760773.226902:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512718336 00000100:00100000:0.0:1789760773.226911:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512718336 00000100:00100000:0.0:1789760773.226964:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 752684162978 time 752733280209 state 0x0 ref 1 00000100:00100000:0.0:1789760773.226979:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 752733540625 time 752733280209 state 0x1 ref 1 00000100:00100000:0.0:1789760773.227000:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 752733540625 time 752791300821 state 0x0 ref 1 00000100:00100000:0.0:1789760773.227011:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febffb9c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512718336:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760773.227053:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27373568, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0xd807 00010000:00010000:0.0:1789760773.227061:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2f7e7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.227077:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=752s v=(0 0) 00010000:00010000:0.0:1789760773.227082:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760773.227087:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2f7e7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27373568->27377663] (req 27373568->27377663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.227112:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2f7e7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27373568->27377663] (req 27373568->27377663) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760773.257094:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27373568 -> 27377664 00010000:00010000:2.0:1789760773.257124:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2f7e7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27373568->27377663] (req 27373568->27377663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.257138:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2f7e7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27373568->27377663] (req 27373568->27377663) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.257147:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2f7e7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27373568->27377663] (req 27373568->27377663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.257154:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760773.257159:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2f7e7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27373568->27377663] (req 27373568->27377663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.257167:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2f7e7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27373568->27377663] (req 27373568->27377663) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.257186:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2f7e7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27373568->27377663] (req 27373568->27377663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.257194:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2f7e7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27373568->27377663] (req 27373568->27377663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760773.257283:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febffb9c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2360:x1876699512718336:12345-192.168.203.56@tcp:4:dd.0 Request processed in 30269us (30462us total) trans 4294974002 rc 0/0 00000100:00100000:2.0:1789760773.257292:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6713 00000100:00100000:2.0:1789760773.257301:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 752733540625 time 752791300821 state 0x2 ref 0 00000001:00080000:3.0:1789760773.257375:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974002 is committed 00000100:00100000:2.0:1789760773.261559:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760773.262322:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512718464 00000100:00100000:3.0:1789760773.262333:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512718464 00000100:00100000:3.0:1789760773.262388:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 752733540625 time 752791300821 state 0x0 ref 1 00000100:00100000:3.0:1789760773.262403:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 752791561237 time 752791300821 state 0x1 ref 1 00000100:00100000:3.0:1789760773.262468:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 752791561237 time 752826766976 state 0x0 ref 1 00000100:00100000:3.0:1789760773.262481:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febffb9f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512718464:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760773.262536:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27377664, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0xd807 00010000:00010000:3.0:1789760773.262546:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2f7ee lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.262562:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=752s v=(0 0) 00010000:00010000:3.0:1789760773.262567:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760773.262573:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2f7ee lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27377664->27381759] (req 27377664->27381759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.262588:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2f7ee lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27377664->27381759] (req 27377664->27381759) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760773.303173:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974003 is committed 00002000:00010000:3.0:1789760773.303203:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27377664 -> 27381760 00010000:00010000:3.0:1789760773.303220:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2f7ee lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27377664->27381759] (req 27377664->27381759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.303232:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2f7ee lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27377664->27381759] (req 27377664->27381759) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.303242:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2f7ee lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27377664->27381759] (req 27377664->27381759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.303250:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760773.303254:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2f7ee lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27377664->27381759] (req 27377664->27381759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.303263:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2f7ee lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27377664->27381759] (req 27377664->27381759) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.303282:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2f7ee lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27377664->27381759] (req 27377664->27381759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.303291:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2f7ee lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27377664->27381759] (req 27377664->27381759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760773.303387:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febffb9f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512718464:12345-192.168.203.56@tcp:4:dd.0 Request processed in 40903us (41893us total) trans 4294974003 rc 0/0 00000100:00100000:3.0:1789760773.303401:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6714 00000100:00100000:3.0:1789760773.303410:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 752791561237 time 752826766976 state 0x2 ref 0 00000100:00100000:0.0:1789760773.309330:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760773.309389:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512718592 00000100:00100000:3.0:1789760773.309401:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512718592 00000100:00100000:3.0:1789760773.309458:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 752791561237 time 752826766976 state 0x0 ref 1 00000100:00100000:3.0:1789760773.309472:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 752827027392 time 752826766976 state 0x1 ref 1 00000100:00100000:3.0:1789760773.309506:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 752827027392 time 752873806306 state 0x0 ref 1 00000100:00100000:3.0:1789760773.309517:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2878380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512718592:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760773.309563:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27381760, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0xd807 00010000:00010000:3.0:1789760773.309572:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2f7f5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.309588:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=752s v=(0 0) 00010000:00010000:3.0:1789760773.309592:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760773.310121:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2f7f5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27381760->27385855] (req 27381760->27385855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.310137:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2f7f5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27381760->27385855] (req 27381760->27385855) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760773.361901:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974004 is committed 00002000:00010000:3.0:1789760773.362177:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27381760 -> 27385856 00010000:00010000:3.0:1789760773.362191:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2f7f5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27381760->27385855] (req 27381760->27385855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.362205:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2f7f5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27381760->27385855] (req 27381760->27385855) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.362214:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2f7f5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27381760->27385855] (req 27381760->27385855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.362222:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760773.362226:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2f7f5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27381760->27385855] (req 27381760->27385855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.362236:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2f7f5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27381760->27385855] (req 27381760->27385855) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.362256:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2f7f5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27381760->27385855] (req 27381760->27385855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.362262:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2f7f5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27381760->27385855] (req 27381760->27385855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760773.362350:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2878380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512718592:12345-192.168.203.56@tcp:4:dd.0 Request processed in 52831us (53026us total) trans 4294974004 rc 0/0 00000100:00100000:3.0:1789760773.362360:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6715 00000100:00100000:3.0:1789760773.362369:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 752827027392 time 752873806306 state 0x2 ref 0 00000100:00100000:2.0:1789760773.365360:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760773.365587:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512718720 00000100:00100000:0.0:1789760773.365598:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512718720 00000100:00100000:0.0:1789760773.365656:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 752827027392 time 752873806306 state 0x0 ref 1 00000100:00100000:0.0:1789760773.365683:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 752874066722 time 752873806306 state 0x1 ref 1 00000100:00100000:0.0:1789760773.365739:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 752874066722 time 752930038600 state 0x0 ref 1 00000100:00100000:0.0:1789760773.365752:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febfb89ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512718720:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760773.365796:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27385856, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0xd807 00010000:00010000:0.0:1789760773.365805:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f7fc lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.365821:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=752s v=(0 0) 00010000:00010000:0.0:1789760773.365825:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760773.365832:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f7fc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27385856->27389951] (req 27385856->27389951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.365845:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f7fc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27385856->27389951] (req 27385856->27389951) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760773.412251:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974005 is committed 00002000:00010000:0.0:1789760773.412347:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27385856 -> 27389952 00010000:00010000:0.0:1789760773.412362:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f7fc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27385856->27389951] (req 27385856->27389951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.412375:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f7fc lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27385856->27389951] (req 27385856->27389951) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.412384:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f7fc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27385856->27389951] (req 27385856->27389951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.412392:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760773.412397:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f7fc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27385856->27389951] (req 27385856->27389951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.412407:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f7fc lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27385856->27389951] (req 27385856->27389951) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.412425:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f7fc lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27385856->27389951] (req 27385856->27389951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.412434:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f7fc lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27385856->27389951] (req 27385856->27389951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760773.412525:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febfb89ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512718720:12345-192.168.203.56@tcp:4:dd.0 Request processed in 46771us (47169us total) trans 4294974005 rc 0/0 00000100:00100000:0.0:1789760773.412538:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6716 00000100:00100000:0.0:1789760773.412547:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 752874066722 time 752930038600 state 0x2 ref 0 00000100:00100000:2.0:1789760773.420952:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760773.422091:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512718848 00000100:00100000:3.0:1789760773.422119:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512718848 00000100:00100000:3.0:1789760773.422189:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 752874066722 time 752930038600 state 0x0 ref 1 00000100:00100000:3.0:1789760773.422206:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 752930299016 time 752930038600 state 0x1 ref 1 00000100:00100000:2.0:1789760773.422362:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 752930299016 time 752986659001 state 0x0 ref 1 00000100:00100000:2.0:1789760773.422379:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febfb89ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512718848:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760773.422433:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27389952, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0xd807 00010000:00010000:2.0:1789760773.422442:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2f803 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.422458:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=752s v=(0 0) 00010000:00010000:2.0:1789760773.422462:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760773.422469:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2f803 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27389952->27394047] (req 27389952->27394047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.422483:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2f803 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27389952->27394047] (req 27389952->27394047) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760773.466084:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974006 is committed 00002000:00010000:0.0:1789760773.466218:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27389952 -> 27394048 00010000:00010000:0.0:1789760773.466234:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2f803 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27389952->27394047] (req 27389952->27394047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.466248:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2f803 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27389952->27394047] (req 27389952->27394047) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.466257:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2f803 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27389952->27394047] (req 27389952->27394047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.466266:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760773.466271:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2f803 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27389952->27394047] (req 27389952->27394047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.466279:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2f803 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27389952->27394047] (req 27389952->27394047) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.466300:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2f803 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27389952->27394047] (req 27389952->27394047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.466309:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2f803 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27389952->27394047] (req 27389952->27394047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760773.466409:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febfb89ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512718848:12345-192.168.203.56@tcp:4:dd.0 Request processed in 44029us (45463us total) trans 4294974006 rc 0/0 00000100:00100000:0.0:1789760773.466418:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6717 00000100:00100000:0.0:1789760773.466428:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 752930299016 time 752986659001 state 0x2 ref 0 00000100:00100000:2.0:1789760773.470038:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760773.470269:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512718976 00000100:00100000:3.0:1789760773.470278:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512718976 00000100:00100000:3.0:1789760773.470334:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 752930299016 time 752986659001 state 0x0 ref 1 00000100:00100000:3.0:1789760773.470350:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 752986919417 time 752986659001 state 0x1 ref 1 00000100:00100000:3.0:1789760773.470384:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 752986919417 time 753034683305 state 0x0 ref 1 00000100:00100000:3.0:1789760773.470398:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febfb89fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512718976:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760773.470458:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27394048, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0xd807 00010000:00010000:3.0:1789760773.470468:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2f80a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.470484:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=753s v=(0 0) 00010000:00010000:3.0:1789760773.470502:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760773.470511:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2f80a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27394048->27398143] (req 27394048->27398143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.470527:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2f80a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27394048->27398143] (req 27394048->27398143) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.491073:0:5835:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:0.0:1789760773.503030:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974007 is committed 00002000:00010000:3.0:1789760773.503195:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27394048 -> 27398144 00010000:00010000:3.0:1789760773.503214:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2f80a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27394048->27398143] (req 27394048->27398143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.503227:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2f80a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27394048->27398143] (req 27394048->27398143) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.503237:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2f80a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27394048->27398143] (req 27394048->27398143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.503245:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760773.503250:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2f80a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27394048->27398143] (req 27394048->27398143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.503258:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2f80a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27394048->27398143] (req 27394048->27398143) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.503275:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2f80a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27394048->27398143] (req 27394048->27398143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.503280:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2f80a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27394048->27398143] (req 27394048->27398143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760773.503359:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febfb89fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512718976:12345-192.168.203.56@tcp:4:dd.0 Request processed in 32960us (33329us total) trans 4294974007 rc 0/0 00000100:00100000:3.0:1789760773.503366:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6718 00000100:00100000:3.0:1789760773.503374:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 752986919417 time 753034683305 state 0x2 ref 0 00000100:00100000:3.0:1789760773.505939:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760773.506252:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512719104 00000100:00100000:0.0:1789760773.506262:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512719104 00000100:00100000:0.0:1789760773.506311:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 752986919417 time 753034683305 state 0x0 ref 1 00000100:00100000:0.0:1789760773.506322:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 753034943721 time 753034683305 state 0x1 ref 1 00000100:00100000:0.0:1789760773.506345:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 753034943721 time 753070646431 state 0x0 ref 1 00000100:00100000:0.0:1789760773.506354:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac531d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512719104:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760773.506394:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27398144, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0xd807 00010000:00010000:0.0:1789760773.506400:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f811 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.506413:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=753s v=(0 0) 00010000:00010000:0.0:1789760773.506417:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760773.506421:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f811 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27398144->27402239] (req 27398144->27402239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.506434:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f811 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27398144->27402239] (req 27398144->27402239) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760773.527303:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974008 is committed 00002000:00010000:0.0:1789760773.527378:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27398144 -> 27402240 00002000:00010000:0.0:1789760773.527383:0:13062:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760772 -> 1789760773 00002000:00010000:0.0:1789760773.527386:0:13062:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760772 -> 1789760773 00010000:00010000:0.0:1789760773.527399:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f811 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27398144->27402239] (req 27398144->27402239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.527408:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f811 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27398144->27402239] (req 27398144->27402239) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.527415:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f811 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27398144->27402239] (req 27398144->27402239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.527419:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760773.527423:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f811 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27398144->27402239] (req 27398144->27402239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.527430:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f811 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27398144->27402239] (req 27398144->27402239) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.527445:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f811 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27398144->27402239] (req 27398144->27402239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.527453:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f811 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27398144->27402239] (req 27398144->27402239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760773.527785:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac531d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512719104:12345-192.168.203.56@tcp:4:dd.0 Request processed in 21219us (21643us total) trans 4294974008 rc 0/0 00000100:00100000:0.0:1789760773.527798:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6719 00000100:00100000:0.0:1789760773.527807:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 753034943721 time 753070646431 state 0x2 ref 0 00000100:00100000:2.0:1789760773.530619:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760773.530679:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512719232 00000100:00100000:0.0:1789760773.530689:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512719232 00000100:00100000:0.0:1789760773.530769:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 753034943721 time 753070646431 state 0x0 ref 1 00000100:00100000:0.0:1789760773.530784:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 753070906847 time 753070646431 state 0x1 ref 1 00000100:00100000:0.0:1789760773.530805:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 753070906847 time 753095104748 state 0x0 ref 1 00000100:00100000:0.0:1789760773.530817:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febfb8d4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512719232:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760773.530865:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27402240, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0xd807 00010000:00010000:0.0:1789760773.530873:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2f818 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.530889:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=753s v=(0 0) 00010000:00010000:0.0:1789760773.530894:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760773.530901:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2f818 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27402240->27406335] (req 27402240->27406335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.530921:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2f818 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27402240->27406335] (req 27402240->27406335) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760773.567965:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27402240 -> 27406336 00000001:00080000:3.0:1789760773.567967:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974009 is committed 00010000:00010000:0.0:1789760773.567983:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2f818 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27402240->27406335] (req 27402240->27406335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.567996:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2f818 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27402240->27406335] (req 27402240->27406335) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.568006:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2f818 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27402240->27406335] (req 27402240->27406335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.568013:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760773.568016:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2f818 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27402240->27406335] (req 27402240->27406335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.568023:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2f818 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27402240->27406335] (req 27402240->27406335) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.568037:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2f818 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27402240->27406335] (req 27402240->27406335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.568042:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2f818 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27402240->27406335] (req 27402240->27406335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760773.568145:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febfb8d4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512719232:12345-192.168.203.56@tcp:4:dd.0 Request processed in 37325us (37530us total) trans 4294974009 rc 0/0 00000100:00100000:0.0:1789760773.568155:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6720 00000100:00100000:0.0:1789760773.568164:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 753070906847 time 753095104748 state 0x2 ref 0 00000100:00100000:2.0:1789760773.571559:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760773.571668:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512719360 00000100:00100000:3.0:1789760773.571677:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512719360 00000100:00100000:3.0:1789760773.571761:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 753070906847 time 753095104748 state 0x0 ref 1 00000100:00100000:3.0:1789760773.571785:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 753095365164 time 753095104748 state 0x1 ref 1 00000100:00100000:3.0:1789760773.571810:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 753095365164 time 753136109902 state 0x0 ref 1 00000100:00100000:3.0:1789760773.571822:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febc7b33480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512719360:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760773.572315:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27406336, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0xd807 00010000:00010000:3.0:1789760773.572324:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2f81f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.572342:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=753s v=(0 0) 00010000:00010000:3.0:1789760773.572347:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760773.572354:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2f81f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27406336->27410431] (req 27406336->27410431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.572367:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2f81f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27406336->27410431] (req 27406336->27410431) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760773.608065:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27406336 -> 27410432 00010000:00010000:2.0:1789760773.608083:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2f81f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27406336->27410431] (req 27406336->27410431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.608106:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2f81f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27406336->27410431] (req 27406336->27410431) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.608116:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2f81f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27406336->27410431] (req 27406336->27410431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.608123:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760773.608128:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2f81f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27406336->27410431] (req 27406336->27410431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.608138:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2f81f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27406336->27410431] (req 27406336->27410431) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.608159:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2f81f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27406336->27410431] (req 27406336->27410431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.608166:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2f81f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27406336->27410431] (req 27406336->27410431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760773.608281:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febc7b33480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2361:x1876699512719360:12345-192.168.203.56@tcp:4:dd.0 Request processed in 36455us (36725us total) trans 4294974010 rc 0/0 00000100:00100000:2.0:1789760773.608292:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6721 00000100:00100000:2.0:1789760773.608301:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 753095365164 time 753136109902 state 0x2 ref 0 00000001:00080000:1.0:1789760773.608904:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974010 is committed 00000100:00100000:3.0:1789760773.612326:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760773.612460:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512719488 00000100:00100000:0.0:1789760773.612470:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512719488 00000100:00100000:0.0:1789760773.612522:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 753095365164 time 753136109902 state 0x0 ref 1 00000100:00100000:0.0:1789760773.612536:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 753136370318 time 753136109902 state 0x1 ref 1 00000100:00100000:0.0:1789760773.612626:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 753136370318 time 753176924520 state 0x0 ref 1 00000100:00100000:0.0:1789760773.612639:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac531e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512719488:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760773.612686:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27410432, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0xd807 00010000:00010000:0.0:1789760773.612719:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f826 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.612736:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=753s v=(0 0) 00010000:00010000:0.0:1789760773.612741:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760773.612747:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f826 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27410432->27414527] (req 27410432->27414527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.612766:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f826 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27410432->27414527] (req 27410432->27414527) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760773.646349:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974011 is committed 00002000:00010000:0.0:1789760773.646470:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27410432 -> 27414528 00010000:00010000:0.0:1789760773.646564:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f826 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27410432->27414527] (req 27410432->27414527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.646649:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f826 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27410432->27414527] (req 27410432->27414527) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.646659:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f826 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27410432->27414527] (req 27410432->27414527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.646666:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760773.646670:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f826 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27410432->27414527] (req 27410432->27414527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.646680:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f826 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27410432->27414527] (req 27410432->27414527) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.646727:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f826 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27410432->27414527] (req 27410432->27414527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.646736:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f826 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27410432->27414527] (req 27410432->27414527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760773.646832:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac531e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512719488:12345-192.168.203.56@tcp:4:dd.0 Request processed in 34190us (34513us total) trans 4294974011 rc 0/0 00000100:00100000:0.0:1789760773.646846:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6722 00000100:00100000:0.0:1789760773.646855:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 753136370318 time 753176924520 state 0x2 ref 0 00000100:00100000:0.0:1789760773.651006:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760773.651074:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512719616 00000100:00100000:2.0:1789760773.651085:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512719616 00000100:00100000:2.0:1789760773.651271:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 753136370318 time 753176924520 state 0x0 ref 1 00000100:00100000:2.0:1789760773.651286:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 753177184936 time 753176924520 state 0x1 ref 1 00000100:00100000:2.0:1789760773.651317:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 753177184936 time 753215617139 state 0x0 ref 1 00000100:00100000:2.0:1789760773.651331:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac287a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512719616:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760773.651379:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27414528, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0xd807 00010000:00010000:2.0:1789760773.651387:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2f82d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.651404:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=753s v=(0 0) 00010000:00010000:2.0:1789760773.651409:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760773.651415:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2f82d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27414528->27418623] (req 27414528->27418623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.651428:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2f82d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27414528->27418623] (req 27414528->27418623) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760773.681113:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27414528 -> 27418624 00000001:00080000:1.0:1789760773.681126:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974012 is committed 00010000:00010000:0.0:1789760773.681132:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2f82d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27414528->27418623] (req 27414528->27418623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.681143:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2f82d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27414528->27418623] (req 27414528->27418623) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.681153:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2f82d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27414528->27418623] (req 27414528->27418623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.681160:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760773.681166:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2f82d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27414528->27418623] (req 27414528->27418623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.681176:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2f82d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27414528->27418623] (req 27414528->27418623) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.681195:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2f82d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27414528->27418623] (req 27414528->27418623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.681204:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2f82d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27414528->27418623] (req 27414528->27418623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760773.681298:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac287a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512719616:12345-192.168.203.56@tcp:4:dd.0 Request processed in 29966us (30298us total) trans 4294974012 rc 0/0 00000100:00100000:0.0:1789760773.681308:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6723 00000100:00100000:0.0:1789760773.681316:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 753177184936 time 753215617139 state 0x2 ref 0 00000100:00100000:1.0:1789760773.684174:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760773.684342:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512719744 00000100:00100000:2.0:1789760773.684353:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512719744 00000100:00100000:2.0:1789760773.684411:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 753177184936 time 753215617139 state 0x0 ref 1 00000100:00100000:2.0:1789760773.684429:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 753215877555 time 753215617139 state 0x1 ref 1 00000100:00100000:2.0:1789760773.684464:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 753215877555 time 753248762580 state 0x0 ref 1 00000100:00100000:2.0:1789760773.684478:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febe7907100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512719744:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760773.684539:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27418624, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0xd807 00010000:00010000:2.0:1789760773.684548:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2f834 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.684564:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=753s v=(0 0) 00010000:00010000:2.0:1789760773.684569:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760773.684720:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2f834 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27418624->27422719] (req 27418624->27422719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.684736:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2f834 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27418624->27422719] (req 27418624->27422719) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760773.725400:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974013 is committed 00002000:00010000:3.0:1789760773.725474:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27418624 -> 27422720 00010000:00010000:3.0:1789760773.725489:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2f834 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27418624->27422719] (req 27418624->27422719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.725502:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2f834 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27418624->27422719] (req 27418624->27422719) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.725510:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2f834 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27418624->27422719] (req 27418624->27422719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.725518:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760773.725523:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2f834 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27418624->27422719] (req 27418624->27422719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.725647:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2f834 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27418624->27422719] (req 27418624->27422719) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.725667:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2f834 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27418624->27422719] (req 27418624->27422719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.725675:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2f834 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27418624->27422719] (req 27418624->27422719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760773.725949:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febe7907100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512719744:12345-192.168.203.56@tcp:4:dd.0 Request processed in 41469us (41779us total) trans 4294974013 rc 0/0 00000100:00100000:3.0:1789760773.725959:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6724 00000100:00100000:3.0:1789760773.725969:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 753215877555 time 753248762580 state 0x2 ref 0 00000100:00100000:3.0:1789760773.728291:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760773.728351:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512719872 00000100:00100000:0.0:1789760773.728361:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512719872 00000100:00100000:0.0:1789760773.728414:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 753215877555 time 753248762580 state 0x0 ref 1 00000100:00100000:0.0:1789760773.728428:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 753249022996 time 753248762580 state 0x1 ref 1 00000100:00100000:1.0:1789760773.728510:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 753249022996 time 753292806881 state 0x0 ref 1 00000100:00100000:1.0:1789760773.728643:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac531ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512719872:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760773.728724:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27422720, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0xd807 00010000:00010000:1.0:1789760773.728734:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2f83b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.728749:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=753s v=(0 0) 00010000:00010000:1.0:1789760773.728753:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760773.728800:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2f83b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27422720->27426815] (req 27422720->27426815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.728818:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2f83b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27422720->27426815] (req 27422720->27426815) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760773.756872:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974014 is committed 00002000:00010000:1.0:1789760773.757019:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27422720 -> 27426816 00010000:00010000:1.0:1789760773.757112:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2f83b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27422720->27426815] (req 27422720->27426815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.757127:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2f83b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27422720->27426815] (req 27422720->27426815) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.757136:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2f83b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27422720->27426815] (req 27422720->27426815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.757143:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760773.757148:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2f83b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27422720->27426815] (req 27422720->27426815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.757158:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2f83b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27422720->27426815] (req 27422720->27426815) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.757184:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2f83b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27422720->27426815] (req 27422720->27426815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.757193:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2f83b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27422720->27426815] (req 27422720->27426815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760773.757397:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac531ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512719872:12345-192.168.203.56@tcp:4:dd.0 Request processed in 28871us (29110us total) trans 4294974014 rc 0/0 00000100:00100000:1.0:1789760773.757408:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6725 00000100:00100000:1.0:1789760773.757417:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 753249022996 time 753292806881 state 0x2 ref 0 00000100:00100000:0.0:1789760773.762596:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760773.762896:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512720000 00000100:00100000:1.0:1789760773.762905:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512720000 00000100:00100000:1.0:1789760773.762952:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 753249022996 time 753292806881 state 0x0 ref 1 00000100:00100000:1.0:1789760773.762967:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 753293067297 time 753292806881 state 0x1 ref 1 00000100:00100000:1.0:1789760773.763030:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 753293067297 time 753327328737 state 0x0 ref 1 00000100:00100000:1.0:1789760773.763042:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2879c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512720000:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760773.763088:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27426816, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0xd807 00010000:00010000:1.0:1789760773.763106:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2f842 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.763122:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=753s v=(0 0) 00010000:00010000:1.0:1789760773.763127:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760773.763133:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2f842 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27426816->27430911] (req 27426816->27430911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.763148:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2f842 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27426816->27430911] (req 27426816->27430911) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760773.788758:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27426816 -> 27430912 00010000:00010000:1.0:1789760773.788786:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2f842 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27426816->27430911] (req 27426816->27430911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.788800:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2f842 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27426816->27430911] (req 27426816->27430911) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.788810:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2f842 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27426816->27430911] (req 27426816->27430911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.788816:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4200 already unused, calling callback (ffffffffc15d8350) 00000001:00080000:0.0:1789760773.788817:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974015 is committed 00010000:00010000:1.0:1789760773.788821:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2f842 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27426816->27430911] (req 27426816->27430911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.788829:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2f842 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27426816->27430911] (req 27426816->27430911) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.788847:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2f842 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27426816->27430911] (req 27426816->27430911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.788856:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2f842 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27426816->27430911] (req 27426816->27430911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760773.788990:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2879c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512720000:12345-192.168.203.56@tcp:4:dd.0 Request processed in 25944us (26397us total) trans 4294974015 rc 0/0 00000100:00100000:1.0:1789760773.789001:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6726 00000100:00100000:1.0:1789760773.789011:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 753293067297 time 753327328737 state 0x2 ref 0 00000100:00100000:2.0:1789760773.792933:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760773.794131:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512720128 00000100:00100000:3.0:1789760773.794143:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512720128 00000100:00100000:3.0:1789760773.794231:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 753293067297 time 753327328737 state 0x0 ref 1 00000100:00100000:3.0:1789760773.794249:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 753327589153 time 753327328737 state 0x1 ref 1 00000100:00100000:3.0:1789760773.794284:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 753327589153 time 753358583958 state 0x0 ref 1 00000100:00100000:3.0:1789760773.794299:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febc7b32680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512720128:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760773.794352:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27430912, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0xd807 00010000:00010000:3.0:1789760773.794361:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2f849 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.794376:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=753s v=(0 0) 00010000:00010000:3.0:1789760773.794380:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760773.794387:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2f849 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27430912->27435007] (req 27430912->27435007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.794400:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2f849 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27430912->27435007] (req 27430912->27435007) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760773.831978:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974016 is committed 00002000:00010000:2.0:1789760773.832137:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27430912 -> 27435008 00010000:00010000:2.0:1789760773.832155:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2f849 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27430912->27435007] (req 27430912->27435007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.832170:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2f849 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27430912->27435007] (req 27430912->27435007) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.832179:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2f849 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27430912->27435007] (req 27430912->27435007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.832186:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760773.832191:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2f849 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27430912->27435007] (req 27430912->27435007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.832201:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2f849 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27430912->27435007] (req 27430912->27435007) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.832222:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2f849 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27430912->27435007] (req 27430912->27435007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.832230:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2f849 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27430912->27435007] (req 27430912->27435007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760773.832370:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febc7b32680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512720128:12345-192.168.203.56@tcp:4:dd.0 Request processed in 38069us (39440us total) trans 4294974016 rc 0/0 00000100:00100000:2.0:1789760773.832381:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6727 00000100:00100000:2.0:1789760773.832390:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 753327589153 time 753358583958 state 0x2 ref 0 00000100:00100000:3.0:1789760773.837324:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760773.837523:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512720256 00000100:00100000:1.0:1789760773.837532:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512720256 00000100:00100000:1.0:1789760773.837585:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 753327589153 time 753358583958 state 0x0 ref 1 00000100:00100000:1.0:1789760773.837600:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 753358844374 time 753358583958 state 0x1 ref 1 00000100:00100000:1.0:1789760773.837677:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 753358844374 time 753401975273 state 0x0 ref 1 00000100:00100000:1.0:1789760773.837690:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac531dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512720256:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760773.837798:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27435008, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0xd807 00010000:00010000:1.0:1789760773.837808:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2f850 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.837825:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=753s v=(0 0) 00010000:00010000:1.0:1789760773.837830:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760773.837836:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2f850 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27435008->27439103] (req 27435008->27439103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.837849:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2f850 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27435008->27439103] (req 27435008->27439103) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760773.880321:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974017 is committed 00002000:00010000:2.0:1789760773.880355:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27435008 -> 27439104 00010000:00010000:2.0:1789760773.880371:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2f850 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27435008->27439103] (req 27435008->27439103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.880382:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2f850 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27435008->27439103] (req 27435008->27439103) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.880391:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2f850 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27435008->27439103] (req 27435008->27439103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.880400:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760773.880406:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2f850 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27435008->27439103] (req 27435008->27439103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.880415:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2f850 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27435008->27439103] (req 27435008->27439103) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.880435:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2f850 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27435008->27439103] (req 27435008->27439103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.880443:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2f850 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27435008->27439103] (req 27435008->27439103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760773.880543:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac531dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512720256:12345-192.168.203.56@tcp:4:dd.0 Request processed in 42850us (43222us total) trans 4294974017 rc 0/0 00000100:00100000:2.0:1789760773.880551:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6728 00000100:00100000:2.0:1789760773.880558:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 753358844374 time 753401975273 state 0x2 ref 0 00000100:00100000:0.0:1789760773.888184:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760773.888285:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512720384 00000100:00100000:1.0:1789760773.888295:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512720384 00000100:00100000:1.0:1789760773.888352:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 753358844374 time 753401975273 state 0x0 ref 1 00000100:00100000:1.0:1789760773.888368:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 753402235689 time 753401975273 state 0x1 ref 1 00000100:00100000:1.0:1789760773.888401:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 753402235689 time 753452700575 state 0x0 ref 1 00000100:00100000:1.0:1789760773.888413:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2878e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512720384:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760773.888458:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27439104, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0xd807 00010000:00010000:1.0:1789760773.888467:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2f857 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.888482:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=753s v=(0 0) 00010000:00010000:1.0:1789760773.888496:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760773.888502:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2f857 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27439104->27443199] (req 27439104->27443199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.888516:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2f857 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27439104->27443199] (req 27439104->27443199) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760773.921757:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27439104 -> 27443200 00010000:00010000:1.0:1789760773.921787:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2f857 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27439104->27443199] (req 27439104->27443199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.921807:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2f857 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27439104->27443199] (req 27439104->27443199) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.921815:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2f857 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27439104->27443199] (req 27439104->27443199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.921823:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760773.921828:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2f857 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27439104->27443199] (req 27439104->27443199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.921837:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2f857 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27439104->27443199] (req 27439104->27443199) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.921856:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2f857 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27439104->27443199] (req 27439104->27443199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.921864:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2f857 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27439104->27443199] (req 27439104->27443199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760773.921947:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2878e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2361:x1876699512720384:12345-192.168.203.56@tcp:4:dd.0 Request processed in 33531us (33767us total) trans 4294974018 rc 0/0 00000100:00100000:1.0:1789760773.921957:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6729 00000100:00100000:1.0:1789760773.921966:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 753402235689 time 753452700575 state 0x2 ref 0 00000001:00080000:1.0:1789760773.922005:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974018 is committed 00000100:00100000:2.0:1789760773.931800:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760773.933401:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512720512 00000100:00100000:3.0:1789760773.933413:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512720512 00000100:00100000:2.0:1789760773.933555:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 753402235689 time 753452700575 state 0x0 ref 1 00000100:00100000:2.0:1789760773.933572:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 753452960991 time 753452700575 state 0x1 ref 1 00000100:00100000:2.0:1789760773.933638:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 753452960991 time 753497937066 state 0x0 ref 1 00000100:00100000:2.0:1789760773.933651:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febc7b33100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512720512:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760773.933725:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27443200, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0xd807 00010000:00010000:2.0:1789760773.933734:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2f85e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.933747:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=753s v=(0 0) 00010000:00010000:2.0:1789760773.933752:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760773.933757:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2f85e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27443200->27447295] (req 27443200->27447295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.933771:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2f85e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27443200->27447295] (req 27443200->27447295) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760773.989465:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27443200 -> 27447296 00010000:00010000:1.0:1789760773.989487:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2f85e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27443200->27447295] (req 27443200->27447295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.989502:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2f85e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27443200->27447295] (req 27443200->27447295) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.989512:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2f85e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27443200->27447295] (req 27443200->27447295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.989520:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760773.989525:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2f85e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27443200->27447295] (req 27443200->27447295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.989534:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2f85e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27443200->27447295] (req 27443200->27447295) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.989555:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2f85e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27443200->27447295] (req 27443200->27447295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.989563:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2f85e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27443200->27447295] (req 27443200->27447295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760773.989665:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febc7b33100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2361:x1876699512720512:12345-192.168.203.56@tcp:4:dd.0 Request processed in 56012us (57870us total) trans 4294974019 rc 0/0 00000100:00100000:1.0:1789760773.989677:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6730 00000100:00100000:1.0:1789760773.989688:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 753452960991 time 753497937066 state 0x2 ref 0 00000001:00080000:1.0:1789760773.993538:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974019 is committed 00000100:00100000:3.0:1789760773.996129:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760773.996524:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512720640 00000100:00100000:2.0:1789760773.996538:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512720640 00000100:00100000:2.0:1789760773.996608:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 753452960991 time 753497937066 state 0x0 ref 1 00000100:00100000:2.0:1789760773.996627:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 753498197482 time 753497937066 state 0x1 ref 1 00000100:00100000:2.0:1789760773.996669:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 753498197482 time 753560968904 state 0x0 ref 1 00000100:00100000:2.0:1789760773.996684:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac531d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512720640:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760773.996771:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27447296, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0xd807 00010000:00010000:2.0:1789760773.996782:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2f865 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.996802:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=753s v=(0 0) 00010000:00010000:2.0:1789760773.996807:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760773.996815:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2f865 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27447296->27451391] (req 27447296->27451391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.996832:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2f865 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27447296->27451391] (req 27447296->27451391) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760774.072027:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27447296 -> 27451392 00010000:00010000:2.0:1789760774.072055:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2f865 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27447296->27451391] (req 27447296->27451391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.072077:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2f865 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27447296->27451391] (req 27447296->27451391) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.072087:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2f865 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27447296->27451391] (req 27447296->27451391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.072109:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760774.072115:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2f865 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27447296->27451391] (req 27447296->27451391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.072125:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2f865 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27447296->27451391] (req 27447296->27451391) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760774.072136:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974020 is committed 00010000:00010000:2.0:1789760774.072150:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2f865 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27447296->27451391] (req 27447296->27451391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.072158:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2f865 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27447296->27451391] (req 27447296->27451391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760774.072283:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac531d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512720640:12345-192.168.203.56@tcp:4:dd.0 Request processed in 75594us (76162us total) trans 4294974020 rc 0/0 00000100:00100000:2.0:1789760774.072294:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6731 00000100:00100000:2.0:1789760774.072306:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 753498197482 time 753560968904 state 0x2 ref 0 00000100:00100000:0.0:1789760774.078634:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760774.079037:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512720768 00000100:00100000:1.0:1789760774.079051:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512720768 00000100:00100000:1.0:1789760774.079135:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 753498197482 time 753560968904 state 0x0 ref 1 00000100:00100000:1.0:1789760774.079155:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 753561229320 time 753560968904 state 0x1 ref 1 00000100:00100000:1.0:1789760774.079189:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 753561229320 time 753643488916 state 0x0 ref 1 00000100:00100000:1.0:1789760774.079203:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac287b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512720768:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760774.079264:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27451392, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0xd807 00010000:00010000:1.0:1789760774.079275:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2f86c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.079295:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=753s v=(0 0) 00010000:00010000:1.0:1789760774.079300:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760774.079307:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2f86c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27451392->27455487] (req 27451392->27455487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.079324:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2f86c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27451392->27455487] (req 27451392->27455487) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760774.139386:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27451392 -> 27455488 00010000:00010000:0.0:1789760774.139417:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2f86c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27451392->27455487] (req 27451392->27455487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.139434:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2f86c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27451392->27455487] (req 27451392->27455487) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.139444:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2f86c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27451392->27455487] (req 27451392->27455487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.139458:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760774.139464:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2f86c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27451392->27455487] (req 27451392->27455487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.139474:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2f86c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27451392->27455487] (req 27451392->27455487) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.139500:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2f86c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27451392->27455487] (req 27451392->27455487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.139508:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2f86c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27451392->27455487] (req 27451392->27455487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760774.139692:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974021 is committed 00000100:00100000:0.0:1789760774.141051:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac287b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512720768:12345-192.168.203.56@tcp:4:dd.0 Request processed in 61826us (62407us total) trans 4294974021 rc 0/0 00000100:00100000:0.0:1789760774.141070:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6732 00000100:00100000:0.0:1789760774.141083:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 753561229320 time 753643488916 state 0x2 ref 0 00000100:00100000:0.0:1789760774.147084:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760774.147167:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512720896 00000100:00100000:1.0:1789760774.147180:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512720896 00000100:00100000:1.0:1789760774.147254:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 753561229320 time 753643488916 state 0x0 ref 1 00000100:00100000:1.0:1789760774.147278:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 753643749332 time 753643488916 state 0x1 ref 1 00000100:00100000:1.0:1789760774.147315:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 753643749332 time 753711614278 state 0x0 ref 1 00000100:00100000:1.0:1789760774.147330:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2878000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512720896:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760774.147396:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27455488, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0xd807 00010000:00010000:1.0:1789760774.147407:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2f873 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.147425:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=753s v=(0 0) 00010000:00010000:1.0:1789760774.147430:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760774.147437:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2f873 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27455488->27459583] (req 27455488->27459583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.147454:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2f873 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27455488->27459583] (req 27455488->27459583) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760774.201799:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27455488 -> 27459584 00010000:00010000:2.0:1789760774.201827:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2f873 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27455488->27459583] (req 27455488->27459583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.201841:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2f873 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27455488->27459583] (req 27455488->27459583) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.201850:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2f873 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27455488->27459583] (req 27455488->27459583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.201858:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760774.201864:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2f873 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27455488->27459583] (req 27455488->27459583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.201873:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2f873 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27455488->27459583] (req 27455488->27459583) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.201897:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2f873 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27455488->27459583] (req 27455488->27459583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.201905:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2f873 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27455488->27459583] (req 27455488->27459583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760774.202036:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2878000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2361:x1876699512720896:12345-192.168.203.56@tcp:4:dd.0 Request processed in 54701us (54958us total) trans 4294974022 rc 0/0 00000100:00100000:2.0:1789760774.202049:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6733 00000100:00100000:2.0:1789760774.202061:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 753643749332 time 753711614278 state 0x2 ref 0 00000001:00080000:2.0:1789760774.202125:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974022 is committed 00000100:00100000:1.0:1789760774.209293:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760774.209471:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512721024 00000100:00100000:2.0:1789760774.209494:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512721024 00000100:00100000:2.0:1789760774.209576:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 753643749332 time 753711614278 state 0x0 ref 1 00000100:00100000:2.0:1789760774.209594:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 753711874694 time 753711614278 state 0x1 ref 1 00000100:00100000:2.0:1789760774.209630:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 753711874694 time 753773930078 state 0x0 ref 1 00000100:00100000:2.0:1789760774.209646:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febe7906300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512721024:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760774.209739:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27459584, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0xd807 00010000:00010000:2.0:1789760774.209750:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2f87a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.209770:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=753s v=(0 0) 00010000:00010000:2.0:1789760774.209775:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760774.209783:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2f87a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27459584->27463679] (req 27459584->27463679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.209798:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2f87a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27459584->27463679] (req 27459584->27463679) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760774.265483:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974023 is committed 00002000:00010000:1.0:1789760774.265643:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27459584 -> 27463680 00010000:00010000:1.0:1789760774.265664:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2f87a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27459584->27463679] (req 27459584->27463679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.265679:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2f87a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27459584->27463679] (req 27459584->27463679) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.265688:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2f87a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27459584->27463679] (req 27459584->27463679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.266059:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760774.266067:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2f87a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27459584->27463679] (req 27459584->27463679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.266079:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2f87a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27459584->27463679] (req 27459584->27463679) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.266113:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2f87a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27459584->27463679] (req 27459584->27463679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.266121:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2f87a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27459584->27463679] (req 27459584->27463679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760774.266640:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febe7906300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512721024:12345-192.168.203.56@tcp:4:dd.0 Request processed in 56989us (57357us total) trans 4294974023 rc 0/0 00000100:00100000:1.0:1789760774.266654:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6734 00000100:00100000:1.0:1789760774.266666:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 753711874694 time 753773930078 state 0x2 ref 0 00000100:00100000:0.0:1789760774.270623:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760774.270851:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512721152 00000100:00100000:1.0:1789760774.270864:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512721152 00000100:00100000:1.0:1789760774.270940:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 753711874694 time 753773930078 state 0x0 ref 1 00000100:00100000:1.0:1789760774.270961:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 753774190494 time 753773930078 state 0x1 ref 1 00000100:00100000:1.0:1789760774.271004:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 753774190494 time 753835302053 state 0x0 ref 1 00000100:00100000:1.0:1789760774.271024:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2879180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512721152:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760774.271109:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27463680, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0xd807 00010000:00010000:1.0:1789760774.271120:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2f881 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.271141:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=753s v=(0 0) 00010000:00010000:1.0:1789760774.271147:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760774.271154:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2f881 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27463680->27467775] (req 27463680->27467775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.271170:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2f881 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27463680->27467775] (req 27463680->27467775) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760774.307653:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27463680 -> 27467776 00010000:00010000:0.0:1789760774.307680:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2f881 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27463680->27467775] (req 27463680->27467775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760774.307735:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974024 is committed 00010000:00010000:0.0:1789760774.307778:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2f881 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27463680->27467775] (req 27463680->27467775) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.307788:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2f881 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27463680->27467775] (req 27463680->27467775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.307796:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760774.307802:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2f881 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27463680->27467775] (req 27463680->27467775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.307813:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2f881 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27463680->27467775] (req 27463680->27467775) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.307839:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2f881 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27463680->27467775] (req 27463680->27467775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.307847:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2f881 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27463680->27467775] (req 27463680->27467775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760774.307968:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2879180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512721152:12345-192.168.203.56@tcp:4:dd.0 Request processed in 36940us (37354us total) trans 4294974024 rc 0/0 00000100:00100000:0.0:1789760774.307980:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6735 00000100:00100000:0.0:1789760774.307992:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 753774190494 time 753835302053 state 0x2 ref 0 00000100:00100000:1.0:1789760774.315255:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760774.315358:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512721280 00000100:00100000:2.0:1789760774.315371:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512721280 00000100:00100000:2.0:1789760774.315511:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 753774190494 time 753835302053 state 0x0 ref 1 00000100:00100000:2.0:1789760774.315534:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 753835562469 time 753835302053 state 0x1 ref 1 00000100:00100000:2.0:1789760774.315569:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 753835562469 time 753879868904 state 0x0 ref 1 00000100:00100000:2.0:1789760774.315582:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febe7904e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512721280:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760774.315660:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27467776, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0xd807 00010000:00010000:2.0:1789760774.315671:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2f888 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.315691:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=753s v=(0 0) 00010000:00010000:2.0:1789760774.315730:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760774.315739:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2f888 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27467776->27471871] (req 27467776->27471871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.315756:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2f888 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27467776->27471871] (req 27467776->27471871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760774.351038:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27467776 -> 27471872 00010000:00010000:1.0:1789760774.351064:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2f888 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27467776->27471871] (req 27467776->27471871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.351080:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2f888 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27467776->27471871] (req 27467776->27471871) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.351100:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2f888 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27467776->27471871] (req 27467776->27471871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.351108:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760774.351114:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2f888 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27467776->27471871] (req 27467776->27471871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.351125:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2f888 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27467776->27471871] (req 27467776->27471871) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.351150:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2f888 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27467776->27471871] (req 27467776->27471871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.351159:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2f888 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27467776->27471871] (req 27467776->27471871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760774.351278:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febe7904e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2360:x1876699512721280:12345-192.168.203.56@tcp:4:dd.0 Request processed in 35690us (36030us total) trans 4294974025 rc 0/0 00000100:00100000:1.0:1789760774.351299:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6736 00000100:00100000:1.0:1789760774.351311:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 753835562469 time 753879868904 state 0x2 ref 0 00000001:00080000:3.0:1789760774.351863:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974025 is committed 00000100:00100000:1.0:1789760774.358363:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760774.359746:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512721408 00000100:00100000:2.0:1789760774.359765:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512721408 00000100:00100000:2.0:1789760774.363361:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 753835562469 time 753879868904 state 0x0 ref 1 00000100:00100000:2.0:1789760774.363381:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 753880129320 time 753879868904 state 0x1 ref 1 00000100:00100000:2.0:1789760774.363491:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 753880129320 time 753927790815 state 0x0 ref 1 00000100:00100000:2.0:1789760774.363522:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2744e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512721408:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760774.363907:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27471872, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0xd807 00010000:00010000:2.0:1789760774.363920:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2f88f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.363941:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=753s v=(0 0) 00010000:00010000:2.0:1789760774.363947:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760774.363955:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2f88f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27471872->27475967] (req 27471872->27475967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.363973:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2f88f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27471872->27475967] (req 27471872->27475967) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760774.401073:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974026 is committed 00002000:00010000:1.0:1789760774.401150:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27471872 -> 27475968 00010000:00010000:1.0:1789760774.401172:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2f88f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27471872->27475967] (req 27471872->27475967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.401187:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2f88f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27471872->27475967] (req 27471872->27475967) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.401197:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2f88f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27471872->27475967] (req 27471872->27475967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.401210:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760774.401215:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2f88f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27471872->27475967] (req 27471872->27475967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.401225:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2f88f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27471872->27475967] (req 27471872->27475967) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.401249:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2f88f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27471872->27475967] (req 27471872->27475967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.401256:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2f88f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27471872->27475967] (req 27471872->27475967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760774.401383:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2744e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512721408:12345-192.168.203.56@tcp:4:dd.0 Request processed in 37871us (43099us total) trans 4294974026 rc 0/0 00000100:00100000:1.0:1789760774.401395:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6737 00000100:00100000:1.0:1789760774.401407:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 753880129320 time 753927790815 state 0x2 ref 0 00000100:00100000:2.0:1789760774.408261:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760774.408382:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512721536 00000100:00100000:0.0:1789760774.408396:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512721536 00000100:00100000:0.0:1789760774.408477:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 753880129320 time 753927790815 state 0x0 ref 1 00000100:00100000:0.0:1789760774.408508:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 753928051231 time 753927790815 state 0x1 ref 1 00000100:00100000:0.0:1789760774.408547:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 753928051231 time 753972847012 state 0x0 ref 1 00000100:00100000:0.0:1789760774.408562:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac41d5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512721536:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760774.408626:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27475968, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0xd807 00010000:00010000:0.0:1789760774.408637:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2f896 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.408657:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=753s v=(0 0) 00010000:00010000:0.0:1789760774.408662:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760774.408669:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2f896 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27475968->27480063] (req 27475968->27480063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.408685:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2f896 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27475968->27480063] (req 27475968->27480063) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760774.454056:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27475968 -> 27480064 00000001:00080000:2.0:1789760774.454059:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974027 is committed 00010000:00010000:1.0:1789760774.454081:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2f896 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27475968->27480063] (req 27475968->27480063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.454106:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2f896 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27475968->27480063] (req 27475968->27480063) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.454122:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2f896 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27475968->27480063] (req 27475968->27480063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.454130:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760774.454136:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2f896 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27475968->27480063] (req 27475968->27480063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.454146:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2f896 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27475968->27480063] (req 27475968->27480063) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.454174:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2f896 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27475968->27480063] (req 27475968->27480063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.454184:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c2f896 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27475968->27480063] (req 27475968->27480063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760774.454394:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac41d5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512721536:12345-192.168.203.56@tcp:4:dd.0 Request processed in 45828us (46143us total) trans 4294974027 rc 0/0 00000100:00100000:1.0:1789760774.454644:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6738 00000100:00100000:1.0:1789760774.454658:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 753928051231 time 753972847012 state 0x2 ref 0 00000100:00100000:2.0:1789760774.458350:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760774.458508:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512721664 00000100:00100000:1.0:1789760774.458523:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512721664 00000100:00100000:1.0:1789760774.458597:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 753928051231 time 753972847012 state 0x0 ref 1 00000100:00100000:1.0:1789760774.458618:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 753973107428 time 753972847012 state 0x1 ref 1 00000100:00100000:3.0:1789760774.458810:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 753973107428 time 754023105532 state 0x0 ref 1 00000100:00100000:3.0:1789760774.458830:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac3302680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512721664:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760774.458893:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27480064, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0xd807 00010000:00010000:3.0:1789760774.458903:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2f89d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.458923:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=754s v=(0 0) 00010000:00010000:3.0:1789760774.458928:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760774.458936:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2f89d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27480064->27484159] (req 27480064->27484159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.458952:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2f89d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27480064->27484159] (req 27480064->27484159) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760774.505273:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974028 is committed 00002000:00010000:3.0:1789760774.505390:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27480064 -> 27484160 00010000:00010000:3.0:1789760774.505411:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2f89d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27480064->27484159] (req 27480064->27484159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.505427:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2f89d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27480064->27484159] (req 27480064->27484159) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.505437:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2f89d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27480064->27484159] (req 27480064->27484159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.505451:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760774.505457:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2f89d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27480064->27484159] (req 27480064->27484159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.505467:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2f89d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27480064->27484159] (req 27480064->27484159) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.505492:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2f89d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27480064->27484159] (req 27480064->27484159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.505501:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2f89d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27480064->27484159] (req 27480064->27484159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760774.505644:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac3302680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512721664:12345-192.168.203.56@tcp:4:dd.0 Request processed in 46816us (47304us total) trans 4294974028 rc 0/0 00000100:00100000:3.0:1789760774.505658:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6739 00000100:00100000:3.0:1789760774.505671:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 753973107428 time 754023105532 state 0x2 ref 0 00000100:00100000:3.0:1789760774.509871:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760774.509961:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512721792 00000100:00100000:1.0:1789760774.509976:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512721792 00000100:00100000:1.0:1789760774.510054:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 753973107428 time 754023105532 state 0x0 ref 1 00000100:00100000:1.0:1789760774.510075:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 754023365948 time 754023105532 state 0x1 ref 1 00000100:00100000:1.0:1789760774.510120:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 754023365948 time 754074418989 state 0x0 ref 1 00000100:00100000:1.0:1789760774.510135:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febfb897100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512721792:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760774.510218:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27484160, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0xd807 00010000:00010000:1.0:1789760774.510229:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2f8a4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.510250:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=754s v=(0 0) 00010000:00010000:1.0:1789760774.510255:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760774.510264:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2f8a4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27484160->27488255] (req 27484160->27488255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.510282:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2f8a4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27484160->27488255] (req 27484160->27488255) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.515186:0:7647:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:1.0:1789760774.540490:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974029 is committed 00002000:00010000:2.0:1789760774.540857:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27484160 -> 27488256 00002000:00010000:2.0:1789760774.540863:0:13063:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760773 -> 1789760774 00002000:00010000:2.0:1789760774.540866:0:13063:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760773 -> 1789760774 00010000:00010000:2.0:1789760774.540884:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2f8a4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27484160->27488255] (req 27484160->27488255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.540900:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2f8a4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27484160->27488255] (req 27484160->27488255) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.540911:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2f8a4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27484160->27488255] (req 27484160->27488255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.540924:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760774.540930:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2f8a4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27484160->27488255] (req 27484160->27488255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.540941:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2f8a4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27484160->27488255] (req 27484160->27488255) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.540965:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2f8a4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27484160->27488255] (req 27484160->27488255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.540975:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2f8a4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27484160->27488255] (req 27484160->27488255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760774.541274:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febfb897100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512721792:12345-192.168.203.56@tcp:4:dd.0 Request processed in 31134us (31409us total) trans 4294974029 rc 0/0 00000100:00100000:2.0:1789760774.541288:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6740 00000100:00100000:2.0:1789760774.541299:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 754023365948 time 754074418989 state 0x2 ref 0 00000100:00100000:2.0:1789760774.547289:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760774.547402:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512721920 00000100:00100000:1.0:1789760774.547416:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512721920 00000100:00100000:1.0:1789760774.547488:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 754023365948 time 754074418989 state 0x0 ref 1 00000100:00100000:1.0:1789760774.547508:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 754074679405 time 754074418989 state 0x1 ref 1 00000100:00100000:1.0:1789760774.547545:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 754074679405 time 754111844259 state 0x0 ref 1 00000100:00100000:1.0:1789760774.547560:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac3300e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512721920:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760774.547628:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27488256, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0xd807 00010000:00010000:1.0:1789760774.547639:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2f8ab lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.547658:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=754s v=(0 0) 00010000:00010000:1.0:1789760774.547663:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760774.547671:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2f8ab lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27488256->27492351] (req 27488256->27492351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.547689:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2f8ab lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27488256->27492351] (req 27488256->27492351) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760774.587626:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974030 is committed 00002000:00010000:1.0:1789760774.588342:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27488256 -> 27492352 00010000:00010000:1.0:1789760774.588363:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2f8ab lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27488256->27492351] (req 27488256->27492351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.588377:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2f8ab lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27488256->27492351] (req 27488256->27492351) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.588388:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2f8ab lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27488256->27492351] (req 27488256->27492351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.588396:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760774.588402:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2f8ab lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27488256->27492351] (req 27488256->27492351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.588413:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2f8ab lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27488256->27492351] (req 27488256->27492351) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.588435:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2f8ab lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27488256->27492351] (req 27488256->27492351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.588443:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2f8ab lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27488256->27492351] (req 27488256->27492351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760774.588571:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac3300e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512721920:12345-192.168.203.56@tcp:4:dd.0 Request processed in 41005us (41290us total) trans 4294974030 rc 0/0 00000100:00100000:1.0:1789760774.588584:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6741 00000100:00100000:1.0:1789760774.588597:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 754074679405 time 754111844259 state 0x2 ref 0 00000100:00100000:1.0:1789760774.593487:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760774.593575:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512722048 00000100:00100000:3.0:1789760774.593588:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512722048 00000100:00100000:3.0:1789760774.593670:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 754074679405 time 754111844259 state 0x0 ref 1 00000100:00100000:3.0:1789760774.593691:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 754112104675 time 754111844259 state 0x1 ref 1 00000100:00100000:3.0:1789760774.593965:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 754112104675 time 754158262711 state 0x0 ref 1 00000100:00100000:3.0:1789760774.593983:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2745f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512722048:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760774.594058:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27492352, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0xd807 00010000:00010000:3.0:1789760774.594080:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2f8b2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.594109:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=754s v=(0 0) 00010000:00010000:3.0:1789760774.594115:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760774.594123:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2f8b2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27492352->27496447] (req 27492352->27496447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.594148:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2f8b2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27492352->27496447] (req 27492352->27496447) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760774.635013:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974031 is committed 00002000:00010000:3.0:1789760774.635182:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27492352 -> 27496448 00010000:00010000:3.0:1789760774.635202:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2f8b2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27492352->27496447] (req 27492352->27496447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.635218:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2f8b2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27492352->27496447] (req 27492352->27496447) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.635229:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2f8b2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27492352->27496447] (req 27492352->27496447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.635242:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760774.635248:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2f8b2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27492352->27496447] (req 27492352->27496447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.635259:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2f8b2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27492352->27496447] (req 27492352->27496447) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.635281:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2f8b2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27492352->27496447] (req 27492352->27496447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.635289:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2f8b2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27492352->27496447] (req 27492352->27496447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760774.635642:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2745f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512722048:12345-192.168.203.56@tcp:4:dd.0 Request processed in 41435us (41942us total) trans 4294974031 rc 0/0 00000100:00100000:3.0:1789760774.635655:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6742 00000100:00100000:3.0:1789760774.635665:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 754112104675 time 754158262711 state 0x2 ref 0 00000100:00100000:3.0:1789760774.638469:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760774.638624:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512722176 00000100:00100000:2.0:1789760774.638638:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512722176 00000100:00100000:2.0:1789760774.638745:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 754112104675 time 754158262711 state 0x0 ref 1 00000100:00100000:2.0:1789760774.638829:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 754158523127 time 754158262711 state 0x1 ref 1 00000100:00100000:2.0:1789760774.638869:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 754158523127 time 754203169107 state 0x0 ref 1 00000100:00100000:2.0:1789760774.638885:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febfb895880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512722176:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760774.638952:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27496448, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0xd807 00010000:00010000:2.0:1789760774.638963:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2f8b9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.638982:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=754s v=(0 0) 00010000:00010000:2.0:1789760774.638988:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760774.638995:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2f8b9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27496448->27500543] (req 27496448->27500543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.639010:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2f8b9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27496448->27500543] (req 27496448->27500543) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760774.663570:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974032 is committed 00002000:00010000:3.0:1789760774.663744:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27496448 -> 27500544 00010000:00010000:3.0:1789760774.663765:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2f8b9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27496448->27500543] (req 27496448->27500543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.663780:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2f8b9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27496448->27500543] (req 27496448->27500543) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.663790:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2f8b9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27496448->27500543] (req 27496448->27500543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.663805:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760774.663812:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2f8b9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27496448->27500543] (req 27496448->27500543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.663822:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2f8b9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27496448->27500543] (req 27496448->27500543) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.663842:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2f8b9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27496448->27500543] (req 27496448->27500543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.663848:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2f8b9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27496448->27500543] (req 27496448->27500543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760774.663939:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febfb895880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512722176:12345-192.168.203.56@tcp:4:dd.0 Request processed in 25053us (25480us total) trans 4294974032 rc 0/0 00000100:00100000:3.0:1789760774.663948:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6743 00000100:00100000:3.0:1789760774.663958:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 754158523127 time 754203169107 state 0x2 ref 0 00000100:00100000:0.0:1789760774.668316:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760774.668408:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512722304 00000100:00100000:1.0:1789760774.668421:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512722304 00000100:00100000:1.0:1789760774.668497:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 754158523127 time 754203169107 state 0x0 ref 1 00000100:00100000:1.0:1789760774.668517:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 754203429523 time 754203169107 state 0x1 ref 1 00000100:00100000:1.0:1789760774.668549:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 754203429523 time 754232848192 state 0x0 ref 1 00000100:00100000:1.0:1789760774.668562:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2879500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512722304:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760774.668630:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27500544, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0xd807 00010000:00010000:1.0:1789760774.668642:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2f8c0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.668662:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=754s v=(0 0) 00010000:00010000:1.0:1789760774.668668:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760774.668674:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2f8c0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27500544->27504639] (req 27500544->27504639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.668688:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2f8c0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27500544->27504639] (req 27500544->27504639) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760774.716746:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974033 is committed 00002000:00010000:0.0:1789760774.716951:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27500544 -> 27504640 00010000:00010000:0.0:1789760774.716970:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2f8c0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27500544->27504639] (req 27500544->27504639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.716992:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2f8c0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27500544->27504639] (req 27500544->27504639) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.717002:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2f8c0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27500544->27504639] (req 27500544->27504639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.717009:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760774.717015:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2f8c0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27500544->27504639] (req 27500544->27504639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.717025:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2f8c0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27500544->27504639] (req 27500544->27504639) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.717046:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2f8c0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27500544->27504639] (req 27500544->27504639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.717054:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2f8c0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27500544->27504639] (req 27500544->27504639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760774.717266:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2879500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512722304:12345-192.168.203.56@tcp:4:dd.0 Request processed in 48700us (48959us total) trans 4294974033 rc 0/0 00000100:00100000:0.0:1789760774.717278:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6744 00000100:00100000:0.0:1789760774.717291:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 754203429523 time 754232848192 state 0x2 ref 0 00000100:00100000:2.0:1789760774.723416:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760774.724333:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512722432 00000100:00100000:3.0:1789760774.724348:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512722432 00000100:00100000:2.0:1789760774.724522:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 754203429523 time 754232848192 state 0x0 ref 1 00000100:00100000:2.0:1789760774.724544:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 754233108608 time 754232848192 state 0x1 ref 1 00000100:00100000:2.0:1789760774.724586:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 754233108608 time 754288885156 state 0x0 ref 1 00000100:00100000:2.0:1789760774.724601:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac3301180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512722432:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760774.724672:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27504640, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0xd807 00010000:00010000:2.0:1789760774.724683:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2f8c7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.724729:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=754s v=(0 0) 00010000:00010000:2.0:1789760774.724734:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760774.724742:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2f8c7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27504640->27508735] (req 27504640->27508735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.724759:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2f8c7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27504640->27508735] (req 27504640->27508735) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760774.765587:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27504640 -> 27508736 00010000:00010000:1.0:1789760774.765615:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2f8c7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27504640->27508735] (req 27504640->27508735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.765631:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2f8c7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27504640->27508735] (req 27504640->27508735) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.765642:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2f8c7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27504640->27508735] (req 27504640->27508735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.765650:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760774.765655:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2f8c7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27504640->27508735] (req 27504640->27508735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.765666:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2f8c7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27504640->27508735] (req 27504640->27508735) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.765691:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2f8c7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27504640->27508735] (req 27504640->27508735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.765745:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2f8c7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27504640->27508735] (req 27504640->27508735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760774.765855:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac3301180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2359:x1876699512722432:12345-192.168.203.56@tcp:4:dd.0 Request processed in 41250us (42448us total) trans 4294974034 rc 0/0 00000100:00100000:1.0:1789760774.765868:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6745 00000100:00100000:1.0:1789760774.765880:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 754233108608 time 754288885156 state 0x2 ref 0 00000001:00080000:3.0:1789760774.766396:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974034 is committed 00000100:00100000:3.0:1789760774.770880:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760774.771061:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512722560 00000100:00100000:2.0:1789760774.771074:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512722560 00000100:00100000:2.0:1789760774.771153:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 754233108608 time 754288885156 state 0x0 ref 1 00000100:00100000:2.0:1789760774.771172:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 754289145572 time 754288885156 state 0x1 ref 1 00000100:00100000:2.0:1789760774.771205:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 754289145572 time 754335504493 state 0x0 ref 1 00000100:00100000:2.0:1789760774.771220:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4076a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512722560:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760774.771290:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27508736, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0xd807 00010000:00010000:2.0:1789760774.771302:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2f8ce lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.771322:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=754s v=(0 0) 00010000:00010000:2.0:1789760774.771327:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760774.771335:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2f8ce lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27508736->27512831] (req 27508736->27512831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.771356:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2f8ce lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27508736->27512831] (req 27508736->27512831) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760774.820491:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974035 is committed 00002000:00010000:0.0:1789760774.820595:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27508736 -> 27512832 00010000:00010000:0.0:1789760774.820617:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2f8ce lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27508736->27512831] (req 27508736->27512831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.820633:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2f8ce lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27508736->27512831] (req 27508736->27512831) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.820643:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2f8ce lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27508736->27512831] (req 27508736->27512831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.820652:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760774.820667:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2f8ce lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27508736->27512831] (req 27508736->27512831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.820679:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2f8ce lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27508736->27512831] (req 27508736->27512831) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.820725:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2f8ce lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27508736->27512831] (req 27508736->27512831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.820733:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2f8ce lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27508736->27512831] (req 27508736->27512831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760774.820861:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4076a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512722560:12345-192.168.203.56@tcp:4:dd.0 Request processed in 49637us (49989us total) trans 4294974035 rc 0/0 00000100:00100000:0.0:1789760774.820873:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6746 00000100:00100000:0.0:1789760774.820886:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 754289145572 time 754335504493 state 0x2 ref 0 00000100:00100000:0.0:1789760774.826448:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760774.826551:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512722688 00000100:00100000:1.0:1789760774.826564:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512722688 00000100:00100000:1.0:1789760774.826641:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 754289145572 time 754335504493 state 0x0 ref 1 00000100:00100000:1.0:1789760774.826661:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 754335764909 time 754335504493 state 0x1 ref 1 00000100:00100000:1.0:1789760774.826769:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 754335764909 time 754391066523 state 0x0 ref 1 00000100:00100000:1.0:1789760774.826784:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac287b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512722688:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760774.826851:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27512832, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0xd807 00010000:00010000:1.0:1789760774.826862:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2f8d5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.826889:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=754s v=(0 0) 00010000:00010000:1.0:1789760774.826895:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760774.826902:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2f8d5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27512832->27516927] (req 27512832->27516927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.826919:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2f8d5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27512832->27516927] (req 27512832->27516927) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760774.859005:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974036 is committed 00002000:00010000:1.0:1789760774.859153:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27512832 -> 27516928 00010000:00010000:1.0:1789760774.859176:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2f8d5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27512832->27516927] (req 27512832->27516927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.859192:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2f8d5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27512832->27516927] (req 27512832->27516927) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.859202:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2f8d5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27512832->27516927] (req 27512832->27516927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.859211:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760774.859217:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2f8d5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27512832->27516927] (req 27512832->27516927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.859228:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2f8d5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27512832->27516927] (req 27512832->27516927) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.859254:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2f8d5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27512832->27516927] (req 27512832->27516927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.859264:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2f8d5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27512832->27516927] (req 27512832->27516927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760774.859401:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac287b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512722688:12345-192.168.203.56@tcp:4:dd.0 Request processed in 32612us (32959us total) trans 4294974036 rc 0/0 00000100:00100000:1.0:1789760774.859414:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6747 00000100:00100000:1.0:1789760774.859427:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 754335764909 time 754391066523 state 0x2 ref 0 00000100:00100000:2.0:1789760774.864650:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760774.864961:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512722816 00000100:00100000:1.0:1789760774.864973:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512722816 00000100:00100000:1.0:1789760774.865045:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 754335764909 time 754391066523 state 0x0 ref 1 00000100:00100000:1.0:1789760774.865065:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 754391326939 time 754391066523 state 0x1 ref 1 00000100:00100000:1.0:1789760774.865146:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 754391326939 time 754429444353 state 0x0 ref 1 00000100:00100000:1.0:1789760774.865160:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac3303480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512722816:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760774.865234:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27516928, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0xd807 00010000:00010000:1.0:1789760774.865245:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2f8dc lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.865266:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=754s v=(0 0) 00010000:00010000:1.0:1789760774.865272:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760774.865280:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2f8dc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27516928->27521023] (req 27516928->27521023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.865303:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2f8dc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27516928->27521023] (req 27516928->27521023) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760774.917640:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974037 is committed 00002000:00010000:1.0:1789760774.918156:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27516928 -> 27521024 00010000:00010000:1.0:1789760774.918177:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2f8dc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27516928->27521023] (req 27516928->27521023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.918193:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2f8dc lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27516928->27521023] (req 27516928->27521023) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.918203:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2f8dc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27516928->27521023] (req 27516928->27521023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.918211:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760774.918217:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2f8dc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27516928->27521023] (req 27516928->27521023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.918226:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2f8dc lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27516928->27521023] (req 27516928->27521023) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.918250:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2f8dc lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27516928->27521023] (req 27516928->27521023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.918259:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2f8dc lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27516928->27521023] (req 27516928->27521023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760774.918447:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac3303480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512722816:12345-192.168.203.56@tcp:4:dd.0 Request processed in 53281us (53805us total) trans 4294974037 rc 0/0 00000100:00100000:1.0:1789760774.918460:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6748 00000100:00100000:1.0:1789760774.918472:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 754391326939 time 754429444353 state 0x2 ref 0 00000100:00100000:3.0:1789760774.922463:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760774.922675:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512722944 00000100:00100000:0.0:1789760774.922689:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512722944 00000100:00100000:0.0:1789760774.922893:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 754391326939 time 754429444353 state 0x0 ref 1 00000100:00100000:0.0:1789760774.922914:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 754429704769 time 754429444353 state 0x1 ref 1 00000100:00100000:0.0:1789760774.922986:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 754429704769 time 754487284011 state 0x0 ref 1 00000100:00100000:0.0:1789760774.923001:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4074e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512722944:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760774.923067:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27521024, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0xd807 00010000:00010000:0.0:1789760774.923078:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2f8e3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.923110:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=754s v=(0 0) 00010000:00010000:0.0:1789760774.923116:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760774.923124:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2f8e3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27521024->27525119] (req 27521024->27525119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.923142:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2f8e3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27521024->27525119] (req 27521024->27525119) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760774.952213:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974038 is committed 00002000:00010000:3.0:1789760774.952291:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27521024 -> 27525120 00010000:00010000:3.0:1789760774.952310:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2f8e3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27521024->27525119] (req 27521024->27525119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.952322:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2f8e3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27521024->27525119] (req 27521024->27525119) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.952329:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2f8e3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27521024->27525119] (req 27521024->27525119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.952335:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760774.952339:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2f8e3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27521024->27525119] (req 27521024->27525119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.952346:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2f8e3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27521024->27525119] (req 27521024->27525119) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.952373:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2f8e3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27521024->27525119] (req 27521024->27525119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.952378:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2f8e3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27521024->27525119] (req 27521024->27525119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760774.952481:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4074e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512722944:12345-192.168.203.56@tcp:4:dd.0 Request processed in 29476us (30027us total) trans 4294974038 rc 0/0 00000100:00100000:3.0:1789760774.952493:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6749 00000100:00100000:3.0:1789760774.952505:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 754429704769 time 754487284011 state 0x2 ref 0 00000100:00100000:1.0:1789760774.960575:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760774.961098:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512723072 00000100:00100000:0.0:1789760774.961114:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512723072 00000100:00100000:0.0:1789760774.961189:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 754429704769 time 754487284011 state 0x0 ref 1 00000100:00100000:0.0:1789760774.961208:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 754487544427 time 754487284011 state 0x1 ref 1 00000100:00100000:0.0:1789760774.961247:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 754487544427 time 754525545656 state 0x0 ref 1 00000100:00100000:0.0:1789760774.961262:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2746a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512723072:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760774.961329:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27525120, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0xd807 00010000:00010000:0.0:1789760774.961340:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2f8ea lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.961362:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=754s v=(0 0) 00010000:00010000:0.0:1789760774.961368:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760774.961376:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2f8ea lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27525120->27529215] (req 27525120->27529215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.961392:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2f8ea lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27525120->27529215] (req 27525120->27529215) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760774.996297:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974039 is committed 00002000:00010000:1.0:1789760774.996417:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27525120 -> 27529216 00010000:00010000:1.0:1789760774.996438:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2f8ea lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27525120->27529215] (req 27525120->27529215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.996525:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2f8ea lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27525120->27529215] (req 27525120->27529215) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.996536:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2f8ea lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27525120->27529215] (req 27525120->27529215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.996544:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760774.996549:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2f8ea lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27525120->27529215] (req 27525120->27529215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.996560:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2f8ea lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27525120->27529215] (req 27525120->27529215) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.996583:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2f8ea lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27525120->27529215] (req 27525120->27529215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.996596:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2f8ea lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27525120->27529215] (req 27525120->27529215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760774.996804:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2746a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512723072:12345-192.168.203.56@tcp:4:dd.0 Request processed in 35537us (36241us total) trans 4294974039 rc 0/0 00000100:00100000:1.0:1789760774.996817:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6750 00000100:00100000:1.0:1789760774.996829:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 754487544427 time 754525545656 state 0x2 ref 0 00000100:00100000:2.0:1789760775.003245:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760775.003975:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512723200 00000100:00100000:1.0:1789760775.003992:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512723200 00000100:00080000:2.0:1789760775.004059:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1789760775.004166:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 754487544427 time 754525545656 state 0x0 ref 1 00000100:00100000:0.0:1789760775.004198:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 754525806072 time 754525545656 state 0x1 ref 1 00000100:00100000:0.0:1789760775.004269:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 754525806072 time 754568567480 state 0x0 ref 1 00000100:00100000:0.0:1789760775.004284:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2fb6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512723200:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760775.004352:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27529216, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0xd807 00010000:00010000:0.0:1789760775.004365:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2f8f1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.004386:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=754s v=(0 0) 00010000:00010000:0.0:1789760775.004392:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760775.004400:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2f8f1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27529216->27533311] (req 27529216->27533311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.004416:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2f8f1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27529216->27533311] (req 27529216->27533311) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760775.032080:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974040 is committed 00002000:00010000:3.0:1789760775.032353:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27529216 -> 27533312 00010000:00010000:3.0:1789760775.032375:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2f8f1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27529216->27533311] (req 27529216->27533311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.032392:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2f8f1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27529216->27533311] (req 27529216->27533311) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.032403:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2f8f1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27529216->27533311] (req 27529216->27533311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.032412:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760775.032418:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2f8f1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27529216->27533311] (req 27529216->27533311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.032426:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2f8f1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27529216->27533311] (req 27529216->27533311) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.032453:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2f8f1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27529216->27533311] (req 27529216->27533311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.032461:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2f8f1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27529216->27533311] (req 27529216->27533311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760775.032578:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2fb6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512723200:12345-192.168.203.56@tcp:4:dd.0 Request processed in 28289us (29342us total) trans 4294974040 rc 0/0 00000100:00100000:3.0:1789760775.032591:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6751 00000100:00100000:3.0:1789760775.032848:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 754525806072 time 754568567480 state 0x2 ref 0 00000100:00100000:3.0:1789760775.037034:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760775.037217:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512723328 00000100:00100000:0.0:1789760775.037229:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512723328 00000100:00100000:0.0:1789760775.037299:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 754525806072 time 754568567480 state 0x0 ref 1 00000100:00100000:0.0:1789760775.037318:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 754568827896 time 754568567480 state 0x1 ref 1 00000100:00100000:0.0:1789760775.037354:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 754568827896 time 754601653488 state 0x0 ref 1 00000100:00100000:0.0:1789760775.037368:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febea488850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512723328:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760775.037632:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27533312, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0xd807 00010000:00010000:0.0:1789760775.037644:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2f8f8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.037664:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=754s v=(0 0) 00010000:00010000:0.0:1789760775.037671:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760775.037679:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2f8f8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27533312->27537407] (req 27533312->27537407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.037823:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2f8f8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27533312->27537407] (req 27533312->27537407) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760775.067969:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974041 is committed 00002000:00010000:3.0:1789760775.068007:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27533312 -> 27537408 00010000:00010000:3.0:1789760775.068028:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2f8f8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27533312->27537407] (req 27533312->27537407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.068041:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2f8f8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27533312->27537407] (req 27533312->27537407) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.068048:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2f8f8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27533312->27537407] (req 27533312->27537407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.068053:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760775.068057:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2f8f8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27533312->27537407] (req 27533312->27537407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.068064:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2f8f8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27533312->27537407] (req 27533312->27537407) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.068095:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2f8f8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27533312->27537407] (req 27533312->27537407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.068104:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2f8f8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27533312->27537407] (req 27533312->27537407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760775.068212:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febea488850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512723328:12345-192.168.203.56@tcp:4:dd.0 Request processed in 30839us (31184us total) trans 4294974041 rc 0/0 00000100:00100000:3.0:1789760775.068228:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6752 00000100:00100000:3.0:1789760775.068240:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 754568827896 time 754601653488 state 0x2 ref 0 00000100:00100000:1.0:1789760775.071219:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760775.071351:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512723456 00000100:00100000:0.0:1789760775.071367:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512723456 00000100:00100000:0.0:1789760775.071455:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 754568827896 time 754601653488 state 0x0 ref 1 00000100:00100000:0.0:1789760775.071474:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 754601913904 time 754601653488 state 0x1 ref 1 00000100:00100000:0.0:1789760775.071527:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 754601913904 time 754635826362 state 0x0 ref 1 00000100:00100000:0.0:1789760775.071541:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2745880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512723456:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760775.071609:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27537408, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0xd807 00010000:00010000:0.0:1789760775.071619:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f8ff lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.071641:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=754s v=(0 0) 00010000:00010000:0.0:1789760775.071646:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760775.071652:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f8ff lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27537408->27541503] (req 27537408->27541503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.071668:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f8ff lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27537408->27541503] (req 27537408->27541503) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760775.111247:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974042 is committed 00002000:00010000:3.0:1789760775.111811:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27537408 -> 27541504 00010000:00010000:3.0:1789760775.111832:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f8ff lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27537408->27541503] (req 27537408->27541503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.111858:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f8ff lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27537408->27541503] (req 27537408->27541503) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.111869:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f8ff lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27537408->27541503] (req 27537408->27541503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.111876:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760775.111882:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f8ff lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27537408->27541503] (req 27537408->27541503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.111893:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f8ff lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27537408->27541503] (req 27537408->27541503) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.111919:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f8ff lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27537408->27541503] (req 27537408->27541503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.111928:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f8ff lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27537408->27541503] (req 27537408->27541503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760775.112060:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2745880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512723456:12345-192.168.203.56@tcp:4:dd.0 Request processed in 40513us (40852us total) trans 4294974042 rc 0/0 00000100:00100000:3.0:1789760775.112073:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6753 00000100:00100000:3.0:1789760775.112094:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 754601913904 time 754635826362 state 0x2 ref 0 00000100:00100000:2.0:1789760775.117935:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760775.118081:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512723584 00000100:00100000:3.0:1789760775.118102:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512723584 00000100:00100000:3.0:1789760775.118172:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 754601913904 time 754635826362 state 0x0 ref 1 00000100:00100000:3.0:1789760775.118190:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 754636086778 time 754635826362 state 0x1 ref 1 00000100:00100000:3.0:1789760775.118222:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 754636086778 time 754682521446 state 0x0 ref 1 00000100:00100000:3.0:1789760775.118234:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2fb5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512723584:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760775.118668:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27541504, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0xd807 00010000:00010000:3.0:1789760775.118680:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2f906 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.118834:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=754s v=(0 0) 00010000:00010000:3.0:1789760775.118896:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760775.118904:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2f906 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27541504->27545599] (req 27541504->27545599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.118922:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2f906 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27541504->27545599] (req 27541504->27545599) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760775.159566:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974043 is committed 00002000:00010000:3.0:1789760775.159669:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27541504 -> 27545600 00010000:00010000:3.0:1789760775.159933:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2f906 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27541504->27545599] (req 27541504->27545599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.159949:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2f906 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27541504->27545599] (req 27541504->27545599) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.159960:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2f906 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27541504->27545599] (req 27541504->27545599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.159968:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760775.159974:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2f906 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27541504->27545599] (req 27541504->27545599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.159984:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2f906 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27541504->27545599] (req 27541504->27545599) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.160010:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2f906 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27541504->27545599] (req 27541504->27545599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.160019:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2f906 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27541504->27545599] (req 27541504->27545599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760775.160193:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2fb5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512723584:12345-192.168.203.56@tcp:4:dd.0 Request processed in 41952us (42266us total) trans 4294974043 rc 0/0 00000100:00100000:3.0:1789760775.160211:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6754 00000100:00100000:3.0:1789760775.160224:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 754636086778 time 754682521446 state 0x2 ref 0 00000100:00100000:1.0:1789760775.167590:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760775.167807:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512723712 00000100:00100000:3.0:1789760775.167821:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512723712 00000100:00100000:3.0:1789760775.167903:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 754636086778 time 754682521446 state 0x0 ref 1 00000100:00100000:3.0:1789760775.167922:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 754682781862 time 754682521446 state 0x1 ref 1 00000100:00100000:3.0:1789760775.167959:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 754682781862 time 754732258098 state 0x0 ref 1 00000100:00100000:3.0:1789760775.167975:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2745180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512723712:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760775.168038:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27545600, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0xd807 00010000:00010000:3.0:1789760775.168049:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2f90d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.168068:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=754s v=(0 0) 00010000:00010000:3.0:1789760775.168073:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760775.168081:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2f90d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27545600->27549695] (req 27545600->27549695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.168107:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2f90d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27545600->27549695] (req 27545600->27549695) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760775.206296:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974044 is committed 00002000:00010000:3.0:1789760775.206333:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27545600 -> 27549696 00010000:00010000:3.0:1789760775.206354:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2f90d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27545600->27549695] (req 27545600->27549695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.206406:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2f90d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27545600->27549695] (req 27545600->27549695) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.206418:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2f90d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27545600->27549695] (req 27545600->27549695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.206426:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760775.206432:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2f90d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27545600->27549695] (req 27545600->27549695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.206443:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2f90d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27545600->27549695] (req 27545600->27549695) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.206471:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2f90d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27545600->27549695] (req 27545600->27549695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.206478:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2f90d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27545600->27549695] (req 27545600->27549695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760775.206614:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2745180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512723712:12345-192.168.203.56@tcp:4:dd.0 Request processed in 38635us (39031us total) trans 4294974044 rc 0/0 00000100:00100000:3.0:1789760775.206626:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6755 00000100:00100000:3.0:1789760775.206653:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 754682781862 time 754732258098 state 0x2 ref 0 00000100:00100000:1.0:1789760775.211310:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760775.211613:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512723840 00000100:00100000:3.0:1789760775.211628:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512723840 00000100:00100000:3.0:1789760775.211879:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 754682781862 time 754732258098 state 0x0 ref 1 00000100:00100000:3.0:1789760775.211902:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 754732518514 time 754732258098 state 0x1 ref 1 00000100:00100000:3.0:1789760775.211940:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 754732518514 time 754776239571 state 0x0 ref 1 00000100:00100000:3.0:1789760775.211956:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2744a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512723840:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760775.212723:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27549696, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0xd807 00010000:00010000:2.0:1789760775.212741:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2f914 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.212762:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=754s v=(0 0) 00010000:00010000:2.0:1789760775.212767:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760775.212777:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2f914 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27549696->27553791] (req 27549696->27553791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.212794:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2f914 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27549696->27553791] (req 27549696->27553791) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760775.252471:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974045 is committed 00002000:00010000:3.0:1789760775.252946:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27549696 -> 27553792 00010000:00010000:3.0:1789760775.252968:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2f914 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27549696->27553791] (req 27549696->27553791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.252982:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2f914 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27549696->27553791] (req 27549696->27553791) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.252991:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2f914 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27549696->27553791] (req 27549696->27553791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.252999:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760775.253005:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2f914 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27549696->27553791] (req 27549696->27553791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.253016:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2f914 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27549696->27553791] (req 27549696->27553791) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.253042:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2f914 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27549696->27553791] (req 27549696->27553791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.253056:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2f914 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27549696->27553791] (req 27549696->27553791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760775.253344:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2744a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512723840:12345-192.168.203.56@tcp:4:dd.0 Request processed in 41385us (42045us total) trans 4294974045 rc 0/0 00000100:00100000:3.0:1789760775.253358:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6756 00000100:00100000:3.0:1789760775.253369:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 754732518514 time 754776239571 state 0x2 ref 0 00000100:00100000:3.0:1789760775.259420:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760775.259515:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512723968 00000100:00100000:0.0:1789760775.259528:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512723968 00000100:00100000:0.0:1789760775.259601:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 754732518514 time 754776239571 state 0x0 ref 1 00000100:00100000:0.0:1789760775.259621:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 754776499987 time 754776239571 state 0x1 ref 1 00000100:00100000:0.0:1789760775.259692:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 754776499987 time 754823990928 state 0x0 ref 1 00000100:00100000:0.0:1789760775.259737:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4076a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512723968:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760775.259902:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27553792, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0xd807 00010000:00010000:0.0:1789760775.259914:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2f91b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.259935:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=754s v=(0 0) 00010000:00010000:0.0:1789760775.259941:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760775.259948:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2f91b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27553792->27557887] (req 27553792->27557887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.259964:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2f91b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27553792->27557887] (req 27553792->27557887) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760775.292011:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974046 is committed 00002000:00010000:1.0:1789760775.292200:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27553792 -> 27557888 00010000:00010000:1.0:1789760775.292220:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2f91b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27553792->27557887] (req 27553792->27557887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.292235:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2f91b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27553792->27557887] (req 27553792->27557887) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.292245:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2f91b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27553792->27557887] (req 27553792->27557887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.292254:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760775.292260:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2f91b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27553792->27557887] (req 27553792->27557887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.292272:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2f91b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27553792->27557887] (req 27553792->27557887) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.292299:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2f91b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27553792->27557887] (req 27553792->27557887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.292308:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2f91b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27553792->27557887] (req 27553792->27557887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760775.292436:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4076a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512723968:12345-192.168.203.56@tcp:4:dd.0 Request processed in 32693us (33022us total) trans 4294974046 rc 0/0 00000100:00100000:1.0:1789760775.292448:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6757 00000100:00100000:1.0:1789760775.292461:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 754776499987 time 754823990928 state 0x2 ref 0 00000100:00100000:1.0:1789760775.299895:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760775.299953:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512724096 00000100:00100000:3.0:1789760775.299967:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512724096 00000100:00100000:3.0:1789760775.300037:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 754776499987 time 754823990928 state 0x0 ref 1 00000100:00100000:3.0:1789760775.300058:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 754824251344 time 754823990928 state 0x1 ref 1 00000100:00100000:1.0:1789760775.300118:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 754824251344 time 754864414265 state 0x0 ref 1 00000100:00100000:1.0:1789760775.300135:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2746300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512724096:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760775.300200:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27557888, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0xd807 00010000:00010000:1.0:1789760775.300211:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2f922 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.300231:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=754s v=(0 0) 00010000:00010000:1.0:1789760775.300237:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760775.300245:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2f922 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27557888->27561983] (req 27557888->27561983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.300260:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2f922 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27557888->27561983] (req 27557888->27561983) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760775.344282:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27557888 -> 27561984 00010000:00010000:2.0:1789760775.344311:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2f922 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27557888->27561983] (req 27557888->27561983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.344333:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2f922 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27557888->27561983] (req 27557888->27561983) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.344344:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2f922 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27557888->27561983] (req 27557888->27561983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.344352:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760775.344358:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2f922 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27557888->27561983] (req 27557888->27561983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.344368:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2f922 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27557888->27561983] (req 27557888->27561983) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.344394:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2f922 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27557888->27561983] (req 27557888->27561983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.344404:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2f922 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27557888->27561983] (req 27557888->27561983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760775.344522:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2746300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2360:x1876699512724096:12345-192.168.203.56@tcp:4:dd.0 Request processed in 44385us (44632us total) trans 4294974047 rc 0/0 00000100:00100000:2.0:1789760775.344535:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6758 00000100:00100000:2.0:1789760775.344548:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 754824251344 time 754864414265 state 0x2 ref 0 00000001:00080000:2.0:1789760775.344599:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974047 is committed 00000100:00100000:2.0:1789760775.348359:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760775.348902:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512724224 00000100:00100000:3.0:1789760775.348916:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512724224 00000100:00100000:3.0:1789760775.348990:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 754824251344 time 754864414265 state 0x0 ref 1 00000100:00100000:3.0:1789760775.349078:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 754864674681 time 754864414265 state 0x1 ref 1 00000100:00100000:3.0:1789760775.349136:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 754864674681 time 754913434328 state 0x0 ref 1 00000100:00100000:3.0:1789760775.349151:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2fb6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512724224:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760775.349217:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27561984, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0xd807 00010000:00010000:3.0:1789760775.349230:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2f929 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.349249:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=754s v=(0 0) 00010000:00010000:3.0:1789760775.349254:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760775.349262:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2f929 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27561984->27566079] (req 27561984->27566079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.349278:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2f929 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27561984->27566079] (req 27561984->27566079) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760775.414972:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974048 is committed 00002000:00010000:1.0:1789760775.415209:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27561984 -> 27566080 00010000:00010000:1.0:1789760775.415231:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2f929 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27561984->27566079] (req 27561984->27566079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.415247:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2f929 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27561984->27566079] (req 27561984->27566079) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.415258:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2f929 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27561984->27566079] (req 27561984->27566079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.415267:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760775.415272:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2f929 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27561984->27566079] (req 27561984->27566079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.415284:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2f929 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27561984->27566079] (req 27561984->27566079) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.415314:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2f929 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27561984->27566079] (req 27561984->27566079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.415324:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2f929 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27561984->27566079] (req 27561984->27566079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760775.415456:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2fb6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512724224:12345-192.168.203.56@tcp:4:dd.0 Request processed in 66302us (67105us total) trans 4294974048 rc 0/0 00000100:00100000:1.0:1789760775.415470:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6759 00000100:00100000:1.0:1789760775.415792:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 754864674681 time 754913434328 state 0x2 ref 0 00000100:00100000:2.0:1789760775.426156:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760775.426788:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512724352 00000100:00100000:1.0:1789760775.426803:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512724352 00000100:00100000:1.0:1789760775.426879:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 754864674681 time 754913434328 state 0x0 ref 1 00000100:00100000:1.0:1789760775.426900:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 754913694744 time 754913434328 state 0x1 ref 1 00000100:00100000:1.0:1789760775.426938:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 754913694744 time 754991237381 state 0x0 ref 1 00000100:00100000:1.0:1789760775.426952:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febc6bd4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512724352:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760775.427073:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27566080, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0xd807 00010000:00010000:1.0:1789760775.427150:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2f930 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.427171:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=754s v=(0 0) 00010000:00010000:1.0:1789760775.427176:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760775.427184:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2f930 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27566080->27570175] (req 27566080->27570175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.427201:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2f930 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27566080->27570175] (req 27566080->27570175) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760775.435833:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760775.435883:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760775.435913:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760775.435949:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512724608 00000100:00100000:1.0:1789760775.435961:0:5847:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512724608 00000100:00100000:1.0:1789760775.436045:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512724736 00000100:00100000:1.0:1789760775.436049:0:5847:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512724736 00000100:00100000:1.0:1789760775.436077:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 504 00000100:00100000:1.0:1789760775.436104:0:5847:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2747b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+8:2359:x1876699512724608:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:1.0:1789760775.436199:0:5847:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2747b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+8:2359:x1876699512724608:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 98us (380us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760775.436208:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 504 00000100:00100000:1.0:1789760775.436225:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 505 00000100:00100000:1.0:1789760775.436232:0:5847:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2747800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:d2c40150-e97b-40da-952b-cb9f18bf7e96+5:2359:x1876699512724736:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:1.0:1789760775.436257:0:5847:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2747800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:d2c40150-e97b-40da-952b-cb9f18bf7e96+5:2359:x1876699512724736:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 26us (344us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760775.436263:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 505 00000100:00100000:1.0:1789760775.436282:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512724480 00000100:00100000:1.0:1789760775.436285:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512724480 00000100:00100000:1.0:1789760775.436313:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 410 00000100:00100000:1.0:1789760775.436319:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febe00c4450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:e6cf7267-9b19-462a-a263-14115df15dab+13:2359:x1876699512724480:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:1.0:1789760775.436346:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febe00c4450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:e6cf7267-9b19-462a-a263-14115df15dab+13:2359:x1876699512724480:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 27us (463us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760775.436352:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 410 00000100:00080000:1.0:1789760775.436384:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00002000:00010000:0.0:1789760775.478367:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27566080 -> 27570176 00010000:00010000:0.0:1789760775.478395:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2f930 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27566080->27570175] (req 27566080->27570175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.478417:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2f930 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27566080->27570175] (req 27566080->27570175) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.478427:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2f930 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27566080->27570175] (req 27566080->27570175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.478435:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760775.478441:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2f930 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27566080->27570175] (req 27566080->27570175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.478451:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2f930 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27566080->27570175] (req 27566080->27570175) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.478478:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2f930 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27566080->27570175] (req 27566080->27570175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.478486:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2f930 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27566080->27570175] (req 27566080->27570175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760775.478612:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febc6bd4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2358:x1876699512724352:12345-192.168.203.56@tcp:4:dd.0 Request processed in 51654us (52463us total) trans 4294974049 rc 0/0 00000100:00100000:0.0:1789760775.478625:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6760 00000100:00100000:0.0:1789760775.478637:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 754913694744 time 754991237381 state 0x2 ref 0 00000001:00080000:3.0:1789760775.478745:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974049 is committed 00000100:00100000:1.0:1789760775.481991:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760775.482109:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512724864 00000100:00100000:2.0:1789760775.482122:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512724864 00000100:00100000:2.0:1789760775.482197:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 754913694744 time 754991237381 state 0x0 ref 1 00000100:00100000:2.0:1789760775.482216:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 754991497797 time 754991237381 state 0x1 ref 1 00000100:00100000:2.0:1789760775.482252:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 754991497797 time 755046551351 state 0x0 ref 1 00000100:00100000:2.0:1789760775.482265:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2745500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512724864:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760775.482328:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27570176, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0xd807 00010000:00010000:2.0:1789760775.482339:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2f937 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.482360:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=755s v=(0 0) 00010000:00010000:2.0:1789760775.482365:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760775.482372:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2f937 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27570176->27574271] (req 27570176->27574271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.482390:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2f937 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27570176->27574271] (req 27570176->27574271) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760775.526299:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974050 is committed 00002000:00010000:3.0:1789760775.526888:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27570176 -> 27574272 00010000:00010000:3.0:1789760775.526910:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2f937 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27570176->27574271] (req 27570176->27574271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.526926:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2f937 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27570176->27574271] (req 27570176->27574271) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.526936:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2f937 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27570176->27574271] (req 27570176->27574271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.526944:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760775.526950:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2f937 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27570176->27574271] (req 27570176->27574271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.526962:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2f937 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27570176->27574271] (req 27570176->27574271) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.526991:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2f937 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27570176->27574271] (req 27570176->27574271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.527000:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2f937 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27570176->27574271] (req 27570176->27574271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760775.527614:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2745500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512724864:12345-192.168.203.56@tcp:4:dd.0 Request processed in 45341us (45628us total) trans 4294974050 rc 0/0 00000100:00100000:3.0:1789760775.527630:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6761 00000100:00100000:3.0:1789760775.527643:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 754991497797 time 755046551351 state 0x2 ref 0 00000100:00100000:1.0:1789760775.531341:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760775.531421:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512724992 00000100:00100000:3.0:1789760775.531439:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512724992 00000100:00100000:3.0:1789760775.531636:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 754991497797 time 755046551351 state 0x0 ref 1 00000100:00100000:3.0:1789760775.531655:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 755046811767 time 755046551351 state 0x1 ref 1 00000100:00100000:3.0:1789760775.531862:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 755046811767 time 755095993391 state 0x0 ref 1 00000100:00100000:2.0:1789760775.531983:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2744e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512724992:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760775.532053:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27574272, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0xd807 00010000:00010000:2.0:1789760775.532066:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2f93e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.532161:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=755s v=(0 0) 00010000:00010000:2.0:1789760775.532168:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760775.532176:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2f93e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27574272->27578367] (req 27574272->27578367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.532193:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2f93e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27574272->27578367] (req 27574272->27578367) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.540170:0:5836:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:2.0:1789760775.568449:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974051 is committed 00002000:00010000:0.0:1789760775.568874:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27574272 -> 27578368 00002000:00010000:0.0:1789760775.568881:0:6708:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760774 -> 1789760775 00002000:00010000:0.0:1789760775.568884:0:6708:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760774 -> 1789760775 00010000:00010000:0.0:1789760775.568900:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2f93e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27574272->27578367] (req 27574272->27578367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.568915:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2f93e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27574272->27578367] (req 27574272->27578367) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.568925:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2f93e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27574272->27578367] (req 27574272->27578367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.569736:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760775.569744:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2f93e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27574272->27578367] (req 27574272->27578367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.569756:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2f93e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27574272->27578367] (req 27574272->27578367) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.569893:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2f93e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27574272->27578367] (req 27574272->27578367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.569902:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2f93e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27574272->27578367] (req 27574272->27578367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760775.570167:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2744e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512724992:12345-192.168.203.56@tcp:4:dd.0 Request processed in 38283us (38829us total) trans 4294974051 rc 0/0 00000100:00100000:1.0:1789760775.570181:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6762 00000100:00100000:1.0:1789760775.570194:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 755046811767 time 755095993391 state 0x2 ref 0 00000100:00100000:1.0:1789760775.576054:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760775.576156:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512725120 00000100:00100000:0.0:1789760775.576170:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512725120 00000100:00100000:0.0:1789760775.576246:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 755046811767 time 755095993391 state 0x0 ref 1 00000100:00100000:0.0:1789760775.576266:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 755096253807 time 755095993391 state 0x1 ref 1 00000100:00100000:0.0:1789760775.576299:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 755096253807 time 755140598160 state 0x0 ref 1 00000100:00100000:0.0:1789760775.576314:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2745f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512725120:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760775.576380:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27578368, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0xd807 00010000:00010000:0.0:1789760775.576390:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2f945 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.576411:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=755s v=(0 0) 00010000:00010000:0.0:1789760775.576416:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760775.576423:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2f945 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27578368->27582463] (req 27578368->27582463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.576439:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2f945 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27578368->27582463] (req 27578368->27582463) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760775.623351:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974052 is committed 00002000:00010000:2.0:1789760775.623440:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27578368 -> 27582464 00010000:00010000:2.0:1789760775.623459:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2f945 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27578368->27582463] (req 27578368->27582463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.623485:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2f945 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27578368->27582463] (req 27578368->27582463) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.623496:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2f945 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27578368->27582463] (req 27578368->27582463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.623504:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760775.623516:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2f945 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27578368->27582463] (req 27578368->27582463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.623527:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2f945 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27578368->27582463] (req 27578368->27582463) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.623552:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2f945 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27578368->27582463] (req 27578368->27582463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.623561:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2f945 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27578368->27582463] (req 27578368->27582463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760775.623676:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2745f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512725120:12345-192.168.203.56@tcp:4:dd.0 Request processed in 47359us (47630us total) trans 4294974052 rc 0/0 00000100:00100000:2.0:1789760775.623687:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6763 00000100:00100000:2.0:1789760775.623739:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 755096253807 time 755140598160 state 0x2 ref 0 00000100:00100000:2.0:1789760775.628182:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760775.628357:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512725248 00000100:00100000:3.0:1789760775.628371:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512725248 00000100:00100000:3.0:1789760775.628444:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 755096253807 time 755140598160 state 0x0 ref 1 00000100:00100000:3.0:1789760775.628465:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 755140858576 time 755140598160 state 0x1 ref 1 00000100:00100000:3.0:1789760775.628510:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 755140858576 time 755192808749 state 0x0 ref 1 00000100:00100000:3.0:1789760775.628525:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febd1360380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512725248:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760775.628589:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27582464, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0xd807 00010000:00010000:3.0:1789760775.628601:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2f94c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.628622:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=755s v=(0 0) 00010000:00010000:3.0:1789760775.628627:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760775.628635:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2f94c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27582464->27586559] (req 27582464->27586559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.628652:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2f94c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27582464->27586559] (req 27582464->27586559) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760775.689726:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974053 is committed 00002000:00010000:3.0:1789760775.690062:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27582464 -> 27586560 00010000:00010000:3.0:1789760775.690090:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2f94c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27582464->27586559] (req 27582464->27586559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.690107:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2f94c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27582464->27586559] (req 27582464->27586559) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.690118:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2f94c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27582464->27586559] (req 27582464->27586559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.690126:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760775.690132:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2f94c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27582464->27586559] (req 27582464->27586559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.690143:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2f94c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27582464->27586559] (req 27582464->27586559) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.690167:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2f94c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27582464->27586559] (req 27582464->27586559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.690176:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2f94c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27582464->27586559] (req 27582464->27586559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760775.690346:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febd1360380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512725248:12345-192.168.203.56@tcp:4:dd.0 Request processed in 61816us (62169us total) trans 4294974053 rc 0/0 00000100:00100000:3.0:1789760775.690360:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6764 00000100:00100000:3.0:1789760775.690373:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 755140858576 time 755192808749 state 0x2 ref 0 00000100:00100000:2.0:1789760775.695120:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760775.695313:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512725376 00000100:00100000:1.0:1789760775.695326:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512725376 00000100:00100000:1.0:1789760775.695395:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 755140858576 time 755192808749 state 0x0 ref 1 00000100:00100000:1.0:1789760775.695413:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 755193069165 time 755192808749 state 0x1 ref 1 00000100:00100000:1.0:1789760775.695448:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 755193069165 time 755259747247 state 0x0 ref 1 00000100:00100000:1.0:1789760775.695462:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac437fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512725376:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760775.695861:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27586560, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0xd807 00010000:00010000:1.0:1789760775.695876:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2f953 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.695896:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=755s v=(0 0) 00010000:00010000:1.0:1789760775.695901:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760775.695976:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2f953 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27586560->27590655] (req 27586560->27590655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.696071:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2f953 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27586560->27590655] (req 27586560->27590655) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760775.739167:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27586560 -> 27590656 00010000:00010000:1.0:1789760775.739194:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2f953 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27586560->27590655] (req 27586560->27590655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.739209:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2f953 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27586560->27590655] (req 27586560->27590655) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.739219:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2f953 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27586560->27590655] (req 27586560->27590655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.739227:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760775.739233:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2f953 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27586560->27590655] (req 27586560->27590655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.739243:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2f953 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27586560->27590655] (req 27586560->27590655) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.739266:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2f953 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27586560->27590655] (req 27586560->27590655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.739275:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2f953 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27586560->27590655] (req 27586560->27590655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760775.739399:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac437fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2360:x1876699512725376:12345-192.168.203.56@tcp:4:dd.0 Request processed in 43932us (44285us total) trans 4294974054 rc 0/0 00000100:00100000:1.0:1789760775.739412:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6765 00000100:00100000:1.0:1789760775.739424:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 755193069165 time 755259747247 state 0x2 ref 0 00000001:00080000:1.0:1789760775.739485:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974054 is committed 00000100:00100000:2.0:1789760775.747919:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760775.748040:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512725504 00000100:00100000:0.0:1789760775.748052:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512725504 00000100:00100000:0.0:1789760775.748139:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 755193069165 time 755259747247 state 0x0 ref 1 00000100:00100000:0.0:1789760775.748158:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 755260007663 time 755259747247 state 0x1 ref 1 00000100:00100000:0.0:1789760775.748242:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 755260007663 time 755312540204 state 0x0 ref 1 00000100:00100000:0.0:1789760775.748271:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac437d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512725504:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760775.748340:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27590656, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0xd807 00010000:00010000:0.0:1789760775.748351:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f95a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.748371:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=755s v=(0 0) 00010000:00010000:0.0:1789760775.748376:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760775.748384:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f95a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27590656->27594751] (req 27590656->27594751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.748400:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f95a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27590656->27594751] (req 27590656->27594751) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760775.784452:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974055 is committed 00002000:00010000:2.0:1789760775.784590:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27590656 -> 27594752 00010000:00010000:2.0:1789760775.784610:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f95a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27590656->27594751] (req 27590656->27594751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.784624:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f95a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27590656->27594751] (req 27590656->27594751) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.784635:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f95a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27590656->27594751] (req 27590656->27594751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.784644:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760775.784650:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f95a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27590656->27594751] (req 27590656->27594751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.784660:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f95a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27590656->27594751] (req 27590656->27594751) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.784683:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f95a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27590656->27594751] (req 27590656->27594751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.784691:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f95a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27590656->27594751] (req 27590656->27594751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760775.784925:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac437d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512725504:12345-192.168.203.56@tcp:4:dd.0 Request processed in 36660us (37013us total) trans 4294974055 rc 0/0 00000100:00100000:2.0:1789760775.784939:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6766 00000100:00100000:2.0:1789760775.784951:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 755260007663 time 755312540204 state 0x2 ref 0 00000100:00100000:2.0:1789760775.790002:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760775.790291:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512725632 00000100:00100000:3.0:1789760775.790306:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512725632 00000100:00100000:3.0:1789760775.790446:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 755260007663 time 755312540204 state 0x0 ref 1 00000100:00100000:3.0:1789760775.790467:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 755312800620 time 755312540204 state 0x1 ref 1 00000100:00100000:3.0:1789760775.790540:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 755312800620 time 755354830816 state 0x0 ref 1 00000100:00100000:3.0:1789760775.790557:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac437df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512725632:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760775.790623:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27594752, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0xd807 00010000:00010000:3.0:1789760775.790634:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2f961 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.790656:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=755s v=(0 0) 00010000:00010000:3.0:1789760775.790661:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760775.790671:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2f961 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27594752->27598847] (req 27594752->27598847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.790687:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2f961 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27594752->27598847] (req 27594752->27598847) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760775.824086:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27594752 -> 27598848 00010000:00010000:3.0:1789760775.824115:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2f961 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27594752->27598847] (req 27594752->27598847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.824131:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2f961 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27594752->27598847] (req 27594752->27598847) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.824141:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2f961 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27594752->27598847] (req 27594752->27598847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.824149:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760775.824155:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2f961 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27594752->27598847] (req 27594752->27598847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.824166:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2f961 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27594752->27598847] (req 27594752->27598847) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.824190:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2f961 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27594752->27598847] (req 27594752->27598847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.824201:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2f961 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27594752->27598847] (req 27594752->27598847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760775.824332:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac437df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2360:x1876699512725632:12345-192.168.203.56@tcp:4:dd.0 Request processed in 33771us (34336us total) trans 4294974056 rc 0/0 00000100:00100000:3.0:1789760775.824344:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6767 00000100:00100000:3.0:1789760775.824356:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 755312800620 time 755354830816 state 0x2 ref 0 00000001:00080000:0.0:1789760775.825042:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974056 is committed 00000100:00100000:3.0:1789760775.832536:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760775.832829:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512725760 00000100:00100000:1.0:1789760775.832843:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512725760 00000100:00100000:1.0:1789760775.832921:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 755312800620 time 755354830816 state 0x0 ref 1 00000100:00100000:1.0:1789760775.832943:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 755355091232 time 755354830816 state 0x1 ref 1 00000100:00100000:1.0:1789760775.832983:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 755355091232 time 755397282167 state 0x0 ref 1 00000100:00100000:1.0:1789760775.832997:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4074e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512725760:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760775.833067:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27598848, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0xd807 00010000:00010000:1.0:1789760775.833091:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2f968 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.833110:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=755s v=(0 0) 00010000:00010000:1.0:1789760775.833115:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760775.833123:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2f968 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27598848->27602943] (req 27598848->27602943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.833139:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2f968 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27598848->27602943] (req 27598848->27602943) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760775.884770:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974057 is committed 00002000:00010000:0.0:1789760775.884962:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27598848 -> 27602944 00010000:00010000:0.0:1789760775.884982:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2f968 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27598848->27602943] (req 27598848->27602943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.884997:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2f968 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27598848->27602943] (req 27598848->27602943) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.885007:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2f968 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27598848->27602943] (req 27598848->27602943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.885015:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760775.885022:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2f968 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27598848->27602943] (req 27598848->27602943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.885031:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2f968 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27598848->27602943] (req 27598848->27602943) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.885054:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2f968 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27598848->27602943] (req 27598848->27602943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.885063:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2f968 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27598848->27602943] (req 27598848->27602943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760775.885192:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4074e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512725760:12345-192.168.203.56@tcp:4:dd.0 Request processed in 52190us (52665us total) trans 4294974057 rc 0/0 00000100:00100000:0.0:1789760775.885204:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6768 00000100:00100000:0.0:1789760775.885216:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 755355091232 time 755397282167 state 0x2 ref 0 00000100:00100000:2.0:1789760775.889534:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760775.889644:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512725888 00000100:00100000:0.0:1789760775.889657:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512725888 00000100:00100000:0.0:1789760775.889785:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 755355091232 time 755397282167 state 0x0 ref 1 00000100:00100000:0.0:1789760775.889805:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 755397542583 time 755397282167 state 0x1 ref 1 00000100:00100000:0.0:1789760775.889840:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 755397542583 time 755454139148 state 0x0 ref 1 00000100:00100000:0.0:1789760775.889853:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac437e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512725888:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760775.889923:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27602944, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0xd807 00010000:00010000:0.0:1789760775.889935:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f96f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.889955:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=755s v=(0 0) 00010000:00010000:0.0:1789760775.889960:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760775.889967:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f96f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27602944->27607039] (req 27602944->27607039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.889983:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f96f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27602944->27607039] (req 27602944->27607039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760775.920223:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974058 is committed 00002000:00010000:0.0:1789760775.920299:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27602944 -> 27607040 00010000:00010000:0.0:1789760775.920319:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f96f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27602944->27607039] (req 27602944->27607039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.920335:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f96f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27602944->27607039] (req 27602944->27607039) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.920345:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f96f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27602944->27607039] (req 27602944->27607039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.920353:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760775.920358:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f96f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27602944->27607039] (req 27602944->27607039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.920369:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f96f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27602944->27607039] (req 27602944->27607039) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.920399:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f96f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27602944->27607039] (req 27602944->27607039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.920405:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f96f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27602944->27607039] (req 27602944->27607039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760775.920617:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac437e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512725888:12345-192.168.203.56@tcp:4:dd.0 Request processed in 30651us (30992us total) trans 4294974058 rc 0/0 00000100:00100000:0.0:1789760775.920630:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6769 00000100:00100000:0.0:1789760775.920642:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 755397542583 time 755454139148 state 0x2 ref 0 00000100:00100000:1.0:1789760775.926134:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760775.926473:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512726016 00000100:00100000:3.0:1789760775.926487:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512726016 00000100:00100000:3.0:1789760775.926569:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 755397542583 time 755454139148 state 0x0 ref 1 00000100:00100000:3.0:1789760775.926588:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 755454399564 time 755454139148 state 0x1 ref 1 00000100:00100000:3.0:1789760775.926627:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 755454399564 time 755490925999 state 0x0 ref 1 00000100:00100000:3.0:1789760775.926641:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2746a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512726016:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760775.927570:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27607040, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0xd807 00010000:00010000:3.0:1789760775.927590:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2f976 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.927611:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=755s v=(0 0) 00010000:00010000:3.0:1789760775.927616:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760775.927626:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2f976 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27607040->27611135] (req 27607040->27611135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.927644:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2f976 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27607040->27611135] (req 27607040->27611135) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760775.981433:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974059 is committed 00002000:00010000:2.0:1789760775.981564:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27607040 -> 27611136 00010000:00010000:2.0:1789760775.981587:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2f976 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27607040->27611135] (req 27607040->27611135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.981602:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2f976 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27607040->27611135] (req 27607040->27611135) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.981617:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2f976 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27607040->27611135] (req 27607040->27611135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.981625:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760775.981631:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2f976 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27607040->27611135] (req 27607040->27611135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.981641:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2f976 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27607040->27611135] (req 27607040->27611135) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.981665:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2f976 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27607040->27611135] (req 27607040->27611135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.981673:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2f976 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27607040->27611135] (req 27607040->27611135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760775.981994:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2746a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512726016:12345-192.168.203.56@tcp:4:dd.0 Request processed in 55347us (55865us total) trans 4294974059 rc 0/0 00000100:00100000:2.0:1789760775.982007:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6770 00000100:00100000:2.0:1789760775.982020:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 755454399564 time 755490925999 state 0x2 ref 0 00000100:00100000:3.0:1789760775.985312:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760775.985444:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512726144 00000100:00100000:0.0:1789760775.985457:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512726144 00000100:00100000:0.0:1789760775.985604:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 755454399564 time 755490925999 state 0x0 ref 1 00000100:00100000:0.0:1789760775.985624:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 755491186415 time 755490925999 state 0x1 ref 1 00000100:00100000:2.0:1789760775.985692:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 755491186415 time 755549987846 state 0x0 ref 1 00000100:00100000:3.0:1789760775.986034:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4075500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512726144:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760775.986290:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27611136, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0xd807 00010000:00010000:3.0:1789760775.986302:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2f97d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.986321:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=755s v=(0 0) 00010000:00010000:3.0:1789760775.986326:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760775.986334:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2f97d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27611136->27615231] (req 27611136->27615231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.986351:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2f97d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27611136->27615231] (req 27611136->27615231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760776.031609:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974060 is committed 00002000:00010000:3.0:1789760776.031924:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27611136 -> 27615232 00010000:00010000:3.0:1789760776.031950:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2f97d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27611136->27615231] (req 27611136->27615231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.031968:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2f97d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27611136->27615231] (req 27611136->27615231) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.031988:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2f97d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27611136->27615231] (req 27611136->27615231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.031997:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760776.032002:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2f97d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27611136->27615231] (req 27611136->27615231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.032012:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2f97d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27611136->27615231] (req 27611136->27615231) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.032037:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2f97d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27611136->27615231] (req 27611136->27615231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.032045:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2f97d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27611136->27615231] (req 27611136->27615231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760776.032176:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4075500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512726144:12345-192.168.203.56@tcp:4:dd.0 Request processed in 46285us (46874us total) trans 4294974060 rc 0/0 00000100:00100000:3.0:1789760776.032189:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6771 00000100:00100000:3.0:1789760776.032200:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 755491186415 time 755549987846 state 0x2 ref 0 00000100:00100000:3.0:1789760776.037178:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760776.037287:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512726272 00000100:00100000:0.0:1789760776.037301:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512726272 00000100:00100000:0.0:1789760776.037381:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 755491186415 time 755549987846 state 0x0 ref 1 00000100:00100000:0.0:1789760776.037399:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 755550248262 time 755549987846 state 0x1 ref 1 00000100:00100000:0.0:1789760776.037435:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 755550248262 time 755601734510 state 0x0 ref 1 00000100:00100000:0.0:1789760776.037449:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4076300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512726272:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760776.037515:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27615232, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0xd807 00010000:00010000:0.0:1789760776.037525:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f984 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.037546:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=755s v=(0 0) 00010000:00010000:0.0:1789760776.037552:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760776.037560:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f984 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27615232->27619327] (req 27615232->27619327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.037577:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f984 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27615232->27619327] (req 27615232->27619327) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760776.079329:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974061 is committed 00002000:00010000:2.0:1789760776.079450:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27615232 -> 27619328 00010000:00010000:2.0:1789760776.079539:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f984 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27615232->27619327] (req 27615232->27619327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.079556:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f984 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27615232->27619327] (req 27615232->27619327) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.079567:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f984 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27615232->27619327] (req 27615232->27619327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.079575:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760776.079580:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f984 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27615232->27619327] (req 27615232->27619327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.079590:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f984 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27615232->27619327] (req 27615232->27619327) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.079615:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f984 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27615232->27619327] (req 27615232->27619327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.079624:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2f984 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27615232->27619327] (req 27615232->27619327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760776.080088:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4076300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512726272:12345-192.168.203.56@tcp:4:dd.0 Request processed in 42624us (42908us total) trans 4294974061 rc 0/0 00000100:00100000:2.0:1789760776.080161:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6772 00000100:00100000:2.0:1789760776.080174:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 755550248262 time 755601734510 state 0x2 ref 0 00000100:00100000:1.0:1789760776.085290:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760776.085457:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512726400 00000100:00100000:2.0:1789760776.085475:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512726400 00000100:00100000:2.0:1789760776.085540:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 755550248262 time 755601734510 state 0x0 ref 1 00000100:00100000:2.0:1789760776.085554:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 755601994926 time 755601734510 state 0x1 ref 1 00000100:00100000:2.0:1789760776.085584:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 755601994926 time 755649883932 state 0x0 ref 1 00000100:00100000:2.0:1789760776.085597:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febd02e0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512726400:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760776.085667:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27619328, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0xd807 00010000:00010000:2.0:1789760776.085677:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2f98b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.085721:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=755s v=(0 0) 00010000:00010000:2.0:1789760776.085726:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760776.085734:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2f98b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27619328->27623423] (req 27619328->27623423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.085751:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2f98b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27619328->27623423] (req 27619328->27623423) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760776.126692:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974062 is committed 00002000:00010000:3.0:1789760776.127025:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27619328 -> 27623424 00010000:00010000:3.0:1789760776.127046:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2f98b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27619328->27623423] (req 27619328->27623423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.127064:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2f98b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27619328->27623423] (req 27619328->27623423) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.127074:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2f98b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27619328->27623423] (req 27619328->27623423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.127124:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760776.127131:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2f98b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27619328->27623423] (req 27619328->27623423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.127142:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2f98b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27619328->27623423] (req 27619328->27623423) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.127168:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2f98b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27619328->27623423] (req 27619328->27623423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.127177:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2f98b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27619328->27623423] (req 27619328->27623423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760776.128224:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febd02e0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512726400:12345-192.168.203.56@tcp:4:dd.0 Request processed in 42615us (42935us total) trans 4294974062 rc 0/0 00000100:00100000:3.0:1789760776.128241:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6773 00000100:00100000:3.0:1789760776.128254:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 755601994926 time 755649883932 state 0x2 ref 0 00000100:00100000:3.0:1789760776.140523:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760776.140651:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512726528 00000100:00100000:1.0:1789760776.140664:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512726528 00000100:00100000:1.0:1789760776.140788:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 755601994926 time 755649883932 state 0x0 ref 1 00000100:00100000:1.0:1789760776.140808:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 755650144348 time 755649883932 state 0x1 ref 1 00000100:00100000:3.0:1789760776.140958:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 755650144348 time 755705254227 state 0x0 ref 1 00000100:00100000:3.0:1789760776.140977:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4075180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512726528:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760776.141045:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27623424, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0xd807 00010000:00010000:3.0:1789760776.141057:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2f992 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.141085:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=755s v=(0 0) 00010000:00010000:3.0:1789760776.141091:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760776.141099:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2f992 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27623424->27627519] (req 27623424->27627519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.141115:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2f992 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27623424->27627519] (req 27623424->27627519) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760776.198198:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974063 is committed 00002000:00010000:3.0:1789760776.198302:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27623424 -> 27627520 00010000:00010000:3.0:1789760776.198323:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2f992 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27623424->27627519] (req 27623424->27627519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.198338:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2f992 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27623424->27627519] (req 27623424->27627519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.198348:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2f992 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27623424->27627519] (req 27623424->27627519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.198356:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760776.198363:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2f992 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27623424->27627519] (req 27623424->27627519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.198374:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2f992 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27623424->27627519] (req 27623424->27627519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.198398:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2f992 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27623424->27627519] (req 27623424->27627519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.198407:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2f992 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27623424->27627519] (req 27623424->27627519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760776.198536:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4075180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512726528:12345-192.168.203.56@tcp:4:dd.0 Request processed in 57557us (58019us total) trans 4294974063 rc 0/0 00000100:00100000:3.0:1789760776.198547:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6774 00000100:00100000:3.0:1789760776.198559:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 755650144348 time 755705254227 state 0x2 ref 0 00000100:00100000:2.0:1789760776.203334:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760776.204146:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512726656 00000100:00100000:0.0:1789760776.204161:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512726656 00000100:00100000:0.0:1789760776.204242:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 755650144348 time 755705254227 state 0x0 ref 1 00000100:00100000:0.0:1789760776.204262:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 755705514643 time 755705254227 state 0x1 ref 1 00000100:00100000:0.0:1789760776.204301:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 755705514643 time 755768600152 state 0x0 ref 1 00000100:00100000:0.0:1789760776.204318:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac437f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512726656:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760776.204381:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27627520, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0xd807 00010000:00010000:0.0:1789760776.204391:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2f999 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.204412:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=755s v=(0 0) 00010000:00010000:0.0:1789760776.204417:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760776.204425:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2f999 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27627520->27631615] (req 27627520->27631615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.204441:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2f999 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27627520->27631615] (req 27627520->27631615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760776.259906:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974064 is committed 00002000:00010000:0.0:1789760776.260259:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27627520 -> 27631616 00010000:00010000:0.0:1789760776.260281:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2f999 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27627520->27631615] (req 27627520->27631615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.260297:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2f999 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27627520->27631615] (req 27627520->27631615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.260308:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2f999 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27627520->27631615] (req 27627520->27631615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.260315:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760776.260321:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2f999 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27627520->27631615] (req 27627520->27631615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.260331:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2f999 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27627520->27631615] (req 27627520->27631615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.260357:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2f999 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27627520->27631615] (req 27627520->27631615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.260371:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2f999 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27627520->27631615] (req 27627520->27631615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760776.260497:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac437f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512726656:12345-192.168.203.56@tcp:4:dd.0 Request processed in 56177us (57171us total) trans 4294974064 rc 0/0 00000100:00100000:0.0:1789760776.260510:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6775 00000100:00100000:0.0:1789760776.260522:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 755705514643 time 755768600152 state 0x2 ref 0 00000100:00100000:1.0:1789760776.267833:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760776.267937:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512726784 00000100:00100000:3.0:1789760776.267951:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512726784 00000100:00100000:3.0:1789760776.268106:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 755705514643 time 755768600152 state 0x0 ref 1 00000100:00100000:3.0:1789760776.268241:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 755768860568 time 755768600152 state 0x1 ref 1 00000100:00100000:3.0:1789760776.268281:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 755768860568 time 755832580275 state 0x0 ref 1 00000100:00100000:3.0:1789760776.268684:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febd02e3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512726784:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760776.268862:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27631616, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0xd807 00010000:00010000:3.0:1789760776.268874:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2f9a0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.268896:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=755s v=(0 0) 00010000:00010000:3.0:1789760776.268901:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760776.268909:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2f9a0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27631616->27635711] (req 27631616->27635711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.268925:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2f9a0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27631616->27635711] (req 27631616->27635711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760776.314735:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27631616 -> 27635712 00000001:00080000:1.0:1789760776.314747:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974065 is committed 00010000:00010000:3.0:1789760776.314770:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2f9a0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27631616->27635711] (req 27631616->27635711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.314787:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2f9a0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27631616->27635711] (req 27631616->27635711) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.314797:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2f9a0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27631616->27635711] (req 27631616->27635711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.314806:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760776.314811:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2f9a0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27631616->27635711] (req 27631616->27635711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.314821:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2f9a0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27631616->27635711] (req 27631616->27635711) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.314851:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2f9a0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27631616->27635711] (req 27631616->27635711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.314860:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2f9a0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27631616->27635711] (req 27631616->27635711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760776.314978:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febd02e3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512726784:12345-192.168.203.56@tcp:4:dd.0 Request processed in 46680us (47155us total) trans 4294974065 rc 0/0 00000100:00100000:3.0:1789760776.314989:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6776 00000100:00100000:3.0:1789760776.314999:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 755768860568 time 755832580275 state 0x2 ref 0 00000100:00100000:3.0:1789760776.318655:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760776.318966:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512726912 00000100:00100000:1.0:1789760776.318980:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512726912 00000100:00100000:1.0:1789760776.319056:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 755768860568 time 755832580275 state 0x0 ref 1 00000100:00100000:1.0:1789760776.319086:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 755832840691 time 755832580275 state 0x1 ref 1 00000100:00100000:1.0:1789760776.319126:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 755832840691 time 755883425195 state 0x0 ref 1 00000100:00100000:1.0:1789760776.319140:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4074a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512726912:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760776.319208:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27635712, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0xd807 00010000:00010000:1.0:1789760776.319219:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2f9a7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.319245:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=755s v=(0 0) 00010000:00010000:1.0:1789760776.319251:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760776.319259:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2f9a7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27635712->27639807] (req 27635712->27639807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.319276:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2f9a7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27635712->27639807] (req 27635712->27639807) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760776.351136:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27635712 -> 27639808 00010000:00010000:1.0:1789760776.351160:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2f9a7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27635712->27639807] (req 27635712->27639807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.351175:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2f9a7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27635712->27639807] (req 27635712->27639807) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.351185:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2f9a7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27635712->27639807] (req 27635712->27639807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.351193:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760776.351199:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2f9a7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27635712->27639807] (req 27635712->27639807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.351209:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2f9a7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27635712->27639807] (req 27635712->27639807) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.351232:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2f9a7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27635712->27639807] (req 27635712->27639807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.351241:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2f9a7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27635712->27639807] (req 27635712->27639807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760776.351402:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974066 is committed 00000100:00100000:1.0:1789760776.351463:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4074a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512726912:12345-192.168.203.56@tcp:4:dd.0 Request processed in 32318us (32817us total) trans 4294974066 rc 0/0 00000100:00100000:1.0:1789760776.351477:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6777 00000100:00100000:1.0:1789760776.351489:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 755832840691 time 755883425195 state 0x2 ref 0 00000100:00100000:2.0:1789760776.357322:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760776.357763:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512727040 00000100:00100000:0.0:1789760776.357777:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512727040 00000100:00100000:0.0:1789760776.357856:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 755832840691 time 755883425195 state 0x0 ref 1 00000100:00100000:0.0:1789760776.357876:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 755883685611 time 755883425195 state 0x1 ref 1 00000100:00100000:0.0:1789760776.357919:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 755883685611 time 755922217981 state 0x0 ref 1 00000100:00100000:0.0:1789760776.357934:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac437d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512727040:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760776.358011:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27639808, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0xd807 00010000:00010000:0.0:1789760776.358022:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2f9ae lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.358042:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=755s v=(0 0) 00010000:00010000:0.0:1789760776.358047:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760776.358055:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2f9ae lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27639808->27643903] (req 27639808->27643903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.358073:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2f9ae lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27639808->27643903] (req 27639808->27643903) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760776.400179:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974067 is committed 00002000:00010000:2.0:1789760776.400324:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27639808 -> 27643904 00010000:00010000:2.0:1789760776.400349:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2f9ae lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27639808->27643903] (req 27639808->27643903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.400364:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2f9ae lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27639808->27643903] (req 27639808->27643903) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.400376:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2f9ae lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27639808->27643903] (req 27639808->27643903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.400385:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760776.400391:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2f9ae lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27639808->27643903] (req 27639808->27643903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.400401:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2f9ae lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27639808->27643903] (req 27639808->27643903) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.400425:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2f9ae lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27639808->27643903] (req 27639808->27643903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.400434:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2f9ae lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27639808->27643903] (req 27639808->27643903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760776.400561:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac437d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512727040:12345-192.168.203.56@tcp:4:dd.0 Request processed in 42622us (43249us total) trans 4294974067 rc 0/0 00000100:00100000:2.0:1789760776.400573:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6778 00000100:00100000:2.0:1789760776.400586:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 755883685611 time 755922217981 state 0x2 ref 0 00000100:00100000:3.0:1789760776.404439:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760776.404870:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512727168 00000100:00100000:2.0:1789760776.404884:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512727168 00000100:00100000:2.0:1789760776.404957:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 755883685611 time 755922217981 state 0x0 ref 1 00000100:00100000:2.0:1789760776.404977:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 755922478397 time 755922217981 state 0x1 ref 1 00000100:00100000:2.0:1789760776.405011:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 755922478397 time 755969310695 state 0x0 ref 1 00000100:00100000:2.0:1789760776.405024:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4074700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512727168:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760776.405100:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27643904, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0xd807 00010000:00010000:2.0:1789760776.405110:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2f9b5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.405130:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=755s v=(0 0) 00010000:00010000:2.0:1789760776.405134:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760776.405141:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2f9b5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27643904->27647999] (req 27643904->27647999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.405157:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2f9b5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27643904->27647999] (req 27643904->27647999) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760776.445899:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974068 is committed 00002000:00010000:3.0:1789760776.446071:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27643904 -> 27648000 00010000:00010000:3.0:1789760776.446099:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2f9b5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27643904->27647999] (req 27643904->27647999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.446114:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2f9b5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27643904->27647999] (req 27643904->27647999) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.446123:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2f9b5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27643904->27647999] (req 27643904->27647999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.446130:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760776.446135:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2f9b5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27643904->27647999] (req 27643904->27647999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.446145:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2f9b5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27643904->27647999] (req 27643904->27647999) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.446175:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2f9b5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27643904->27647999] (req 27643904->27647999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.446182:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2f9b5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27643904->27647999] (req 27643904->27647999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760776.446342:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4074700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512727168:12345-192.168.203.56@tcp:4:dd.0 Request processed in 41310us (41909us total) trans 4294974068 rc 0/0 00000100:00100000:3.0:1789760776.446355:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6779 00000100:00100000:3.0:1789760776.446368:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 755922478397 time 755969310695 state 0x2 ref 0 00000100:00100000:0.0:1789760776.450236:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760776.450527:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512727296 00000100:00100000:1.0:1789760776.450540:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512727296 00000100:00100000:1.0:1789760776.450605:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 755922478397 time 755969310695 state 0x0 ref 1 00000100:00100000:1.0:1789760776.450619:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 755969571111 time 755969310695 state 0x1 ref 1 00000100:00100000:1.0:1789760776.450673:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 755969571111 time 756014973135 state 0x0 ref 1 00000100:00100000:1.0:1789760776.450684:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac287bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512727296:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760776.450775:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27648000, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0xd807 00010000:00010000:1.0:1789760776.450784:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2f9bc lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.450803:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=756s v=(0 0) 00010000:00010000:1.0:1789760776.450813:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760776.450819:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2f9bc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27648000->27652095] (req 27648000->27652095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.450836:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2f9bc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27648000->27652095] (req 27648000->27652095) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760776.485735:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974069 is committed 00002000:00010000:1.0:1789760776.485945:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27648000 -> 27652096 00010000:00010000:1.0:1789760776.485966:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2f9bc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27648000->27652095] (req 27648000->27652095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.485981:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2f9bc lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27648000->27652095] (req 27648000->27652095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.485991:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2f9bc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27648000->27652095] (req 27648000->27652095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.485999:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760776.486005:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2f9bc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27648000->27652095] (req 27648000->27652095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.486015:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2f9bc lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27648000->27652095] (req 27648000->27652095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.486042:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2f9bc lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27648000->27652095] (req 27648000->27652095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.486050:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2f9bc lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27648000->27652095] (req 27648000->27652095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760776.486183:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac287bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512727296:12345-192.168.203.56@tcp:4:dd.0 Request processed in 35493us (35955us total) trans 4294974069 rc 0/0 00000100:00100000:1.0:1789760776.486196:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6780 00000100:00100000:1.0:1789760776.486207:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 755969571111 time 756014973135 state 0x2 ref 0 00000100:00100000:2.0:1789760776.492287:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760776.492401:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512727424 00000100:00100000:0.0:1789760776.492415:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512727424 00000100:00100000:0.0:1789760776.492497:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 755969571111 time 756014973135 state 0x0 ref 1 00000100:00100000:0.0:1789760776.492516:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 756015233551 time 756014973135 state 0x1 ref 1 00000100:00100000:0.0:1789760776.492553:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 756015233551 time 756056852584 state 0x0 ref 1 00000100:00100000:0.0:1789760776.492568:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac437c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512727424:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760776.492633:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27652096, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0xd807 00010000:00010000:0.0:1789760776.492645:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f9c3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.492666:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=756s v=(0 0) 00010000:00010000:0.0:1789760776.492671:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760776.492679:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f9c3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27652096->27656191] (req 27652096->27656191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.492968:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f9c3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27652096->27656191] (req 27652096->27656191) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760776.529507:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27652096 -> 27656192 00002000:00010000:0.0:1789760776.529519:0:13062:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760775 -> 1789760776 00002000:00010000:0.0:1789760776.529522:0:13062:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760775 -> 1789760776 00010000:00010000:0.0:1789760776.529539:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f9c3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27652096->27656191] (req 27652096->27656191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.529554:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f9c3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27652096->27656191] (req 27652096->27656191) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.529564:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f9c3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27652096->27656191] (req 27652096->27656191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.529573:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760776.529579:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f9c3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27652096->27656191] (req 27652096->27656191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.529589:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f9c3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27652096->27656191] (req 27652096->27656191) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.529611:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f9c3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27652096->27656191] (req 27652096->27656191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.529618:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f9c3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27652096->27656191] (req 27652096->27656191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760776.529987:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac437c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2361:x1876699512727424:12345-192.168.203.56@tcp:4:dd.0 Request processed in 37410us (37702us total) trans 4294974070 rc 0/0 00000100:00100000:0.0:1789760776.530001:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6781 00000100:00100000:0.0:1789760776.530013:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 756015233551 time 756056852584 state 0x2 ref 0 00000001:00080000:1.0:1789760776.530608:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974070 is committed 00000100:00100000:3.0:1789760776.536918:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760776.538658:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512727552 00000100:00100000:2.0:1789760776.538725:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512727552 00000100:00100000:2.0:1789760776.538791:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 756015233551 time 756056852584 state 0x0 ref 1 00000100:00100000:2.0:1789760776.538806:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 756057113000 time 756056852584 state 0x1 ref 1 00000100:00100000:2.0:1789760776.538839:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 756057113000 time 756103138248 state 0x0 ref 1 00000100:00100000:2.0:1789760776.538850:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2fba680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512727552:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760776.538905:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27656192, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0xd807 00010000:00010000:2.0:1789760776.538914:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2f9ca lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.538930:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=756s v=(0 0) 00010000:00010000:2.0:1789760776.538934:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760776.538940:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2f9ca lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27656192->27660287] (req 27656192->27660287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.538959:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2f9ca lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27656192->27660287] (req 27656192->27660287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.562888:0:5835:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:3.0:1789760776.575460:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974071 is committed 00002000:00010000:2.0:1789760776.575602:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27656192 -> 27660288 00010000:00010000:2.0:1789760776.575624:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2f9ca lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27656192->27660287] (req 27656192->27660287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.575640:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2f9ca lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27656192->27660287] (req 27656192->27660287) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.575650:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2f9ca lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27656192->27660287] (req 27656192->27660287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.575658:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760776.575662:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2f9ca lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27656192->27660287] (req 27656192->27660287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.575673:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2f9ca lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27656192->27660287] (req 27656192->27660287) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.575823:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2f9ca lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27656192->27660287] (req 27656192->27660287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.575834:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2f9ca lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27656192->27660287] (req 27656192->27660287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760776.576004:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2fba680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512727552:12345-192.168.203.56@tcp:4:dd.0 Request processed in 37148us (39088us total) trans 4294974071 rc 0/0 00000100:00100000:2.0:1789760776.576016:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6782 00000100:00100000:2.0:1789760776.576028:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 756057113000 time 756103138248 state 0x2 ref 0 00000100:00100000:2.0:1789760776.582220:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760776.582367:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512727680 00000100:00100000:1.0:1789760776.582381:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512727680 00000100:00100000:1.0:1789760776.582454:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 756057113000 time 756103138248 state 0x0 ref 1 00000100:00100000:1.0:1789760776.582473:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 756103398664 time 756103138248 state 0x1 ref 1 00000100:00100000:1.0:1789760776.582520:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 756103398664 time 756146810849 state 0x0 ref 1 00000100:00100000:1.0:1789760776.582535:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febfc977100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512727680:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760776.582599:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27660288, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0xd807 00010000:00010000:1.0:1789760776.582610:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2f9d1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.582631:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=756s v=(0 0) 00010000:00010000:1.0:1789760776.582637:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760776.582645:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2f9d1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27660288->27664383] (req 27660288->27664383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.582660:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2f9d1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27660288->27664383] (req 27660288->27664383) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760776.630205:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974072 is committed 00002000:00010000:2.0:1789760776.631630:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27660288 -> 27664384 00010000:00010000:2.0:1789760776.631658:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2f9d1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27660288->27664383] (req 27660288->27664383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.631674:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2f9d1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27660288->27664383] (req 27660288->27664383) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.631685:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2f9d1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27660288->27664383] (req 27660288->27664383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.631693:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760776.631753:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2f9d1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27660288->27664383] (req 27660288->27664383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.631769:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2f9d1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27660288->27664383] (req 27660288->27664383) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.631793:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2f9d1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27660288->27664383] (req 27660288->27664383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.631802:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2f9d1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27660288->27664383] (req 27660288->27664383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760776.631941:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febfc977100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512727680:12345-192.168.203.56@tcp:4:dd.0 Request processed in 49400us (49730us total) trans 4294974072 rc 0/0 00000100:00100000:2.0:1789760776.631954:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6783 00000100:00100000:2.0:1789760776.631967:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 756103398664 time 756146810849 state 0x2 ref 0 00000100:00100000:3.0:1789760776.637043:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760776.637223:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512727808 00000100:00100000:0.0:1789760776.637238:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512727808 00000100:00100000:0.0:1789760776.637308:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 756103398664 time 756146810849 state 0x0 ref 1 00000100:00100000:0.0:1789760776.637328:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 756147071265 time 756146810849 state 0x1 ref 1 00000100:00100000:0.0:1789760776.637398:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 756147071265 time 756201695560 state 0x0 ref 1 00000100:00100000:0.0:1789760776.637413:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2fb8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512727808:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760776.637479:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27664384, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0xd807 00010000:00010000:0.0:1789760776.637489:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f9d8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.637511:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=756s v=(0 0) 00010000:00010000:0.0:1789760776.637517:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760776.637524:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f9d8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27664384->27668479] (req 27664384->27668479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.637540:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f9d8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27664384->27668479] (req 27664384->27668479) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760776.677644:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27664384 -> 27668480 00010000:00010000:2.0:1789760776.677674:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f9d8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27664384->27668479] (req 27664384->27668479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.677691:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f9d8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27664384->27668479] (req 27664384->27668479) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.677733:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f9d8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27664384->27668479] (req 27664384->27668479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.677742:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760776.677749:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f9d8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27664384->27668479] (req 27664384->27668479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.677759:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f9d8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27664384->27668479] (req 27664384->27668479) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.677783:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f9d8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27664384->27668479] (req 27664384->27668479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.677791:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2f9d8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27664384->27668479] (req 27664384->27668479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760776.677886:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974073 is committed 00000100:00100000:0.0:1789760776.677986:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2fb8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512727808:12345-192.168.203.56@tcp:4:dd.0 Request processed in 40569us (40954us total) trans 4294974073 rc 0/0 00000100:00100000:0.0:1789760776.678000:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6784 00000100:00100000:0.0:1789760776.678011:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 756147071265 time 756201695560 state 0x2 ref 0 00000100:00100000:3.0:1789760776.685958:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760776.686040:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512727936 00000100:00100000:2.0:1789760776.686053:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512727936 00000100:00100000:2.0:1789760776.686135:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 756147071265 time 756201695560 state 0x0 ref 1 00000100:00100000:2.0:1789760776.686155:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 756201955976 time 756201695560 state 0x1 ref 1 00000100:00100000:2.0:1789760776.686196:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 756201955976 time 756250494522 state 0x0 ref 1 00000100:00100000:2.0:1789760776.686210:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2fb8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512727936:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760776.686275:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27668480, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0xd807 00010000:00010000:2.0:1789760776.686285:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2f9df lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.686306:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=756s v=(0 0) 00010000:00010000:2.0:1789760776.686310:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760776.686319:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2f9df lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27668480->27672575] (req 27668480->27672575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.686334:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2f9df lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27668480->27672575] (req 27668480->27672575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760776.717643:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974074 is committed 00002000:00010000:2.0:1789760776.717962:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27668480 -> 27672576 00010000:00010000:2.0:1789760776.717983:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2f9df lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27668480->27672575] (req 27668480->27672575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.717997:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2f9df lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27668480->27672575] (req 27668480->27672575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.718008:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2f9df lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27668480->27672575] (req 27668480->27672575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.718016:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760776.718022:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2f9df lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27668480->27672575] (req 27668480->27672575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.718032:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2f9df lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27668480->27672575] (req 27668480->27672575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.718060:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2f9df lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27668480->27672575] (req 27668480->27672575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.718068:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2f9df lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27668480->27672575] (req 27668480->27672575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760776.718200:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2fb8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512727936:12345-192.168.203.56@tcp:4:dd.0 Request processed in 31985us (32249us total) trans 4294974074 rc 0/0 00000100:00100000:2.0:1789760776.718211:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6785 00000100:00100000:2.0:1789760776.718223:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 756201955976 time 756250494522 state 0x2 ref 0 00000100:00100000:2.0:1789760776.722519:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760776.722634:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512728064 00000100:00100000:1.0:1789760776.722646:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512728064 00000100:00100000:1.0:1789760776.722914:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 756201955976 time 756250494522 state 0x0 ref 1 00000100:00100000:1.0:1789760776.722935:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 756250754938 time 756250494522 state 0x1 ref 1 00000100:00100000:0.0:1789760776.723090:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 756250754938 time 756287386417 state 0x0 ref 1 00000100:00100000:0.0:1789760776.723109:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2875c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512728064:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760776.723174:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27672576, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0xd807 00010000:00010000:0.0:1789760776.723185:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2f9e6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.723204:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=756s v=(0 0) 00010000:00010000:0.0:1789760776.723209:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760776.723216:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2f9e6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27672576->27676671] (req 27672576->27676671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.723233:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2f9e6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27672576->27676671] (req 27672576->27676671) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760776.767925:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974075 is committed 00002000:00010000:0.0:1789760776.768441:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27672576 -> 27676672 00010000:00010000:0.0:1789760776.768462:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2f9e6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27672576->27676671] (req 27672576->27676671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.768485:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2f9e6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27672576->27676671] (req 27672576->27676671) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.768497:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2f9e6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27672576->27676671] (req 27672576->27676671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.768506:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760776.768513:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2f9e6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27672576->27676671] (req 27672576->27676671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.768523:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2f9e6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27672576->27676671] (req 27672576->27676671) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.769590:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2f9e6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27672576->27676671] (req 27672576->27676671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.769599:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2f9e6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27672576->27676671] (req 27672576->27676671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760776.769768:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2875c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512728064:12345-192.168.203.56@tcp:4:dd.0 Request processed in 46657us (47254us total) trans 4294974075 rc 0/0 00000100:00100000:0.0:1789760776.769781:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6786 00000100:00100000:0.0:1789760776.769793:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 756250754938 time 756287386417 state 0x2 ref 0 00000100:00100000:3.0:1789760776.775959:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760776.776093:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512728192 00000100:00100000:0.0:1789760776.776107:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512728192 00000100:00100000:0.0:1789760776.776183:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 756250754938 time 756287386417 state 0x0 ref 1 00000100:00100000:0.0:1789760776.776203:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 756287646833 time 756287386417 state 0x1 ref 1 00000100:00100000:0.0:1789760776.776242:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 756287646833 time 756340541250 state 0x0 ref 1 00000100:00100000:0.0:1789760776.776257:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac28a2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512728192:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760776.776324:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27676672, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0xd807 00010000:00010000:0.0:1789760776.776334:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2f9ed lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.776353:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=756s v=(0 0) 00010000:00010000:0.0:1789760776.776359:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760776.776366:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2f9ed lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27676672->27680767] (req 27676672->27680767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.776384:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2f9ed lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27676672->27680767] (req 27676672->27680767) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760776.833614:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27676672 -> 27680768 00010000:00010000:3.0:1789760776.833640:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2f9ed lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27676672->27680767] (req 27676672->27680767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.833662:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2f9ed lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27676672->27680767] (req 27676672->27680767) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.833672:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2f9ed lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27676672->27680767] (req 27676672->27680767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.833680:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760776.833686:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2f9ed lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27676672->27680767] (req 27676672->27680767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.833746:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2f9ed lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27676672->27680767] (req 27676672->27680767) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.833775:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2f9ed lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27676672->27680767] (req 27676672->27680767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.833784:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2f9ed lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27676672->27680767] (req 27676672->27680767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760776.833854:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974076 is committed 00000100:00100000:3.0:1789760776.833914:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac28a2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512728192:12345-192.168.203.56@tcp:4:dd.0 Request processed in 57653us (57964us total) trans 4294974076 rc 0/0 00000100:00100000:3.0:1789760776.833927:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6787 00000100:00100000:3.0:1789760776.833939:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 756287646833 time 756340541250 state 0x2 ref 0 00000100:00100000:3.0:1789760776.840505:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760776.840577:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512728320 00000100:00100000:2.0:1789760776.840591:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512728320 00000100:00100000:2.0:1789760776.840668:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 756287646833 time 756340541250 state 0x0 ref 1 00000100:00100000:2.0:1789760776.840694:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 756340801666 time 756340541250 state 0x1 ref 1 00000100:00100000:2.0:1789760776.840765:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 756340801666 time 756405064894 state 0x0 ref 1 00000100:00100000:2.0:1789760776.840780:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac28a0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512728320:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760776.840845:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27680768, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0xd807 00010000:00010000:2.0:1789760776.840856:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2f9f4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.840875:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=756s v=(0 0) 00010000:00010000:2.0:1789760776.840886:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760776.840894:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2f9f4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27680768->27684863] (req 27680768->27684863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.840915:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2f9f4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27680768->27684863] (req 27680768->27684863) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760776.898563:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974077 is committed 00002000:00010000:2.0:1789760776.899446:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27680768 -> 27684864 00010000:00010000:2.0:1789760776.899512:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2f9f4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27680768->27684863] (req 27680768->27684863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.899530:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2f9f4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27680768->27684863] (req 27680768->27684863) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.899541:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2f9f4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27680768->27684863] (req 27680768->27684863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.899550:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760776.899555:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2f9f4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27680768->27684863] (req 27680768->27684863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.899566:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2f9f4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27680768->27684863] (req 27680768->27684863) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.899652:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2f9f4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27680768->27684863] (req 27680768->27684863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.899663:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2f9f4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27680768->27684863] (req 27680768->27684863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760776.899872:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac28a0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512728320:12345-192.168.203.56@tcp:4:dd.0 Request processed in 59086us (59374us total) trans 4294974077 rc 0/0 00000100:00100000:2.0:1789760776.899884:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6788 00000100:00100000:2.0:1789760776.899897:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 756340801666 time 756405064894 state 0x2 ref 0 00000100:00100000:2.0:1789760776.903350:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760776.903425:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512728448 00000100:00100000:0.0:1789760776.903439:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512728448 00000100:00100000:1.0:1789760776.903620:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 756340801666 time 756405064894 state 0x0 ref 1 00000100:00100000:1.0:1789760776.903641:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 756405325310 time 756405064894 state 0x1 ref 1 00000100:00100000:1.0:1789760776.903684:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 756405325310 time 756467983533 state 0x0 ref 1 00000100:00100000:1.0:1789760776.903724:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2877100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512728448:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760776.903801:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27684864, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0xd807 00010000:00010000:1.0:1789760776.903813:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2f9fb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.903832:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=756s v=(0 0) 00010000:00010000:1.0:1789760776.903837:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760776.903845:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2f9fb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27684864->27688959] (req 27684864->27688959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.903861:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2f9fb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27684864->27688959] (req 27684864->27688959) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760776.953476:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974078 is committed 00002000:00010000:1.0:1789760776.953682:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27684864 -> 27688960 00010000:00010000:1.0:1789760776.953724:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2f9fb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27684864->27688959] (req 27684864->27688959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.953809:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2f9fb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27684864->27688959] (req 27684864->27688959) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.953821:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2f9fb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27684864->27688959] (req 27684864->27688959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.953830:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760776.953835:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2f9fb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27684864->27688959] (req 27684864->27688959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.953846:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2f9fb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27684864->27688959] (req 27684864->27688959) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.953870:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2f9fb lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27684864->27688959] (req 27684864->27688959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.953879:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2f9fb lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27684864->27688959] (req 27684864->27688959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760776.954042:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2877100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512728448:12345-192.168.203.56@tcp:4:dd.0 Request processed in 50340us (50699us total) trans 4294974078 rc 0/0 00000100:00100000:1.0:1789760776.954058:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6789 00000100:00100000:1.0:1789760776.954077:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 756405325310 time 756467983533 state 0x2 ref 0 00000100:00100000:3.0:1789760776.957353:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760776.957454:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512728576 00000100:00100000:0.0:1789760776.957523:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512728576 00000100:00100000:0.0:1789760776.957599:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 756405325310 time 756467983533 state 0x0 ref 1 00000100:00100000:0.0:1789760776.957620:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 756468243949 time 756467983533 state 0x1 ref 1 00000100:00100000:0.0:1789760776.957787:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 756468243949 time 756522084595 state 0x0 ref 1 00000100:00100000:0.0:1789760776.957802:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac28a3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512728576:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760776.957857:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27688960, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0xd807 00010000:00010000:0.0:1789760776.957865:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2fa02 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.957881:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=756s v=(0 0) 00010000:00010000:0.0:1789760776.957885:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760776.957891:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2fa02 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27688960->27693055] (req 27688960->27693055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.957904:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2fa02 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27688960->27693055] (req 27688960->27693055) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760777.010681:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974079 is committed 00002000:00010000:3.0:1789760777.011046:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27688960 -> 27693056 00010000:00010000:3.0:1789760777.011066:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2fa02 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27688960->27693055] (req 27688960->27693055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.011095:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2fa02 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27688960->27693055] (req 27688960->27693055) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.011105:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2fa02 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27688960->27693055] (req 27688960->27693055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.011113:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760777.011120:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2fa02 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27688960->27693055] (req 27688960->27693055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.011131:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2fa02 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27688960->27693055] (req 27688960->27693055) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.011156:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2fa02 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27688960->27693055] (req 27688960->27693055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.011165:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2fa02 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27688960->27693055] (req 27688960->27693055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760777.011860:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac28a3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512728576:12345-192.168.203.56@tcp:4:dd.0 Request processed in 53518us (53978us total) trans 4294974079 rc 0/0 00000100:00100000:3.0:1789760777.011876:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6790 00000100:00100000:3.0:1789760777.011890:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 756468243949 time 756522084595 state 0x2 ref 0 00000100:00100000:1.0:1789760777.027206:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760777.027567:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512728704 00000100:00100000:2.0:1789760777.027582:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512728704 00000100:00100000:2.0:1789760777.027676:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 756468243949 time 756522084595 state 0x0 ref 1 00000100:00100000:2.0:1789760777.027717:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 756522345011 time 756522084595 state 0x1 ref 1 00000100:00100000:3.0:1789760777.027781:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 756522345011 time 756592076907 state 0x0 ref 1 00000100:00100000:3.0:1789760777.027803:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febd02e3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512728704:12345-192.168.203.56@tcp:4:dd.0 00000100:00080000:1.0:1789760777.027839:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00010000:3.0:1789760777.027872:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27693056, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0xd807 00010000:00010000:3.0:1789760777.027884:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2fa09 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.027911:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=756s v=(0 0) 00010000:00010000:3.0:1789760777.027916:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760777.027923:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2fa09 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27693056->27697151] (req 27693056->27697151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.027940:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2fa09 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27693056->27697151] (req 27693056->27697151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760777.090917:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974080 is committed 00002000:00010000:3.0:1789760777.091182:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27693056 -> 27697152 00010000:00010000:3.0:1789760777.091204:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2fa09 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27693056->27697151] (req 27693056->27697151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.091219:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2fa09 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27693056->27697151] (req 27693056->27697151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.091230:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2fa09 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27693056->27697151] (req 27693056->27697151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.091243:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760777.091248:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2fa09 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27693056->27697151] (req 27693056->27697151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.091258:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2fa09 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27693056->27697151] (req 27693056->27697151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.091282:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2fa09 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27693056->27697151] (req 27693056->27697151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.091289:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2fa09 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27693056->27697151] (req 27693056->27697151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760777.091422:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febd02e3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512728704:12345-192.168.203.56@tcp:4:dd.0 Request processed in 63620us (64226us total) trans 4294974080 rc 0/0 00000100:00100000:3.0:1789760777.091436:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6791 00000100:00100000:3.0:1789760777.091448:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 756522345011 time 756592076907 state 0x2 ref 0 00000100:00100000:0.0:1789760777.099934:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760777.100045:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512728832 00000100:00100000:1.0:1789760777.100059:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512728832 00000100:00100000:1.0:1789760777.100147:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 756522345011 time 756592076907 state 0x0 ref 1 00000100:00100000:1.0:1789760777.100166:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 756592337323 time 756592076907 state 0x1 ref 1 00000100:00100000:1.0:1789760777.100202:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 756592337323 time 756664501313 state 0x0 ref 1 00000100:00100000:1.0:1789760777.100216:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2878380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512728832:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760777.100286:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27697152, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0xd807 00010000:00010000:1.0:1789760777.100297:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2fa10 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.100317:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=756s v=(0 0) 00010000:00010000:1.0:1789760777.100322:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760777.100329:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2fa10 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27697152->27701247] (req 27697152->27701247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.100347:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2fa10 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27697152->27701247] (req 27697152->27701247) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760777.139426:0:3312:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1789760777.139455:0:3312:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8feac4130000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3312:1876699540230400:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1789760777.139546:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760777.139627:0:6706:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540230400 00000100:00100000:1.0:1789760777.139636:0:6706:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540230400 00000100:00100000:1.0:1789760777.139718:0:6706:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 210 00000100:00100000:1.0:1789760777.139729:0:6706:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4133480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3312:x1876699540230400:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1789760777.139762:0:6706:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 527892480 pending 1703936 free 3830448128 avail 3815768064 00000100:00100000:1.0:1789760777.139809:0:6706:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4133480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3312:x1876699540230400:12345-0@lo:13:osp-pre-0-0.0 Request processed in 82us (263us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760777.139817:0:6706:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 210 00000100:00080000:1.0:1789760777.139838:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1789760777.139880:0:3312:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8feac4130000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3312:1876699540230400:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1789760777.140961:0:3309:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8febffb9f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3309:1876699540230656:0@lo:400:kworker.0 00000100:00100000:3.0:1789760777.141019:0:3309:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760777.141064:0:3309:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1789760777.141078:0:3309:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8febffb9ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3309:1876699540230784:0@lo:400:kworker.0 00000100:00100000:3.0:1789760777.141104:0:3309:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760777.141139:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540230656 00000100:00100000:3.0:1789760777.141146:0:5847:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540230656 00000100:00100000:3.0:1789760777.141185:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540230784 00000100:00100000:3.0:1789760777.141189:0:5847:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540230784 00000100:00100000:3.0:1789760777.141222:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 506 00000100:00100000:3.0:1789760777.141230:0:5847:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febffb9f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3309:x1876699540230656:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760777.141280:0:5847:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febffb9f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3309:x1876699540230656:12345-0@lo:400:kworker.0 Request processed in 51us (260us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760777.141287:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 506 00000100:00100000:3.0:1789760777.141302:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 507 00000100:00100000:3.0:1789760777.141309:0:5847:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febfb89ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3309:x1876699540230784:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760777.141337:0:5847:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febfb89ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3309:x1876699540230784:12345-0@lo:400:kworker.0 Request processed in 29us (233us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760777.141343:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 507 00000100:00080000:3.0:1789760777.141370:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789760777.141396:0:3309:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8febffb9ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3309:1876699540230784:0@lo:400:kworker.0 00000100:00100000:3.0:1789760777.141406:0:3309:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8febffb9f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3309:1876699540230656:0@lo:400:kworker.0 00000100:00100000:2.0:1789760777.141474:0:3311:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8febffb9c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3311:1876699540230912:0@lo:400:kworker.0 00000100:00100000:2.0:1789760777.141509:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760777.141539:0:3311:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [3->2] 00000100:00100000:2.0:1789760777.141545:0:3311:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8febffb9e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:6df98d54-eb52-4c5c-8fc9-ee9d74f90f98:3311:1876699540230528:0@lo:400:kworker.0 00000100:00100000:2.0:1789760777.141576:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760777.141599:0:3311:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8febffb9c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3311:1876699540231040:0@lo:400:kworker.0 00000100:00100000:2.0:1789760777.141625:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760777.141653:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540230528 00000100:00100000:2.0:1789760777.141657:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540230528 00000100:00100000:2.0:1789760777.141686:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 411 00000100:00100000:2.0:1789760777.141693:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febe00c7850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:6df98d54-eb52-4c5c-8fc9-ee9d74f90f98+19:3311:x1876699540230528:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760777.141770:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febe00c7850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:6df98d54-eb52-4c5c-8fc9-ee9d74f90f98+19:3311:x1876699540230528:12345-0@lo:400:kworker.0 Request processed in 76us (192us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760777.141778:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 411 00000100:00100000:2.0:1789760777.141811:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540231040 00000100:00100000:2.0:1789760777.141814:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540231040 00000100:00100000:2.0:1789760777.141845:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 192 00000100:00100000:2.0:1789760777.141852:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2875f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3311:x1876699540231040:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760777.141916:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2875f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3311:x1876699540231040:12345-0@lo:400:kworker.0 Request processed in 32us (257us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760777.141923:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 192 00000100:00100000:2.0:1789760777.141949:0:3311:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8febffb9e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:6df98d54-eb52-4c5c-8fc9-ee9d74f90f98:3311:1876699540230528:0@lo:400:kworker.0 00000100:00100000:2.0:1789760777.141960:0:3311:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8febffb9c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3311:1876699540231040:0@lo:400:kworker.0 00000100:00080000:3.0:1789760777.142999:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1789760777.143022:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540230912 00000100:00100000:3.0:1789760777.143029:0:5847:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540230912 00000100:00100000:3.0:1789760777.143109:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 508 00000100:00100000:3.0:1789760777.143119:0:5847:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2875500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3311:x1876699540230912:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760777.143201:0:5847:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2875500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3311:x1876699540230912:12345-0@lo:400:kworker.0 Request processed in 84us (1691us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760777.143208:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 508 00000100:00100000:3.0:1789760777.143255:0:3311:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8febffb9c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3311:1876699540230912:0@lo:400:kworker.0 00000100:00080000:2.0:1789760777.144000:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000001:00080000:3.0:1789760777.165342:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974081 is committed 00002000:00010000:1.0:1789760777.165454:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27697152 -> 27701248 00010000:00010000:1.0:1789760777.165478:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2fa10 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27697152->27701247] (req 27697152->27701247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.165493:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2fa10 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27697152->27701247] (req 27697152->27701247) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.165503:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2fa10 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27697152->27701247] (req 27697152->27701247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.165511:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760777.165516:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2fa10 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27697152->27701247] (req 27697152->27701247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.165526:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2fa10 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27697152->27701247] (req 27697152->27701247) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.165549:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2fa10 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27697152->27701247] (req 27697152->27701247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.165559:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2fa10 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27697152->27701247] (req 27697152->27701247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760777.165775:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2878380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512728832:12345-192.168.203.56@tcp:4:dd.0 Request processed in 65553us (65850us total) trans 4294974081 rc 0/0 00000100:00100000:1.0:1789760777.165789:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6792 00000100:00100000:1.0:1789760777.165801:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 756592337323 time 756664501313 state 0x2 ref 0 00000100:00100000:1.0:1789760777.170389:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760777.170573:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512728960 00000100:00100000:3.0:1789760777.170587:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512728960 00000100:00100000:3.0:1789760777.170662:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 756592337323 time 756664501313 state 0x0 ref 1 00000100:00100000:3.0:1789760777.170682:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 756664761729 time 756664501313 state 0x1 ref 1 00000100:00100000:3.0:1789760777.172252:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 756664761729 time 756736537680 state 0x0 ref 1 00000100:00100000:3.0:1789760777.172289:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4130000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512728960:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760777.172376:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27701248, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0xd807 00010000:00010000:3.0:1789760777.172388:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2fa17 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.172409:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=756s v=(0 0) 00010000:00010000:3.0:1789760777.172414:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760777.172423:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2fa17 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27701248->27705343] (req 27701248->27705343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.172440:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2fa17 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27701248->27705343] (req 27701248->27705343) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760777.218683:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27701248 -> 27705344 00010000:00010000:0.0:1789760777.218746:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2fa17 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27701248->27705343] (req 27701248->27705343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.218763:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2fa17 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27701248->27705343] (req 27701248->27705343) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.218773:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2fa17 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27701248->27705343] (req 27701248->27705343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.218788:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760777.218794:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2fa17 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27701248->27705343] (req 27701248->27705343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.218805:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2fa17 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27701248->27705343] (req 27701248->27705343) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.218829:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2fa17 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27701248->27705343] (req 27701248->27705343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.218838:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2fa17 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27701248->27705343] (req 27701248->27705343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760777.218967:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4130000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2361:x1876699512728960:12345-192.168.203.56@tcp:4:dd.0 Request processed in 46686us (48584us total) trans 4294974082 rc 0/0 00000100:00100000:0.0:1789760777.218984:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6793 00000100:00100000:0.0:1789760777.218997:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 756664761729 time 756736537680 state 0x2 ref 0 00000001:00080000:3.0:1789760777.219917:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974082 is committed 00000100:00100000:3.0:1789760777.222983:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760777.223114:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512729088 00000100:00100000:0.0:1789760777.223130:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512729088 00000100:00100000:0.0:1789760777.223205:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 756664761729 time 756736537680 state 0x0 ref 1 00000100:00100000:0.0:1789760777.223224:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 756736798096 time 756736537680 state 0x1 ref 1 00000100:00100000:0.0:1789760777.223263:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 756736798096 time 756787561708 state 0x0 ref 1 00000100:00100000:0.0:1789760777.223283:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febfb89ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512729088:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760777.223351:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27705344, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0xd807 00010000:00010000:0.0:1789760777.223363:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2fa1e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.223385:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=756s v=(0 0) 00010000:00010000:0.0:1789760777.223390:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760777.223399:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2fa1e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27705344->27709439] (req 27705344->27709439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.223416:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2fa1e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27705344->27709439] (req 27705344->27709439) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760777.257075:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974083 is committed 00002000:00010000:3.0:1789760777.257206:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27705344 -> 27709440 00010000:00010000:3.0:1789760777.257228:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2fa1e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27705344->27709439] (req 27705344->27709439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.257243:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2fa1e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27705344->27709439] (req 27705344->27709439) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.257255:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2fa1e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27705344->27709439] (req 27705344->27709439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.257264:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760777.257270:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2fa1e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27705344->27709439] (req 27705344->27709439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.257281:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2fa1e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27705344->27709439] (req 27705344->27709439) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.257307:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2fa1e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27705344->27709439] (req 27705344->27709439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.257316:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2fa1e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27705344->27709439] (req 27705344->27709439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760777.257439:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febfb89ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512729088:12345-192.168.203.56@tcp:4:dd.0 Request processed in 34153us (34936us total) trans 4294974083 rc 0/0 00000100:00100000:3.0:1789760777.257450:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6794 00000100:00100000:3.0:1789760777.257463:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 756736798096 time 756787561708 state 0x2 ref 0 00000100:00100000:3.0:1789760777.260981:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760777.261112:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512729216 00000100:00100000:2.0:1789760777.261127:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512729216 00000100:00100000:2.0:1789760777.261200:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 756736798096 time 756787561708 state 0x0 ref 1 00000100:00100000:2.0:1789760777.261220:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 756787822124 time 756787561708 state 0x1 ref 1 00000100:00100000:2.0:1789760777.261260:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 756787822124 time 756825558787 state 0x0 ref 1 00000100:00100000:2.0:1789760777.261274:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febfb89fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512729216:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760777.261342:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27709440, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0xd807 00010000:00010000:2.0:1789760777.261353:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2fa25 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.261380:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=756s v=(0 0) 00010000:00010000:2.0:1789760777.261385:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760777.261393:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2fa25 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27709440->27713535] (req 27709440->27713535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.261409:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2fa25 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27709440->27713535] (req 27709440->27713535) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760777.293618:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974084 is committed 00002000:00010000:1.0:1789760777.293813:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27709440 -> 27713536 00010000:00010000:1.0:1789760777.293833:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2fa25 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27709440->27713535] (req 27709440->27713535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.293849:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2fa25 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27709440->27713535] (req 27709440->27713535) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.293860:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2fa25 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27709440->27713535] (req 27709440->27713535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.293875:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760777.293881:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2fa25 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27709440->27713535] (req 27709440->27713535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.293892:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2fa25 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27709440->27713535] (req 27709440->27713535) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.293917:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2fa25 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27709440->27713535] (req 27709440->27713535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.293925:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2fa25 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27709440->27713535] (req 27709440->27713535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760777.294092:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febfb89fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512729216:12345-192.168.203.56@tcp:4:dd.0 Request processed in 32814us (33119us total) trans 4294974084 rc 0/0 00000100:00100000:1.0:1789760777.294105:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6795 00000100:00100000:1.0:1789760777.294117:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 756787822124 time 756825558787 state 0x2 ref 0 00000100:00100000:3.0:1789760777.297305:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760777.297388:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512729344 00000100:00100000:1.0:1789760777.297400:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512729344 00000100:00100000:1.0:1789760777.297484:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 756787822124 time 756825558787 state 0x0 ref 1 00000100:00100000:1.0:1789760777.297503:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 756825819203 time 756825558787 state 0x1 ref 1 00000100:00100000:1.0:1789760777.297538:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 756825819203 time 756861836832 state 0x0 ref 1 00000100:00100000:1.0:1789760777.297555:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febffb9c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512729344:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760777.297620:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27713536, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0xd807 00010000:00010000:1.0:1789760777.297630:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2fa2c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.297651:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=756s v=(0 0) 00010000:00010000:1.0:1789760777.297656:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760777.297664:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2fa2c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27713536->27717631] (req 27713536->27717631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.297680:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2fa2c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27713536->27717631] (req 27713536->27717631) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760777.355631:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974085 is committed 00002000:00010000:1.0:1789760777.356399:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27713536 -> 27717632 00010000:00010000:1.0:1789760777.356421:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2fa2c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27713536->27717631] (req 27713536->27717631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.356436:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2fa2c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27713536->27717631] (req 27713536->27717631) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.356446:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2fa2c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27713536->27717631] (req 27713536->27717631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.356453:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760777.356459:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2fa2c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27713536->27717631] (req 27713536->27717631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.356469:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2fa2c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27713536->27717631] (req 27713536->27717631) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.356495:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2fa2c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27713536->27717631] (req 27713536->27717631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.356504:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2fa2c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27713536->27717631] (req 27713536->27717631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760777.356625:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febffb9c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512729344:12345-192.168.203.56@tcp:4:dd.0 Request processed in 59064us (59328us total) trans 4294974085 rc 0/0 00000100:00100000:1.0:1789760777.356637:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6796 00000100:00100000:1.0:1789760777.356650:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 756825819203 time 756861836832 state 0x2 ref 0 00000100:00100000:0.0:1789760777.360779:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760777.360920:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512729472 00000100:00100000:3.0:1789760777.360933:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512729472 00000100:00100000:3.0:1789760777.361013:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 756825819203 time 756861836832 state 0x0 ref 1 00000100:00100000:3.0:1789760777.361033:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 756862097248 time 756861836832 state 0x1 ref 1 00000100:00100000:3.0:1789760777.361080:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 756862097248 time 756925379024 state 0x0 ref 1 00000100:00100000:3.0:1789760777.361096:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac287a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512729472:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760777.361159:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27717632, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0xd807 00010000:00010000:3.0:1789760777.361170:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2fa33 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.361191:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=756s v=(0 0) 00010000:00010000:3.0:1789760777.361196:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760777.361204:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2fa33 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27717632->27721727] (req 27717632->27721727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.361220:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2fa33 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27717632->27721727] (req 27717632->27721727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760777.394086:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974086 is committed 00002000:00010000:1.0:1789760777.394180:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27717632 -> 27721728 00010000:00010000:1.0:1789760777.394200:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2fa33 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27717632->27721727] (req 27717632->27721727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.394216:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2fa33 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27717632->27721727] (req 27717632->27721727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.394226:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2fa33 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27717632->27721727] (req 27717632->27721727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.394235:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760777.394242:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2fa33 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27717632->27721727] (req 27717632->27721727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.394253:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2fa33 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27717632->27721727] (req 27717632->27721727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.394276:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2fa33 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27717632->27721727] (req 27717632->27721727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.394282:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2fa33 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27717632->27721727] (req 27717632->27721727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760777.394601:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac287a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512729472:12345-192.168.203.56@tcp:4:dd.0 Request processed in 33500us (33832us total) trans 4294974086 rc 0/0 00000100:00100000:1.0:1789760777.394613:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6797 00000100:00100000:1.0:1789760777.394625:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 756862097248 time 756925379024 state 0x2 ref 0 00000100:00100000:3.0:1789760777.397985:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760777.398062:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512729600 00000100:00100000:1.0:1789760777.398084:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512729600 00000100:00100000:1.0:1789760777.398153:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 756862097248 time 756925379024 state 0x0 ref 1 00000100:00100000:1.0:1789760777.398175:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 756925639440 time 756925379024 state 0x1 ref 1 00000100:00100000:1.0:1789760777.398212:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 756925639440 time 756962511798 state 0x0 ref 1 00000100:00100000:1.0:1789760777.398226:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febffb9c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512729600:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760777.398294:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27721728, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0xd807 00010000:00010000:1.0:1789760777.398306:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2fa3a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.398326:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=756s v=(0 0) 00010000:00010000:1.0:1789760777.398331:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760777.398339:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2fa3a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27721728->27725823] (req 27721728->27725823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.398356:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2fa3a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27721728->27725823] (req 27721728->27725823) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760777.435465:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27721728 -> 27725824 00010000:00010000:0.0:1789760777.435495:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2fa3a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27721728->27725823] (req 27721728->27725823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.435511:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2fa3a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27721728->27725823] (req 27721728->27725823) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.435522:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2fa3a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27721728->27725823] (req 27721728->27725823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.435530:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760777.435538:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2fa3a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27721728->27725823] (req 27721728->27725823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.435547:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2fa3a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27721728->27725823] (req 27721728->27725823) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.435570:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2fa3a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27721728->27725823] (req 27721728->27725823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.435578:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c2fa3a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27721728->27725823] (req 27721728->27725823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760777.435768:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febffb9c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2358:x1876699512729600:12345-192.168.203.56@tcp:4:dd.0 Request processed in 37535us (37789us total) trans 4294974087 rc 0/0 00000100:00100000:0.0:1789760777.435781:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6798 00000100:00100000:0.0:1789760777.435794:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 756925639440 time 756962511798 state 0x2 ref 0 00000001:00080000:3.0:1789760777.436037:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974087 is committed 00000100:00100000:3.0:1789760777.439624:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760777.439799:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512729728 00000100:00100000:1.0:1789760777.439813:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512729728 00000100:00100000:1.0:1789760777.439887:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 756925639440 time 756962511798 state 0x0 ref 1 00000100:00100000:1.0:1789760777.439905:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 756962772214 time 756962511798 state 0x1 ref 1 00000100:00100000:1.0:1789760777.439936:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 756962772214 time 757004235008 state 0x0 ref 1 00000100:00100000:1.0:1789760777.439949:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febffb9e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512729728:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760777.440013:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27725824, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0xd807 00010000:00010000:1.0:1789760777.440023:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2fa41 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.440043:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=757s v=(0 0) 00010000:00010000:1.0:1789760777.440049:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760777.440056:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2fa41 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27725824->27729919] (req 27725824->27729919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.440085:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2fa41 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27725824->27729919] (req 27725824->27729919) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760777.479834:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974088 is committed 00002000:00010000:1.0:1789760777.480377:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27725824 -> 27729920 00010000:00010000:1.0:1789760777.480399:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2fa41 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27725824->27729919] (req 27725824->27729919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.480413:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2fa41 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27725824->27729919] (req 27725824->27729919) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.480424:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2fa41 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27725824->27729919] (req 27725824->27729919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.480433:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760777.480439:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2fa41 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27725824->27729919] (req 27725824->27729919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.480450:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2fa41 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27725824->27729919] (req 27725824->27729919) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.480487:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2fa41 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27725824->27729919] (req 27725824->27729919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.480495:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2fa41 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27725824->27729919] (req 27725824->27729919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760777.480624:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febffb9e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512729728:12345-192.168.203.56@tcp:4:dd.0 Request processed in 40670us (41007us total) trans 4294974088 rc 0/0 00000100:00100000:1.0:1789760777.480636:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6799 00000100:00100000:1.0:1789760777.480648:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 756962772214 time 757004235008 state 0x2 ref 0 00000100:00100000:1.0:1789760777.484223:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760777.484332:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512729856 00000100:00100000:3.0:1789760777.484346:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512729856 00000100:00100000:3.0:1789760777.484416:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 756962772214 time 757004235008 state 0x0 ref 1 00000100:00100000:3.0:1789760777.484436:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 757004495424 time 757004235008 state 0x1 ref 1 00000100:00100000:3.0:1789760777.484472:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 757004495424 time 757048771312 state 0x0 ref 1 00000100:00100000:3.0:1789760777.484486:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4130700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512729856:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760777.484554:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27729920, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0xd807 00010000:00010000:3.0:1789760777.484565:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2fa48 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.484585:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=757s v=(0 0) 00010000:00010000:3.0:1789760777.484590:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760777.484597:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2fa48 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27729920->27734015] (req 27729920->27734015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.484614:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2fa48 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27729920->27734015] (req 27729920->27734015) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760777.520215:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974089 is committed 00002000:00010000:1.0:1789760777.520329:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27729920 -> 27734016 00010000:00010000:1.0:1789760777.520352:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2fa48 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27729920->27734015] (req 27729920->27734015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.520368:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2fa48 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27729920->27734015] (req 27729920->27734015) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.520379:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2fa48 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27729920->27734015] (req 27729920->27734015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.520388:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760777.520394:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2fa48 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27729920->27734015] (req 27729920->27734015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.520404:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2fa48 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27729920->27734015] (req 27729920->27734015) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.520432:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2fa48 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27729920->27734015] (req 27729920->27734015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.520447:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2fa48 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27729920->27734015] (req 27729920->27734015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760777.520635:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4130700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512729856:12345-192.168.203.56@tcp:4:dd.0 Request processed in 36145us (36421us total) trans 4294974089 rc 0/0 00000100:00100000:1.0:1789760777.520645:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6800 00000100:00100000:1.0:1789760777.520656:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 757004495424 time 757048771312 state 0x2 ref 0 00000100:00100000:3.0:1789760777.525904:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760777.526054:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512729984 00000100:00100000:0.0:1789760777.526064:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512729984 00000100:00100000:0.0:1789760777.526123:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 757004495424 time 757048771312 state 0x0 ref 1 00000100:00100000:0.0:1789760777.526209:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 757049031728 time 757048771312 state 0x1 ref 1 00000100:00100000:0.0:1789760777.526242:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 757049031728 time 757090541594 state 0x0 ref 1 00000100:00100000:0.0:1789760777.526254:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febffb9f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512729984:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760777.526303:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27734016, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0xd807 00010000:00010000:0.0:1789760777.526312:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2fa4f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.526328:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=757s v=(0 0) 00010000:00010000:0.0:1789760777.526332:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760777.526339:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2fa4f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27734016->27738111] (req 27734016->27738111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.526354:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2fa4f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27734016->27738111] (req 27734016->27738111) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760777.567581:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974090 is committed 00002000:00010000:3.0:1789760777.568058:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27734016 -> 27738112 00002000:00010000:3.0:1789760777.568073:0:13063:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760776 -> 1789760777 00002000:00010000:3.0:1789760777.568076:0:13063:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760776 -> 1789760777 00010000:00010000:3.0:1789760777.568094:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2fa4f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27734016->27738111] (req 27734016->27738111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.568110:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2fa4f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27734016->27738111] (req 27734016->27738111) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.568122:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2fa4f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27734016->27738111] (req 27734016->27738111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.568131:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760777.568137:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2fa4f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27734016->27738111] (req 27734016->27738111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.568148:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2fa4f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27734016->27738111] (req 27734016->27738111) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.568172:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2fa4f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27734016->27738111] (req 27734016->27738111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.568180:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2fa4f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27734016->27738111] (req 27734016->27738111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760777.568308:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febffb9f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512729984:12345-192.168.203.56@tcp:4:dd.0 Request processed in 42048us (42405us total) trans 4294974090 rc 0/0 00000100:00100000:3.0:1789760777.568320:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6801 00000100:00100000:3.0:1789760777.568334:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 757049031728 time 757090541594 state 0x2 ref 0 00000100:00100000:3.0:1789760777.571004:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760777.571178:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512730112 00000100:00100000:1.0:1789760777.571191:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512730112 00000100:00100000:1.0:1789760777.571265:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 757049031728 time 757090541594 state 0x0 ref 1 00000100:00100000:1.0:1789760777.571284:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 757090802010 time 757090541594 state 0x1 ref 1 00000100:00100000:1.0:1789760777.571324:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 757090802010 time 757135623104 state 0x0 ref 1 00000100:00100000:1.0:1789760777.571338:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febffb9ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512730112:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760777.571403:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27738112, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0xd807 00010000:00010000:1.0:1789760777.571413:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2fa56 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.571432:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=757s v=(0 0) 00010000:00010000:1.0:1789760777.571436:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760777.571444:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2fa56 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27738112->27742207] (req 27738112->27742207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.571460:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2fa56 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27738112->27742207] (req 27738112->27742207) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.586917:0:7647:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:1.0:1789760777.632561:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27738112 -> 27742208 00010000:00010000:1.0:1789760777.632587:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2fa56 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27738112->27742207] (req 27738112->27742207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.632600:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2fa56 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27738112->27742207] (req 27738112->27742207) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.632609:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2fa56 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27738112->27742207] (req 27738112->27742207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.632616:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760777.632621:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2fa56 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27738112->27742207] (req 27738112->27742207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.632630:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2fa56 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27738112->27742207] (req 27738112->27742207) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.632651:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2fa56 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27738112->27742207] (req 27738112->27742207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.632659:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2fa56 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27738112->27742207] (req 27738112->27742207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760777.633104:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febffb9ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2359:x1876699512730112:12345-192.168.203.56@tcp:4:dd.0 Request processed in 61762us (62121us total) trans 4294974091 rc 0/0 00000100:00100000:1.0:1789760777.633114:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6802 00000100:00100000:1.0:1789760777.633124:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 757090802010 time 757135623104 state 0x2 ref 0 00000001:00080000:3.0:1789760777.633775:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974091 is committed 00000100:00100000:1.0:1789760777.636444:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760777.636551:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512730240 00000100:00100000:3.0:1789760777.636560:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512730240 00000100:00100000:3.0:1789760777.636620:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 757090802010 time 757135623104 state 0x0 ref 1 00000100:00100000:3.0:1789760777.636636:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 757135883520 time 757135623104 state 0x1 ref 1 00000100:00100000:3.0:1789760777.636801:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 757135883520 time 757201038730 state 0x0 ref 1 00000100:00100000:3.0:1789760777.636818:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4131180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512730240:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760777.636869:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27742208, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0xd807 00010000:00010000:3.0:1789760777.636879:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2fa5d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.636894:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=757s v=(0 0) 00010000:00010000:3.0:1789760777.636899:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760777.636905:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2fa5d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27742208->27746303] (req 27742208->27746303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.636918:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2fa5d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27742208->27746303] (req 27742208->27746303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760777.652414:0:3310:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1789760777.652434:0:3310:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8febea772680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3310:1876699540231168:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789760777.652508:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760777.653502:0:6706:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540231168 00000100:00100000:1.0:1789760777.653513:0:6706:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540231168 00000100:00100000:1.0:1789760777.653560:0:6706:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 211 00000100:00100000:1.0:1789760777.653570:0:6706:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac403b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3310:x1876699540231168:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1789760777.653606:0:6706:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3859808256 avail 3857711104 00000100:00100000:1.0:1789760777.653661:0:6706:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac403b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3310:x1876699540231168:12345-0@lo:13:osp-pre-1-0.0 Request processed in 93us (1153us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760777.653668:0:6706:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 211 00000100:00100000:1.0:1789760777.653731:0:3310:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8febea772680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3310:1876699540231168:0@lo:13:osp-pre-1-0.0 00000001:00080000:3.0:1789760777.665394:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974092 is committed 00002000:00010000:1.0:1789760777.665553:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27742208 -> 27746304 00010000:00010000:1.0:1789760777.665568:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2fa5d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27742208->27746303] (req 27742208->27746303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.665735:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2fa5d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27742208->27746303] (req 27742208->27746303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.665746:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2fa5d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27742208->27746303] (req 27742208->27746303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.665758:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760777.665763:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2fa5d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27742208->27746303] (req 27742208->27746303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.665773:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2fa5d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27742208->27746303] (req 27742208->27746303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.665792:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2fa5d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27742208->27746303] (req 27742208->27746303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.665800:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2fa5d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27742208->27746303] (req 27742208->27746303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760777.665932:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4131180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512730240:12345-192.168.203.56@tcp:4:dd.0 Request processed in 29114us (29491us total) trans 4294974092 rc 0/0 00000100:00100000:1.0:1789760777.665943:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6803 00000100:00100000:1.0:1789760777.665952:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 757135883520 time 757201038730 state 0x2 ref 0 00000100:00100000:1.0:1789760777.670804:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760777.670910:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512730368 00000100:00100000:3.0:1789760777.670920:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512730368 00000100:00100000:3.0:1789760777.670978:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 757135883520 time 757201038730 state 0x0 ref 1 00000100:00100000:3.0:1789760777.670993:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 757201299146 time 757201038730 state 0x1 ref 1 00000100:00100000:3.0:1789760777.671028:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 757201299146 time 757235327128 state 0x0 ref 1 00000100:00100000:3.0:1789760777.671115:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4132300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512730368:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760777.671161:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27746304, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0xd807 00010000:00010000:3.0:1789760777.671171:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2fa64 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.671188:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=757s v=(0 0) 00010000:00010000:3.0:1789760777.671193:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760777.671199:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2fa64 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27746304->27750399] (req 27746304->27750399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.671212:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2fa64 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27746304->27750399] (req 27746304->27750399) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760777.702407:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974093 is committed 00002000:00010000:3.0:1789760777.702545:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27746304 -> 27750400 00010000:00010000:3.0:1789760777.702560:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2fa64 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27746304->27750399] (req 27746304->27750399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.702573:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2fa64 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27746304->27750399] (req 27746304->27750399) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.702582:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2fa64 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27746304->27750399] (req 27746304->27750399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.702589:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760777.702594:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2fa64 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27746304->27750399] (req 27746304->27750399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.702603:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2fa64 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27746304->27750399] (req 27746304->27750399) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.702623:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2fa64 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27746304->27750399] (req 27746304->27750399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.702630:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2fa64 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27746304->27750399] (req 27746304->27750399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760777.702763:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4132300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512730368:12345-192.168.203.56@tcp:4:dd.0 Request processed in 31646us (31963us total) trans 4294974093 rc 0/0 00000100:00100000:3.0:1789760777.702772:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6804 00000100:00100000:3.0:1789760777.702782:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 757201299146 time 757235327128 state 0x2 ref 0 00000100:00100000:3.0:1789760777.709805:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760777.709998:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512730496 00000100:00100000:1.0:1789760777.710008:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512730496 00000100:00100000:1.0:1789760777.710080:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 757201299146 time 757235327128 state 0x0 ref 1 00000100:00100000:1.0:1789760777.710096:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 757235587544 time 757235327128 state 0x1 ref 1 00000100:00100000:1.0:1789760777.710127:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 757235587544 time 757274427066 state 0x0 ref 1 00000100:00100000:1.0:1789760777.710140:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febfb8d4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512730496:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760777.710189:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27750400, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0xd807 00010000:00010000:1.0:1789760777.710198:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2fa6b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.710213:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=757s v=(0 0) 00010000:00010000:1.0:1789760777.710217:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760777.710223:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2fa6b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27750400->27754495] (req 27750400->27754495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.710237:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2fa6b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27750400->27754495] (req 27750400->27754495) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760777.748587:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27750400 -> 27754496 00010000:00010000:2.0:1789760777.748607:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2fa6b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27750400->27754495] (req 27750400->27754495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.748620:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2fa6b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27750400->27754495] (req 27750400->27754495) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.748629:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2fa6b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27750400->27754495] (req 27750400->27754495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.748636:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760777.748641:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2fa6b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27750400->27754495] (req 27750400->27754495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.748654:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2fa6b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27750400->27754495] (req 27750400->27754495) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.748674:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2fa6b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27750400->27754495] (req 27750400->27754495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.748681:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2fa6b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27750400->27754495] (req 27750400->27754495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760777.748900:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febfb8d4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2358:x1876699512730496:12345-192.168.203.56@tcp:4:dd.0 Request processed in 38756us (39098us total) trans 4294974094 rc 0/0 00000100:00100000:2.0:1789760777.748911:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6805 00000100:00100000:2.0:1789760777.748921:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 757235587544 time 757274427066 state 0x2 ref 0 00000001:00080000:3.0:1789760777.750471:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974094 is committed 00000100:00100000:0.0:1789760777.752513:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760777.752649:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512730624 00000100:00100000:1.0:1789760777.752659:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512730624 00000100:00100000:1.0:1789760777.752754:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 757235587544 time 757274427066 state 0x0 ref 1 00000100:00100000:1.0:1789760777.752771:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 757274687482 time 757274427066 state 0x1 ref 1 00000100:00100000:1.0:1789760777.752791:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 757274687482 time 757317091274 state 0x0 ref 1 00000100:00100000:1.0:1789760777.752803:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2879c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512730624:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760777.752848:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27754496, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0xd807 00010000:00010000:1.0:1789760777.752857:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2fa72 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.752877:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=757s v=(0 0) 00010000:00010000:1.0:1789760777.752881:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760777.752888:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2fa72 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27754496->27758591] (req 27754496->27758591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.752902:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2fa72 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27754496->27758591] (req 27754496->27758591) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760777.789880:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27754496 -> 27758592 00010000:00010000:2.0:1789760777.789900:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2fa72 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27754496->27758591] (req 27754496->27758591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.789913:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2fa72 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27754496->27758591] (req 27754496->27758591) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.789921:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2fa72 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27754496->27758591] (req 27754496->27758591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.789928:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760777.789934:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2fa72 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27754496->27758591] (req 27754496->27758591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.789944:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2fa72 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27754496->27758591] (req 27754496->27758591) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.789963:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2fa72 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27754496->27758591] (req 27754496->27758591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.789970:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c2fa72 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27754496->27758591] (req 27754496->27758591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760777.790082:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2879c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2359:x1876699512730624:12345-192.168.203.56@tcp:4:dd.0 Request processed in 37275us (37572us total) trans 4294974095 rc 0/0 00000100:00100000:2.0:1789760777.790093:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6806 00000100:00100000:2.0:1789760777.790101:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 757274687482 time 757317091274 state 0x2 ref 0 00000001:00080000:0.0:1789760777.791812:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974095 is committed 00000100:00100000:0.0:1789760777.796313:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760777.796401:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512730752 00000100:00100000:3.0:1789760777.796410:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512730752 00000100:00100000:3.0:1789760777.796465:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 757274687482 time 757317091274 state 0x0 ref 1 00000100:00100000:3.0:1789760777.796480:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 757317351690 time 757317091274 state 0x1 ref 1 00000100:00100000:3.0:1789760777.796509:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 757317351690 time 757360808270 state 0x0 ref 1 00000100:00100000:3.0:1789760777.796522:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2878e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512730752:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760777.796571:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27758592, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0xd807 00010000:00010000:3.0:1789760777.796580:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2fa79 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.796596:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=757s v=(0 0) 00010000:00010000:3.0:1789760777.796600:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760777.796607:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2fa79 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27758592->27762687] (req 27758592->27762687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.796621:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2fa79 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27758592->27762687] (req 27758592->27762687) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760777.836153:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974096 is committed 00002000:00010000:2.0:1789760777.836236:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27758592 -> 27762688 00010000:00010000:2.0:1789760777.836252:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2fa79 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27758592->27762687] (req 27758592->27762687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.836265:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2fa79 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27758592->27762687] (req 27758592->27762687) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.836275:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2fa79 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27758592->27762687] (req 27758592->27762687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.836288:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760777.836293:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2fa79 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27758592->27762687] (req 27758592->27762687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.836303:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2fa79 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27758592->27762687] (req 27758592->27762687) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.836322:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2fa79 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27758592->27762687] (req 27758592->27762687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.836329:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2fa79 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27758592->27762687] (req 27758592->27762687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760777.836499:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2878e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512730752:12345-192.168.203.56@tcp:4:dd.0 Request processed in 39973us (40189us total) trans 4294974096 rc 0/0 00000100:00100000:2.0:1789760777.836510:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6807 00000100:00100000:2.0:1789760777.836520:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 757317351690 time 757360808270 state 0x2 ref 0 00000100:00100000:3.0:1789760777.842411:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760777.842682:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512730880 00000100:00100000:2.0:1789760777.842692:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512730880 00000100:00100000:2.0:1789760777.842835:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 757317351690 time 757360808270 state 0x0 ref 1 00000100:00100000:2.0:1789760777.842850:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 757361068686 time 757360808270 state 0x1 ref 1 00000100:00100000:2.0:1789760777.842869:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 757361068686 time 757407169749 state 0x0 ref 1 00000100:00100000:2.0:1789760777.842883:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febc7b33480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512730880:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760777.842931:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27762688, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0xd807 00010000:00010000:2.0:1789760777.842939:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2fa80 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.842956:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=757s v=(0 0) 00010000:00010000:2.0:1789760777.842959:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760777.842965:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2fa80 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27762688->27766783] (req 27762688->27766783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.842979:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2fa80 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27762688->27766783] (req 27762688->27766783) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760777.894625:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974097 is committed 00002000:00010000:2.0:1789760777.894964:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27762688 -> 27766784 00010000:00010000:2.0:1789760777.894981:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2fa80 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27762688->27766783] (req 27762688->27766783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.894994:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2fa80 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27762688->27766783] (req 27762688->27766783) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.895004:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2fa80 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27762688->27766783] (req 27762688->27766783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.895011:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760777.895016:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2fa80 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27762688->27766783] (req 27762688->27766783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.895026:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2fa80 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27762688->27766783] (req 27762688->27766783) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.895046:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2fa80 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27762688->27766783] (req 27762688->27766783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.895226:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2fa80 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27762688->27766783] (req 27762688->27766783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760777.895321:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febc7b33480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512730880:12345-192.168.203.56@tcp:4:dd.0 Request processed in 52434us (52919us total) trans 4294974097 rc 0/0 00000100:00100000:2.0:1789760777.895331:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6808 00000100:00100000:2.0:1789760777.895340:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 757361068686 time 757407169749 state 0x2 ref 0 00000100:00100000:0.0:1789760777.903294:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760777.903453:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512731008 00000100:00100000:2.0:1789760777.903464:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512731008 00000100:00100000:2.0:1789760777.903518:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 757361068686 time 757407169749 state 0x0 ref 1 00000100:00100000:2.0:1789760777.903533:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 757407430165 time 757407169749 state 0x1 ref 1 00000100:00100000:2.0:1789760777.903568:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 757407430165 time 757467867267 state 0x0 ref 1 00000100:00100000:2.0:1789760777.903581:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febea488c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512731008:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760777.903640:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27766784, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0xd807 00010000:00010000:2.0:1789760777.903655:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2fa87 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.903672:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=757s v=(0 0) 00010000:00010000:2.0:1789760777.903677:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760777.903683:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2fa87 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27766784->27770879] (req 27766784->27770879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.903728:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2fa87 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27766784->27770879] (req 27766784->27770879) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760777.933430:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974098 is committed 00002000:00010000:1.0:1789760777.933559:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27766784 -> 27770880 00010000:00010000:1.0:1789760777.933574:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2fa87 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27766784->27770879] (req 27766784->27770879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.933587:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2fa87 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27766784->27770879] (req 27766784->27770879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.933597:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2fa87 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27766784->27770879] (req 27766784->27770879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.933604:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760777.933609:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2fa87 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27766784->27770879] (req 27766784->27770879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.933620:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2fa87 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27766784->27770879] (req 27766784->27770879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.933639:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2fa87 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27766784->27770879] (req 27766784->27770879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.933648:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2fa87 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27766784->27770879] (req 27766784->27770879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760777.933847:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febea488c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512731008:12345-192.168.203.56@tcp:4:dd.0 Request processed in 30263us (30557us total) trans 4294974098 rc 0/0 00000100:00100000:1.0:1789760777.933858:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6809 00000100:00100000:1.0:1789760777.933867:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 757407430165 time 757467867267 state 0x2 ref 0 00000100:00100000:0.0:1789760777.937745:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760777.937907:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512731136 00000100:00100000:2.0:1789760777.937917:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512731136 00000100:00100000:2.0:1789760777.937967:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 757407430165 time 757467867267 state 0x0 ref 1 00000100:00100000:2.0:1789760777.937982:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 757468127683 time 757467867267 state 0x1 ref 1 00000100:00100000:2.0:1789760777.938003:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 757468127683 time 757502302931 state 0x0 ref 1 00000100:00100000:2.0:1789760777.938015:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac287b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512731136:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760777.938082:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27770880, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0xd807 00010000:00010000:2.0:1789760777.938092:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2fa8e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.938106:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=757s v=(0 0) 00010000:00010000:2.0:1789760777.938111:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760777.938117:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2fa8e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27770880->27774975] (req 27770880->27774975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.938135:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2fa8e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27770880->27774975] (req 27770880->27774975) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760777.966334:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974099 is committed 00002000:00010000:0.0:1789760777.966399:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27770880 -> 27774976 00010000:00010000:0.0:1789760777.966414:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2fa8e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27770880->27774975] (req 27770880->27774975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.966427:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2fa8e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27770880->27774975] (req 27770880->27774975) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.966437:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2fa8e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27770880->27774975] (req 27770880->27774975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.966445:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760777.966451:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2fa8e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27770880->27774975] (req 27770880->27774975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.966460:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2fa8e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27770880->27774975] (req 27770880->27774975) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.966480:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2fa8e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27770880->27774975] (req 27770880->27774975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.966488:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2fa8e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27770880->27774975] (req 27770880->27774975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760777.966617:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac287b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512731136:12345-192.168.203.56@tcp:4:dd.0 Request processed in 28599us (28927us total) trans 4294974099 rc 0/0 00000100:00100000:0.0:1789760777.966628:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6810 00000100:00100000:0.0:1789760777.966638:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 757468127683 time 757502302931 state 0x2 ref 0 00000100:00100000:0.0:1789760777.970121:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760777.970207:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512731264 00000100:00100000:2.0:1789760777.970216:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512731264 00000100:00100000:2.0:1789760777.970268:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 757468127683 time 757502302931 state 0x0 ref 1 00000100:00100000:2.0:1789760777.970283:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 757502563347 time 757502302931 state 0x1 ref 1 00000100:00100000:2.0:1789760777.970312:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 757502563347 time 757534612140 state 0x0 ref 1 00000100:00100000:2.0:1789760777.970325:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2878000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512731264:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760777.970374:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27774976, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0xd807 00010000:00010000:2.0:1789760777.970384:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2fa95 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.970400:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=757s v=(0 0) 00010000:00010000:2.0:1789760777.970405:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760777.970411:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2fa95 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27774976->27779071] (req 27774976->27779071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.970426:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2fa95 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27774976->27779071] (req 27774976->27779071) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760778.003762:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974100 is committed 00002000:00010000:0.0:1789760778.003775:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27774976 -> 27779072 00010000:00010000:0.0:1789760778.003790:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2fa95 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27774976->27779071] (req 27774976->27779071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.003804:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2fa95 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27774976->27779071] (req 27774976->27779071) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.003819:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2fa95 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27774976->27779071] (req 27774976->27779071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.003826:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760778.003831:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2fa95 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27774976->27779071] (req 27774976->27779071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.003839:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2fa95 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27774976->27779071] (req 27774976->27779071) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.003857:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2fa95 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27774976->27779071] (req 27774976->27779071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.003865:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c2fa95 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27774976->27779071] (req 27774976->27779071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760778.003953:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2878000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512731264:12345-192.168.203.56@tcp:4:dd.0 Request processed in 33625us (33837us total) trans 4294974100 rc 0/0 00000100:00100000:0.0:1789760778.003964:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6811 00000100:00100000:0.0:1789760778.003974:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 757502563347 time 757534612140 state 0x2 ref 0 00000100:00100000:2.0:1789760778.015049:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760778.015494:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512731392 00000100:00100000:1.0:1789760778.015504:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512731392 00000100:00100000:1.0:1789760778.015561:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 757502563347 time 757534612140 state 0x0 ref 1 00000100:00100000:1.0:1789760778.015576:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 757534872556 time 757534612140 state 0x1 ref 1 00000100:00100000:1.0:1789760778.015609:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 757534872556 time 757579908814 state 0x0 ref 1 00000100:00100000:1.0:1789760778.015622:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac403b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512731392:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760778.015672:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27779072, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0xd807 00010000:00010000:1.0:1789760778.015681:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2fa9c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.015722:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=757s v=(0 0) 00010000:00010000:1.0:1789760778.015727:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760778.015734:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2fa9c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27779072->27783167] (req 27779072->27783167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.015750:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2fa9c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27779072->27783167] (req 27779072->27783167) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760778.052181:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974101 is committed 00002000:00010000:2.0:1789760778.052396:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27779072 -> 27783168 00010000:00010000:2.0:1789760778.052412:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2fa9c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27779072->27783167] (req 27779072->27783167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.052425:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2fa9c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27779072->27783167] (req 27779072->27783167) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.052433:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2fa9c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27779072->27783167] (req 27779072->27783167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.052440:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760778.052445:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2fa9c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27779072->27783167] (req 27779072->27783167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.052454:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2fa9c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27779072->27783167] (req 27779072->27783167) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.052472:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2fa9c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27779072->27783167] (req 27779072->27783167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.052481:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2fa9c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27779072->27783167] (req 27779072->27783167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760778.052608:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac403b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512731392:12345-192.168.203.56@tcp:4:dd.0 Request processed in 36982us (37562us total) trans 4294974101 rc 0/0 00000100:00100000:2.0:1789760778.052619:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6812 00000100:00100000:2.0:1789760778.052628:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 757534872556 time 757579908814 state 0x2 ref 0 00000100:00100000:1.0:1789760778.061037:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760778.062723:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512731520 00000100:00100000:0.0:1789760778.062830:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512731520 00000100:00100000:0.0:1789760778.062895:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 757534872556 time 757579908814 state 0x0 ref 1 00000100:00100000:0.0:1789760778.062910:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 757580169230 time 757579908814 state 0x1 ref 1 00000100:00100000:0.0:1789760778.063014:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 757580169230 time 757627313591 state 0x0 ref 1 00000100:00100000:0.0:1789760778.063028:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4130000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512731520:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760778.063087:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27783168, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0xd807 00010000:00010000:0.0:1789760778.063096:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2faa3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.063113:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=757s v=(0 0) 00010000:00010000:0.0:1789760778.063117:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760778.063123:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2faa3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27783168->27787263] (req 27783168->27787263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.063137:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2faa3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27783168->27787263] (req 27783168->27787263) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760778.107070:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974102 is committed 00002000:00010000:0.0:1789760778.107139:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27783168 -> 27787264 00010000:00010000:0.0:1789760778.107155:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2faa3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27783168->27787263] (req 27783168->27787263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.107168:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2faa3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27783168->27787263] (req 27783168->27787263) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.107177:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2faa3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27783168->27787263] (req 27783168->27787263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.107191:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760778.107196:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2faa3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27783168->27787263] (req 27783168->27787263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.107206:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2faa3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27783168->27787263] (req 27783168->27787263) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.107227:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2faa3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27783168->27787263] (req 27783168->27787263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.107235:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2faa3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27783168->27787263] (req 27783168->27787263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760778.107325:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4130000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512731520:12345-192.168.203.56@tcp:4:dd.0 Request processed in 44296us (46293us total) trans 4294974102 rc 0/0 00000100:00100000:0.0:1789760778.107336:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6813 00000100:00100000:0.0:1789760778.107346:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 757580169230 time 757627313591 state 0x2 ref 0 00000100:00100000:0.0:1789760778.110813:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760778.113637:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512731648 00000100:00100000:2.0:1789760778.113840:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512731648 00000100:00100000:2.0:1789760778.113900:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 757580169230 time 757627313591 state 0x0 ref 1 00000100:00100000:2.0:1789760778.113915:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 757627574007 time 757627313591 state 0x1 ref 1 00000100:00100000:2.0:1789760778.113974:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 757627574007 time 757678272936 state 0x0 ref 1 00000100:00100000:2.0:1789760778.113987:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2879180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512731648:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760778.114037:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27787264, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0xd807 00010000:00010000:2.0:1789760778.114045:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2faaa lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.114069:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=757s v=(0 0) 00010000:00010000:2.0:1789760778.114074:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760778.114081:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2faaa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27787264->27791359] (req 27787264->27791359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.114096:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2faaa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27787264->27791359] (req 27787264->27791359) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760778.162159:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974103 is committed 00002000:00010000:1.0:1789760778.162319:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27787264 -> 27791360 00010000:00010000:1.0:1789760778.162336:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2faaa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27787264->27791359] (req 27787264->27791359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.162349:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2faaa lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27787264->27791359] (req 27787264->27791359) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.162359:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2faaa lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27787264->27791359] (req 27787264->27791359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.162367:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760778.162372:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2faaa lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27787264->27791359] (req 27787264->27791359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.162383:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2faaa lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27787264->27791359] (req 27787264->27791359) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.162406:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2faaa lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27787264->27791359] (req 27787264->27791359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.162415:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2faaa lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27787264->27791359] (req 27787264->27791359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760778.162609:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2879180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512731648:12345-192.168.203.56@tcp:4:dd.0 Request processed in 48618us (51798us total) trans 4294974103 rc 0/0 00000100:00100000:1.0:1789760778.162689:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6814 00000100:00100000:1.0:1789760778.162894:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 757627574007 time 757678272936 state 0x2 ref 0 00000100:00100000:0.0:1789760778.168575:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760778.169552:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512731776 00000100:00100000:2.0:1789760778.169565:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512731776 00000100:00100000:2.0:1789760778.169866:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 757627574007 time 757678272936 state 0x0 ref 1 00000100:00100000:2.0:1789760778.169884:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 757678533352 time 757678272936 state 0x1 ref 1 00000100:00100000:2.0:1789760778.169917:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 757678533352 time 757734217052 state 0x0 ref 1 00000100:00100000:2.0:1789760778.169930:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2879500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512731776:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760778.169999:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27791360, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0xd807 00010000:00010000:2.0:1789760778.170008:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2fab1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.170028:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=757s v=(0 0) 00010000:00010000:2.0:1789760778.170033:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760778.170043:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2fab1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27791360->27795455] (req 27791360->27795455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.170070:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2fab1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27791360->27795455] (req 27791360->27795455) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760778.219578:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27791360 -> 27795456 00010000:00010000:1.0:1789760778.219598:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2fab1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27791360->27795455] (req 27791360->27795455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.219610:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2fab1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27791360->27795455] (req 27791360->27795455) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.219618:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2fab1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27791360->27795455] (req 27791360->27795455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.219625:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760778.219629:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2fab1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27791360->27795455] (req 27791360->27795455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.219638:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2fab1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27791360->27795455] (req 27791360->27795455) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.219656:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2fab1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27791360->27795455] (req 27791360->27795455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.219668:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2fab1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27791360->27795455] (req 27791360->27795455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760778.219851:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2879500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2359:x1876699512731776:12345-192.168.203.56@tcp:4:dd.0 Request processed in 49918us (51280us total) trans 4294974104 rc 0/0 00000100:00100000:1.0:1789760778.219862:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6815 00000100:00100000:1.0:1789760778.219872:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 757678533352 time 757734217052 state 0x2 ref 0 00000001:00080000:3.0:1789760778.220505:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974104 is committed 00000100:00100000:0.0:1789760778.224830:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760778.225368:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512731904 00000100:00100000:2.0:1789760778.225378:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512731904 00000100:00100000:2.0:1789760778.225435:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 757678533352 time 757734217052 state 0x0 ref 1 00000100:00100000:2.0:1789760778.225450:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 757734477468 time 757734217052 state 0x1 ref 1 00000100:00100000:2.0:1789760778.225484:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 757734477468 time 757789783553 state 0x0 ref 1 00000100:00100000:2.0:1789760778.225497:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac287b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512731904:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760778.225547:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27795456, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0xd807 00010000:00010000:2.0:1789760778.225556:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2fab8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.225572:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=757s v=(0 0) 00010000:00010000:2.0:1789760778.225577:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760778.225583:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2fab8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27795456->27799551] (req 27795456->27799551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.225598:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2fab8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27795456->27799551] (req 27795456->27799551) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760778.256419:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974105 is committed 00002000:00010000:0.0:1789760778.256441:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27795456 -> 27799552 00010000:00010000:0.0:1789760778.256457:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2fab8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27795456->27799551] (req 27795456->27799551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.256476:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2fab8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27795456->27799551] (req 27795456->27799551) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.256485:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2fab8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27795456->27799551] (req 27795456->27799551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.256498:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760778.256503:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2fab8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27795456->27799551] (req 27795456->27799551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.256513:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2fab8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27795456->27799551] (req 27795456->27799551) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.256533:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2fab8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27795456->27799551] (req 27795456->27799551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.256541:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2fab8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27795456->27799551] (req 27795456->27799551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760778.256753:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac287b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512731904:12345-192.168.203.56@tcp:4:dd.0 Request processed in 31135us (31809us total) trans 4294974105 rc 0/0 00000100:00100000:0.0:1789760778.256766:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6816 00000100:00100000:0.0:1789760778.256776:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 757734477468 time 757789783553 state 0x2 ref 0 00000100:00100000:1.0:1789760778.259825:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760778.259993:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512732032 00000100:00100000:2.0:1789760778.260003:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512732032 00000100:00100000:2.0:1789760778.260075:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 757734477468 time 757789783553 state 0x0 ref 1 00000100:00100000:2.0:1789760778.260091:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 757790043969 time 757789783553 state 0x1 ref 1 00000100:00100000:2.0:1789760778.260117:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 757790043969 time 757824417243 state 0x0 ref 1 00000100:00100000:2.0:1789760778.260131:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4130700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512732032:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760778.260178:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27799552, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0xd807 00010000:00010000:2.0:1789760778.260187:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2fabf lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.260202:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=757s v=(0 0) 00010000:00010000:2.0:1789760778.260207:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760778.260213:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2fabf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27799552->27803647] (req 27799552->27803647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.260225:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2fabf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27799552->27803647] (req 27799552->27803647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760778.309495:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974106 is committed 00002000:00010000:2.0:1789760778.309592:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27799552 -> 27803648 00010000:00010000:2.0:1789760778.309608:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2fabf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27799552->27803647] (req 27799552->27803647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.309621:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2fabf lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27799552->27803647] (req 27799552->27803647) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.309631:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2fabf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27799552->27803647] (req 27799552->27803647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.309638:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760778.309643:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2fabf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27799552->27803647] (req 27799552->27803647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.309652:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2fabf lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27799552->27803647] (req 27799552->27803647) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.309671:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2fabf lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27799552->27803647] (req 27799552->27803647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.309679:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2fabf lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27799552->27803647] (req 27799552->27803647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760778.310394:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4130700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512732032:12345-192.168.203.56@tcp:4:dd.0 Request processed in 49730us (50042us total) trans 4294974106 rc 0/0 00000100:00100000:2.0:1789760778.310406:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6817 00000100:00100000:2.0:1789760778.310416:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 757790043969 time 757824417243 state 0x2 ref 0 00000100:00100000:2.0:1789760778.314185:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760778.314275:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512732160 00000100:00100000:1.0:1789760778.314285:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512732160 00000100:00100000:1.0:1789760778.314347:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 757790043969 time 757824417243 state 0x0 ref 1 00000100:00100000:1.0:1789760778.314363:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 757824677659 time 757824417243 state 0x1 ref 1 00000100:00100000:1.0:1789760778.314391:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 757824677659 time 757878691032 state 0x0 ref 1 00000100:00100000:1.0:1789760778.314403:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4038380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512732160:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760778.314449:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27803648, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0xd807 00010000:00010000:1.0:1789760778.314457:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2fac6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.314472:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=757s v=(0 0) 00010000:00010000:1.0:1789760778.314476:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760778.314483:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2fac6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27803648->27807743] (req 27803648->27807743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.314497:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2fac6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27803648->27807743] (req 27803648->27807743) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760778.352338:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27803648 -> 27807744 00010000:00010000:3.0:1789760778.352359:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2fac6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27803648->27807743] (req 27803648->27807743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.352373:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2fac6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27803648->27807743] (req 27803648->27807743) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.352383:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2fac6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27803648->27807743] (req 27803648->27807743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.352391:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760778.352398:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2fac6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27803648->27807743] (req 27803648->27807743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.352408:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2fac6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27803648->27807743] (req 27803648->27807743) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.352427:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2fac6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27803648->27807743] (req 27803648->27807743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.352435:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2fac6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27803648->27807743] (req 27803648->27807743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760778.352533:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4038380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2360:x1876699512732160:12345-192.168.203.56@tcp:4:dd.0 Request processed in 38125us (38354us total) trans 4294974107 rc 0/0 00000100:00100000:3.0:1789760778.352544:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6818 00000100:00100000:3.0:1789760778.352554:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 757824677659 time 757878691032 state 0x2 ref 0 00000001:00080000:2.0:1789760778.353121:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974107 is committed 00000100:00100000:1.0:1789760778.355288:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760778.355379:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512732288 00000100:00100000:0.0:1789760778.355388:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512732288 00000100:00100000:0.0:1789760778.355440:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 757824677659 time 757878691032 state 0x0 ref 1 00000100:00100000:0.0:1789760778.355454:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 757878951448 time 757878691032 state 0x1 ref 1 00000100:00100000:0.0:1789760778.355481:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 757878951448 time 757919781437 state 0x0 ref 1 00000100:00100000:0.0:1789760778.355493:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4131180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512732288:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760778.355535:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27807744, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0xd807 00010000:00010000:0.0:1789760778.355543:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2facd lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.355558:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=757s v=(0 0) 00010000:00010000:0.0:1789760778.355562:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760778.355568:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2facd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27807744->27811839] (req 27807744->27811839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.355580:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2facd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27807744->27811839] (req 27807744->27811839) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760778.385292:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27807744 -> 27811840 00010000:00010000:0.0:1789760778.385312:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2facd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27807744->27811839] (req 27807744->27811839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.385325:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2facd lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27807744->27811839] (req 27807744->27811839) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.385341:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2facd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27807744->27811839] (req 27807744->27811839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.385347:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760778.385353:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2facd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27807744->27811839] (req 27807744->27811839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.385361:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2facd lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27807744->27811839] (req 27807744->27811839) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.385380:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2facd lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27807744->27811839] (req 27807744->27811839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.385388:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2facd lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27807744->27811839] (req 27807744->27811839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760778.385477:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4131180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2361:x1876699512732288:12345-192.168.203.56@tcp:4:dd.0 Request processed in 29981us (30195us total) trans 4294974108 rc 0/0 00000100:00100000:0.0:1789760778.385488:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6819 00000100:00100000:0.0:1789760778.385498:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 757878951448 time 757919781437 state 0x2 ref 0 00000001:00080000:3.0:1789760778.386139:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974108 is committed 00000100:00100000:2.0:1789760778.394609:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760778.396808:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512732416 00000100:00100000:1.0:1789760778.396822:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512732416 00000100:00100000:1.0:1789760778.396883:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 757878951448 time 757919781437 state 0x0 ref 1 00000100:00100000:1.0:1789760778.396899:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 757920041853 time 757919781437 state 0x1 ref 1 00000100:00100000:1.0:1789760778.396937:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 757920041853 time 757961236455 state 0x0 ref 1 00000100:00100000:1.0:1789760778.396950:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4039500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512732416:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760778.396999:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27811840, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0xd807 00010000:00010000:1.0:1789760778.397008:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2fad4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.397024:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=757s v=(0 0) 00010000:00010000:1.0:1789760778.397030:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760778.397037:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2fad4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27811840->27815935] (req 27811840->27815935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.397070:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2fad4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27811840->27815935] (req 27811840->27815935) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760778.433007:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974109 is committed 00002000:00010000:2.0:1789760778.433118:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27811840 -> 27815936 00010000:00010000:2.0:1789760778.433134:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2fad4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27811840->27815935] (req 27811840->27815935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.433147:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2fad4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27811840->27815935] (req 27811840->27815935) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.433157:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2fad4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27811840->27815935] (req 27811840->27815935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.433165:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760778.433170:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2fad4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27811840->27815935] (req 27811840->27815935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.433179:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2fad4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27811840->27815935] (req 27811840->27815935) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.433204:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2fad4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27811840->27815935] (req 27811840->27815935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.433212:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2fad4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27811840->27815935] (req 27811840->27815935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760778.433343:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4039500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512732416:12345-192.168.203.56@tcp:4:dd.0 Request processed in 36391us (38738us total) trans 4294974109 rc 0/0 00000100:00100000:2.0:1789760778.433353:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6820 00000100:00100000:2.0:1789760778.433363:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 757920041853 time 757961236455 state 0x2 ref 0 00000100:00100000:2.0:1789760778.436392:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760778.436522:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512732544 00000100:00100000:3.0:1789760778.436532:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512732544 00000100:00100000:3.0:1789760778.436585:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 757920041853 time 757961236455 state 0x0 ref 1 00000100:00100000:3.0:1789760778.436600:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 757961496871 time 757961236455 state 0x1 ref 1 00000100:00100000:3.0:1789760778.436631:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 757961496871 time 758000931042 state 0x0 ref 1 00000100:00100000:3.0:1789760778.436644:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac403a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512732544:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760778.436691:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27815936, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0xd807 00010000:00010000:3.0:1789760778.436723:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2fadb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.436738:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=757s v=(0 0) 00010000:00010000:3.0:1789760778.436743:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760778.436748:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2fadb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27815936->27820031] (req 27815936->27820031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.436763:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2fadb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27815936->27820031] (req 27815936->27820031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760778.471354:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974110 is committed 00002000:00010000:2.0:1789760778.471515:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27815936 -> 27820032 00010000:00010000:2.0:1789760778.471530:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2fadb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27815936->27820031] (req 27815936->27820031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.471544:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2fadb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27815936->27820031] (req 27815936->27820031) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.471553:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2fadb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27815936->27820031] (req 27815936->27820031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.471560:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760778.471565:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2fadb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27815936->27820031] (req 27815936->27820031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.471579:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2fadb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27815936->27820031] (req 27815936->27820031) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.471599:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2fadb lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27815936->27820031] (req 27815936->27820031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.471607:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2fadb lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27815936->27820031] (req 27815936->27820031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760778.471725:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac403a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512732544:12345-192.168.203.56@tcp:4:dd.0 Request processed in 35054us (35313us total) trans 4294974110 rc 0/0 00000100:00100000:2.0:1789760778.471746:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6821 00000100:00100000:2.0:1789760778.471755:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 757961496871 time 758000931042 state 0x2 ref 0 00000100:00100000:2.0:1789760778.476099:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760778.476204:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512732672 00000100:00100000:0.0:1789760778.476215:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512732672 00000100:00100000:0.0:1789760778.476267:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 757961496871 time 758000931042 state 0x0 ref 1 00000100:00100000:0.0:1789760778.476281:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 758001191458 time 758000931042 state 0x1 ref 1 00000100:00100000:0.0:1789760778.476346:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 758001191458 time 758040644410 state 0x0 ref 1 00000100:00100000:0.0:1789760778.476358:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4039c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512732672:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760778.476404:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27820032, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0xd807 00010000:00010000:0.0:1789760778.476412:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2fae2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.476427:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=758s v=(0 0) 00010000:00010000:0.0:1789760778.476432:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760778.476438:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2fae2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27820032->27824127] (req 27820032->27824127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.476464:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2fae2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27820032->27824127] (req 27820032->27824127) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760778.503268:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974111 is committed 00002000:00010000:0.0:1789760778.503727:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27820032 -> 27824128 00010000:00010000:0.0:1789760778.503892:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2fae2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27820032->27824127] (req 27820032->27824127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.503906:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2fae2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27820032->27824127] (req 27820032->27824127) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.503915:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2fae2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27820032->27824127] (req 27820032->27824127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.503923:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760778.503928:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2fae2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27820032->27824127] (req 27820032->27824127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.503937:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2fae2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27820032->27824127] (req 27820032->27824127) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.503955:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2fae2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27820032->27824127] (req 27820032->27824127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.503963:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2fae2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27820032->27824127] (req 27820032->27824127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760778.504054:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4039c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512732672:12345-192.168.203.56@tcp:4:dd.0 Request processed in 27694us (27961us total) trans 4294974111 rc 0/0 00000100:00100000:0.0:1789760778.504074:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6822 00000100:00100000:0.0:1789760778.504083:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 758001191458 time 758040644410 state 0x2 ref 0 00000100:00100000:2.0:1789760778.506980:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760778.507098:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512732800 00000100:00100000:1.0:1789760778.507108:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512732800 00000100:00100000:1.0:1789760778.507162:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 758001191458 time 758040644410 state 0x0 ref 1 00000100:00100000:1.0:1789760778.507178:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 758040904826 time 758040644410 state 0x1 ref 1 00000100:00100000:1.0:1789760778.507241:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 758040904826 time 758071541104 state 0x0 ref 1 00000100:00100000:1.0:1789760778.507254:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4038e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512732800:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760778.507299:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27824128, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0xd807 00010000:00010000:1.0:1789760778.507307:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2fae9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.507321:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=758s v=(0 0) 00010000:00010000:1.0:1789760778.507325:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760778.507331:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2fae9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27824128->27828223] (req 27824128->27828223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.507344:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2fae9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27824128->27828223] (req 27824128->27828223) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760778.533755:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974112 is committed 00002000:00010000:3.0:1789760778.533906:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27824128 -> 27828224 00002000:00010000:3.0:1789760778.533911:0:13062:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760777 -> 1789760778 00002000:00010000:3.0:1789760778.533913:0:13062:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760777 -> 1789760778 00010000:00010000:3.0:1789760778.533928:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2fae9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27824128->27828223] (req 27824128->27828223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.533942:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2fae9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27824128->27828223] (req 27824128->27828223) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.533950:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2fae9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27824128->27828223] (req 27824128->27828223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.533958:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760778.533962:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2fae9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27824128->27828223] (req 27824128->27828223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.533971:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2fae9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27824128->27828223] (req 27824128->27828223) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.533990:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2fae9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27824128->27828223] (req 27824128->27828223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.533999:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2fae9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27824128->27828223] (req 27824128->27828223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760778.534137:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4038e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512732800:12345-192.168.203.56@tcp:4:dd.0 Request processed in 26881us (27161us total) trans 4294974112 rc 0/0 00000100:00100000:3.0:1789760778.534149:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6823 00000100:00100000:3.0:1789760778.534159:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 758040904826 time 758071541104 state 0x2 ref 0 00000100:00100000:1.0:1789760778.537435:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760778.537586:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512732928 00000100:00100000:2.0:1789760778.537597:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512732928 00000100:00100000:2.0:1789760778.537647:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 758040904826 time 758071541104 state 0x0 ref 1 00000100:00100000:2.0:1789760778.537662:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 758071801520 time 758071541104 state 0x1 ref 1 00000100:00100000:2.0:1789760778.537855:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 758071801520 time 758102153181 state 0x0 ref 1 00000100:00100000:2.0:1789760778.537868:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4132300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512732928:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760778.537917:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27828224, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0xd807 00010000:00010000:2.0:1789760778.537926:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2faf0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.537944:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=758s v=(0 0) 00010000:00010000:2.0:1789760778.537948:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760778.537954:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2faf0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27828224->27832319] (req 27828224->27832319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.537967:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2faf0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27828224->27832319] (req 27828224->27832319) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760778.568939:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974113 is committed 00002000:00010000:3.0:1789760778.569054:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27828224 -> 27832320 00010000:00010000:3.0:1789760778.569077:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2faf0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27828224->27832319] (req 27828224->27832319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.569090:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2faf0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27828224->27832319] (req 27828224->27832319) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.569099:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2faf0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27828224->27832319] (req 27828224->27832319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.569106:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760778.569113:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2faf0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27828224->27832319] (req 27828224->27832319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.569123:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2faf0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27828224->27832319] (req 27828224->27832319) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.569141:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2faf0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27828224->27832319] (req 27828224->27832319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.569149:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2faf0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27828224->27832319] (req 27828224->27832319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760778.569243:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4132300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512732928:12345-192.168.203.56@tcp:4:dd.0 Request processed in 31373us (31814us total) trans 4294974113 rc 0/0 00000100:00100000:3.0:1789760778.569252:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6824 00000100:00100000:3.0:1789760778.569262:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 758071801520 time 758102153181 state 0x2 ref 0 00000100:00100000:2.0:1789760778.572315:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760778.572400:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512733056 00000100:00100000:0.0:1789760778.572411:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512733056 00000100:00100000:0.0:1789760778.572530:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 758071801520 time 758102153181 state 0x0 ref 1 00000100:00100000:0.0:1789760778.572545:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 758102413597 time 758102153181 state 0x1 ref 1 00000100:00100000:0.0:1789760778.572606:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 758102413597 time 758136905681 state 0x0 ref 1 00000100:00100000:0.0:1789760778.572620:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac403b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512733056:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760778.572666:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27832320, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0xd807 00010000:00010000:0.0:1789760778.572674:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2faf7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.572690:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=758s v=(0 0) 00010000:00010000:0.0:1789760778.572809:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760778.572818:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2faf7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27832320->27836415] (req 27832320->27836415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.572833:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2faf7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27832320->27836415] (req 27832320->27836415) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760778.598415:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974114 is committed 00002000:00010000:0.0:1789760778.598589:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27832320 -> 27836416 00010000:00010000:0.0:1789760778.598605:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2faf7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27832320->27836415] (req 27832320->27836415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.598616:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2faf7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27832320->27836415] (req 27832320->27836415) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.598623:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2faf7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27832320->27836415] (req 27832320->27836415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.598630:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760778.598635:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2faf7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27832320->27836415] (req 27832320->27836415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.598644:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2faf7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27832320->27836415] (req 27832320->27836415) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.598661:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2faf7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27832320->27836415] (req 27832320->27836415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.598668:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2faf7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27832320->27836415] (req 27832320->27836415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760778.598885:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac403b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512733056:12345-192.168.203.56@tcp:4:dd.0 Request processed in 26262us (26572us total) trans 4294974114 rc 0/0 00000100:00100000:0.0:1789760778.598896:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6825 00000100:00100000:0.0:1789760778.598905:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 758102413597 time 758136905681 state 0x2 ref 0 00000100:00100000:0.0:1789760778.601502:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760778.601674:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512733184 00000100:00100000:3.0:1789760778.601683:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512733184 00000100:00100000:3.0:1789760778.601766:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 758102413597 time 758136905681 state 0x0 ref 1 00000100:00100000:3.0:1789760778.601781:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 758137166097 time 758136905681 state 0x1 ref 1 00000100:00100000:3.0:1789760778.601802:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 758137166097 time 758166102102 state 0x0 ref 1 00000100:00100000:3.0:1789760778.601812:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac287bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512733184:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760778.601855:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27836416, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0xd807 00010000:00010000:3.0:1789760778.601864:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2fafe lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.601879:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=758s v=(0 0) 00010000:00010000:3.0:1789760778.601883:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760778.601890:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2fafe lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27836416->27840511] (req 27836416->27840511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.601903:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2fafe lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27836416->27840511] (req 27836416->27840511) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.612199:0:5836:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:0.0:1789760778.637570:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974115 is committed 00002000:00010000:3.0:1789760778.637847:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27836416 -> 27840512 00010000:00010000:3.0:1789760778.637862:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2fafe lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27836416->27840511] (req 27836416->27840511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.637874:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2fafe lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27836416->27840511] (req 27836416->27840511) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.637883:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2fafe lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27836416->27840511] (req 27836416->27840511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.637891:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760778.637896:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2fafe lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27836416->27840511] (req 27836416->27840511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.637906:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2fafe lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27836416->27840511] (req 27836416->27840511) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.637926:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2fafe lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27836416->27840511] (req 27836416->27840511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.637934:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2fafe lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27836416->27840511] (req 27836416->27840511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760778.638075:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac287bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512733184:12345-192.168.203.56@tcp:4:dd.0 Request processed in 36257us (36575us total) trans 4294974115 rc 0/0 00000100:00100000:3.0:1789760778.638085:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6826 00000100:00100000:3.0:1789760778.638094:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 758137166097 time 758166102102 state 0x2 ref 0 00000100:00100000:3.0:1789760778.646389:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760778.646477:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512733312 00000100:00100000:0.0:1789760778.646487:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512733312 00000100:00100000:0.0:1789760778.646537:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 758137166097 time 758166102102 state 0x0 ref 1 00000100:00100000:0.0:1789760778.646564:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 758166362518 time 758166102102 state 0x1 ref 1 00000100:00100000:0.0:1789760778.646594:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 758166362518 time 758210893471 state 0x0 ref 1 00000100:00100000:0.0:1789760778.646607:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febc7b32680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512733312:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760778.646659:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27840512, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0xd807 00010000:00010000:0.0:1789760778.646668:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2fb05 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.646683:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=758s v=(0 0) 00010000:00010000:0.0:1789760778.646688:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760778.646714:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2fb05 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27840512->27844607] (req 27840512->27844607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.646729:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2fb05 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27840512->27844607] (req 27840512->27844607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760778.685243:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27840512 -> 27844608 00010000:00010000:0.0:1789760778.685265:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2fb05 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27840512->27844607] (req 27840512->27844607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.685279:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2fb05 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27840512->27844607] (req 27840512->27844607) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.685289:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2fb05 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27840512->27844607] (req 27840512->27844607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.685298:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760778.685303:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2fb05 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27840512->27844607] (req 27840512->27844607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.685313:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2fb05 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27840512->27844607] (req 27840512->27844607) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.685402:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2fb05 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27840512->27844607] (req 27840512->27844607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.685412:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2fb05 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27840512->27844607] (req 27840512->27844607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760778.685612:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febc7b32680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2358:x1876699512733312:12345-192.168.203.56@tcp:4:dd.0 Request processed in 39002us (39230us total) trans 4294974116 rc 0/0 00000100:00100000:0.0:1789760778.685623:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6827 00000100:00100000:0.0:1789760778.685632:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 758166362518 time 758210893471 state 0x2 ref 0 00000001:00080000:0.0:1789760778.685671:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974116 is committed 00000100:00100000:0.0:1789760778.689263:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760778.689429:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512733440 00000100:00100000:2.0:1789760778.689438:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512733440 00000100:00100000:2.0:1789760778.689542:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 758166362518 time 758210893471 state 0x0 ref 1 00000100:00100000:2.0:1789760778.689556:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 758211153887 time 758210893471 state 0x1 ref 1 00000100:00100000:2.0:1789760778.689668:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 758211153887 time 758253965902 state 0x0 ref 1 00000100:00100000:2.0:1789760778.689681:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2878380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512733440:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760778.689770:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27844608, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0xd807 00010000:00010000:2.0:1789760778.689778:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2fb0c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.689798:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=758s v=(0 0) 00010000:00010000:2.0:1789760778.689801:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760778.689807:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2fb0c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27844608->27848703] (req 27844608->27848703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.689820:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2fb0c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27844608->27848703] (req 27844608->27848703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760778.729683:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27844608 -> 27848704 00010000:00010000:2.0:1789760778.729735:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2fb0c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27844608->27848703] (req 27844608->27848703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.729749:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2fb0c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27844608->27848703] (req 27844608->27848703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.729758:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2fb0c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27844608->27848703] (req 27844608->27848703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.729765:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760778.729770:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2fb0c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27844608->27848703] (req 27844608->27848703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.729778:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2fb0c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27844608->27848703] (req 27844608->27848703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.729797:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2fb0c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27844608->27848703] (req 27844608->27848703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.729805:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2fb0c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27844608->27848703] (req 27844608->27848703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760778.729897:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2878380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2358:x1876699512733440:12345-192.168.203.56@tcp:4:dd.0 Request processed in 40214us (40638us total) trans 4294974117 rc 0/0 00000100:00100000:2.0:1789760778.729907:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6828 00000100:00100000:2.0:1789760778.729916:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 758211153887 time 758253965902 state 0x2 ref 0 00000001:00080000:1.0:1789760778.732379:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974117 is committed 00000100:00100000:0.0:1789760778.733351:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760778.733524:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512733568 00000100:00100000:3.0:1789760778.733533:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512733568 00000100:00100000:3.0:1789760778.733588:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 758211153887 time 758253965902 state 0x0 ref 1 00000100:00100000:3.0:1789760778.733603:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 758254226318 time 758253965902 state 0x1 ref 1 00000100:00100000:3.0:1789760778.733662:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 758254226318 time 758297961192 state 0x0 ref 1 00000100:00100000:3.0:1789760778.733675:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac287a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512733568:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760778.733749:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27848704, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0xd807 00010000:00010000:3.0:1789760778.733759:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2fb13 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.733780:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=758s v=(0 0) 00010000:00010000:3.0:1789760778.733785:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760778.733790:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2fb13 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27848704->27852799] (req 27848704->27852799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.733804:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2fb13 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27848704->27852799] (req 27848704->27852799) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760778.760917:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974118 is committed 00002000:00010000:3.0:1789760778.760988:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27848704 -> 27852800 00010000:00010000:3.0:1789760778.761005:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2fb13 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27848704->27852799] (req 27848704->27852799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.761018:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2fb13 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27848704->27852799] (req 27848704->27852799) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.761027:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2fb13 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27848704->27852799] (req 27848704->27852799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.761159:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760778.761165:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2fb13 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27848704->27852799] (req 27848704->27852799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.761176:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2fb13 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27848704->27852799] (req 27848704->27852799) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.761195:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2fb13 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27848704->27852799] (req 27848704->27852799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.761203:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2fb13 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27848704->27852799] (req 27848704->27852799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760778.764169:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac287a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512733568:12345-192.168.203.56@tcp:4:dd.0 Request processed in 30484us (30811us total) trans 4294974118 rc 0/0 00000100:00100000:3.0:1789760778.764185:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6829 00000100:00100000:3.0:1789760778.764195:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 758254226318 time 758297961192 state 0x2 ref 0 00000100:00100000:0.0:1789760778.769558:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760778.769729:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512733696 00000100:00100000:3.0:1789760778.769739:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512733696 00000100:00100000:3.0:1789760778.769795:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 758254226318 time 758297961192 state 0x0 ref 1 00000100:00100000:3.0:1789760778.769811:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 758298221608 time 758297961192 state 0x1 ref 1 00000100:00100000:3.0:1789760778.769846:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 758298221608 time 758334145960 state 0x0 ref 1 00000100:00100000:3.0:1789760778.769860:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2879c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512733696:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760778.769906:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27852800, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0xd807 00010000:00010000:3.0:1789760778.769916:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2fb1a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.769931:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=758s v=(0 0) 00010000:00010000:3.0:1789760778.769935:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760778.769941:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2fb1a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27852800->27856895] (req 27852800->27856895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.769957:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2fb1a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27852800->27856895] (req 27852800->27856895) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760778.828125:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974119 is committed 00002000:00010000:0.0:1789760778.828519:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27852800 -> 27856896 00010000:00010000:0.0:1789760778.828534:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2fb1a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27852800->27856895] (req 27852800->27856895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.828819:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2fb1a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27852800->27856895] (req 27852800->27856895) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.828828:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2fb1a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27852800->27856895] (req 27852800->27856895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.828835:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760778.828841:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2fb1a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27852800->27856895] (req 27852800->27856895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.828851:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2fb1a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27852800->27856895] (req 27852800->27856895) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.828870:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2fb1a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27852800->27856895] (req 27852800->27856895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.828877:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2fb1a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27852800->27856895] (req 27852800->27856895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760778.829017:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2879c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512733696:12345-192.168.203.56@tcp:4:dd.0 Request processed in 59155us (59463us total) trans 4294974119 rc 0/0 00000100:00100000:0.0:1789760778.829027:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6830 00000100:00100000:0.0:1789760778.829037:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 758298221608 time 758334145960 state 0x2 ref 0 00000100:00100000:3.0:1789760778.836514:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760778.836841:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512733824 00000100:00100000:2.0:1789760778.836851:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512733824 00000100:00100000:2.0:1789760778.836908:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 758298221608 time 758334145960 state 0x0 ref 1 00000100:00100000:2.0:1789760778.836923:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 758334406376 time 758334145960 state 0x1 ref 1 00000100:00100000:2.0:1789760778.836986:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 758334406376 time 758401285090 state 0x0 ref 1 00000100:00100000:2.0:1789760778.836998:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febc7b33100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512733824:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760778.837042:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27856896, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0xd807 00010000:00010000:2.0:1789760778.837052:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2fb21 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.837079:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=758s v=(0 0) 00010000:00010000:2.0:1789760778.837084:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760778.837090:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2fb21 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27856896->27860991] (req 27856896->27860991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.837110:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2fb21 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27856896->27860991] (req 27856896->27860991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760778.861511:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974120 is committed 00002000:00010000:0.0:1789760778.861580:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27856896 -> 27860992 00010000:00010000:0.0:1789760778.861595:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2fb21 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27856896->27860991] (req 27856896->27860991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.861608:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2fb21 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27856896->27860991] (req 27856896->27860991) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.861617:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2fb21 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27856896->27860991] (req 27856896->27860991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.861629:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760778.861634:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2fb21 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27856896->27860991] (req 27856896->27860991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.861645:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2fb21 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27856896->27860991] (req 27856896->27860991) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.861664:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2fb21 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27856896->27860991] (req 27856896->27860991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.861672:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2fb21 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27856896->27860991] (req 27856896->27860991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760778.861792:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febc7b33100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512733824:12345-192.168.203.56@tcp:4:dd.0 Request processed in 24790us (25284us total) trans 4294974120 rc 0/0 00000100:00100000:0.0:1789760778.861802:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6831 00000100:00100000:0.0:1789760778.861810:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 758334406376 time 758401285090 state 0x2 ref 0 00000100:00100000:1.0:1789760778.864234:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760778.864335:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512733952 00000100:00100000:3.0:1789760778.864345:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512733952 00000100:00100000:3.0:1789760778.864409:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 758334406376 time 758401285090 state 0x0 ref 1 00000100:00100000:3.0:1789760778.864424:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 758401545506 time 758401285090 state 0x1 ref 1 00000100:00100000:3.0:1789760778.864483:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 758401545506 time 758428783202 state 0x0 ref 1 00000100:00100000:3.0:1789760778.864496:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4131f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512733952:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760778.864554:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27860992, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0xd807 00010000:00010000:3.0:1789760778.864562:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2fb28 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.864578:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=758s v=(0 0) 00010000:00010000:3.0:1789760778.864582:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760778.864588:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2fb28 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27860992->27865087] (req 27860992->27865087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.864602:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2fb28 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27860992->27865087] (req 27860992->27865087) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760778.910982:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974121 is committed 00002000:00010000:0.0:1789760778.911085:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27860992 -> 27865088 00010000:00010000:0.0:1789760778.911101:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2fb28 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27860992->27865087] (req 27860992->27865087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.911114:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2fb28 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27860992->27865087] (req 27860992->27865087) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.911130:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2fb28 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27860992->27865087] (req 27860992->27865087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.911137:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760778.911142:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2fb28 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27860992->27865087] (req 27860992->27865087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.911152:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2fb28 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27860992->27865087] (req 27860992->27865087) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.911170:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2fb28 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27860992->27865087] (req 27860992->27865087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.911178:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2fb28 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27860992->27865087] (req 27860992->27865087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760778.911305:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4131f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512733952:12345-192.168.203.56@tcp:4:dd.0 Request processed in 46806us (47074us total) trans 4294974121 rc 0/0 00000100:00100000:0.0:1789760778.911315:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6832 00000100:00100000:0.0:1789760778.911325:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 758401545506 time 758428783202 state 0x2 ref 0 00000100:00100000:1.0:1789760778.920219:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760778.920417:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512734080 00000100:00100000:0.0:1789760778.920427:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512734080 00000100:00100000:0.0:1789760778.920481:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 758401545506 time 758428783202 state 0x0 ref 1 00000100:00100000:0.0:1789760778.920497:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 758429043618 time 758428783202 state 0x1 ref 1 00000100:00100000:0.0:1789760778.920530:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 758429043618 time 758484829525 state 0x0 ref 1 00000100:00100000:0.0:1789760778.920543:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4130380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512734080:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760778.920594:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27865088, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0xd807 00010000:00010000:0.0:1789760778.920603:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2fb2f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.920618:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=758s v=(0 0) 00010000:00010000:0.0:1789760778.920623:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760778.920629:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2fb2f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27865088->27869183] (req 27865088->27869183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.920642:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2fb2f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27865088->27869183] (req 27865088->27869183) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760778.955131:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974122 is committed 00002000:00010000:2.0:1789760778.955228:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27865088 -> 27869184 00010000:00010000:2.0:1789760778.955243:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2fb2f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27865088->27869183] (req 27865088->27869183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.955256:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2fb2f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27865088->27869183] (req 27865088->27869183) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.955266:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2fb2f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27865088->27869183] (req 27865088->27869183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.955274:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760778.955278:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2fb2f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27865088->27869183] (req 27865088->27869183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.955288:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2fb2f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27865088->27869183] (req 27865088->27869183) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.955307:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2fb2f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27865088->27869183] (req 27865088->27869183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.955314:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2fb2f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27865088->27869183] (req 27865088->27869183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760778.955409:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4130380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512734080:12345-192.168.203.56@tcp:4:dd.0 Request processed in 34864us (35195us total) trans 4294974122 rc 0/0 00000100:00100000:2.0:1789760778.955419:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6833 00000100:00100000:2.0:1789760778.955429:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 758429043618 time 758484829525 state 0x2 ref 0 00000100:00100000:3.0:1789760778.958107:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760778.958354:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512734208 00000100:00100000:0.0:1789760778.958363:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512734208 00000100:00100000:0.0:1789760778.958413:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 758429043618 time 758484829525 state 0x0 ref 1 00000100:00100000:0.0:1789760778.958428:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 758485089941 time 758484829525 state 0x1 ref 1 00000100:00100000:0.0:1789760778.958459:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 758485089941 time 758522758028 state 0x0 ref 1 00000100:00100000:0.0:1789760778.958471:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febc7b33480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512734208:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760778.958515:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27869184, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0xd807 00010000:00010000:0.0:1789760778.958524:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2fb36 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.958538:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=758s v=(0 0) 00010000:00010000:0.0:1789760778.958543:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760778.958549:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2fb36 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27869184->27873279] (req 27869184->27873279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.958563:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2fb36 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27869184->27873279] (req 27869184->27873279) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760778.991499:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974123 is committed 00002000:00010000:1.0:1789760778.991607:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27869184 -> 27873280 00010000:00010000:1.0:1789760778.991622:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2fb36 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27869184->27873279] (req 27869184->27873279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.991634:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2fb36 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27869184->27873279] (req 27869184->27873279) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.991642:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2fb36 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27869184->27873279] (req 27869184->27873279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.991650:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760778.991656:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2fb36 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27869184->27873279] (req 27869184->27873279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.991665:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2fb36 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27869184->27873279] (req 27869184->27873279) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.991684:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2fb36 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27869184->27873279] (req 27869184->27873279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.991692:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2fb36 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27869184->27873279] (req 27869184->27873279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760778.991817:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febc7b33480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512734208:12345-192.168.203.56@tcp:4:dd.0 Request processed in 33343us (33712us total) trans 4294974123 rc 0/0 00000100:00100000:1.0:1789760778.991828:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6834 00000100:00100000:1.0:1789760778.991838:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 758485089941 time 758522758028 state 0x2 ref 0 00000100:00100000:0.0:1789760778.996078:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760778.996216:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512734336 00000100:00100000:3.0:1789760778.996226:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512734336 00000100:00100000:3.0:1789760778.996283:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 758485089941 time 758522758028 state 0x0 ref 1 00000100:00100000:3.0:1789760778.996299:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 758523018444 time 758522758028 state 0x1 ref 1 00000100:00100000:3.0:1789760778.996332:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 758523018444 time 758560631320 state 0x0 ref 1 00000100:00100000:3.0:1789760778.996344:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2878e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512734336:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760778.996391:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27873280, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0xd807 00010000:00010000:3.0:1789760778.996400:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2fb3d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.996416:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=758s v=(0 0) 00010000:00010000:3.0:1789760778.996421:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760778.996426:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2fb3d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27873280->27877375] (req 27873280->27877375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.996439:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2fb3d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27873280->27877375] (req 27873280->27877375) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760779.035918:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27873280 -> 27877376 00010000:00010000:3.0:1789760779.035939:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2fb3d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27873280->27877375] (req 27873280->27877375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.035953:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2fb3d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27873280->27877375] (req 27873280->27877375) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.035962:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2fb3d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27873280->27877375] (req 27873280->27877375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.035971:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760779.035976:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2fb3d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27873280->27877375] (req 27873280->27877375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.035985:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2fb3d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27873280->27877375] (req 27873280->27877375) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.036004:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2fb3d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27873280->27877375] (req 27873280->27877375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.036012:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2fb3d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27873280->27877375] (req 27873280->27877375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760779.036179:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2878e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2359:x1876699512734336:12345-192.168.203.56@tcp:4:dd.0 Request processed in 39831us (40103us total) trans 4294974124 rc 0/0 00000100:00100000:3.0:1789760779.036193:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6835 00000100:00100000:3.0:1789760779.036203:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 758523018444 time 758560631320 state 0x2 ref 0 00000001:00080000:1.0:1789760779.036277:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974124 is committed 00000100:00100000:0.0:1789760779.040009:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760779.040093:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512734464 00000100:00100000:2.0:1789760779.040103:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512734464 00000100:00100000:2.0:1789760779.040199:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 758523018444 time 758560631320 state 0x0 ref 1 00000100:00100000:2.0:1789760779.040215:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 758560891736 time 758560631320 state 0x1 ref 1 00000100:00100000:2.0:1789760779.040245:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 758560891736 time 758604544338 state 0x0 ref 1 00000100:00100000:2.0:1789760779.040258:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febea89b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512734464:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760779.040309:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27877376, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0xd807 00010000:00010000:2.0:1789760779.040318:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2fb44 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.040335:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=758s v=(0 0) 00010000:00010000:2.0:1789760779.040340:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760779.040347:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2fb44 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27877376->27881471] (req 27877376->27881471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.040360:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2fb44 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27877376->27881471] (req 27877376->27881471) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1789760779.041203:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:0.0:1789760779.075344:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974125 is committed 00002000:00010000:3.0:1789760779.075462:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27877376 -> 27881472 00010000:00010000:3.0:1789760779.075478:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2fb44 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27877376->27881471] (req 27877376->27881471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.075490:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2fb44 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27877376->27881471] (req 27877376->27881471) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.075499:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2fb44 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27877376->27881471] (req 27877376->27881471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.075507:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760779.075512:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2fb44 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27877376->27881471] (req 27877376->27881471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.075522:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2fb44 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27877376->27881471] (req 27877376->27881471) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.075541:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2fb44 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27877376->27881471] (req 27877376->27881471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.075548:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2fb44 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27877376->27881471] (req 27877376->27881471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760779.075675:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febea89b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512734464:12345-192.168.203.56@tcp:4:dd.0 Request processed in 35416us (35671us total) trans 4294974125 rc 0/0 00000100:00100000:3.0:1789760779.075685:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6836 00000100:00100000:3.0:1789760779.075694:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 758560891736 time 758604544338 state 0x2 ref 0 00000100:00100000:0.0:1789760779.079211:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760779.079320:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512734592 00000100:00100000:1.0:1789760779.079329:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512734592 00000100:00100000:1.0:1789760779.079383:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 758560891736 time 758604544338 state 0x0 ref 1 00000100:00100000:1.0:1789760779.079399:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 758604804754 time 758604544338 state 0x1 ref 1 00000100:00100000:1.0:1789760779.079435:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 758604804754 time 758643733049 state 0x0 ref 1 00000100:00100000:1.0:1789760779.079447:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5305500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512734592:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760779.079495:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27881472, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0xd807 00010000:00010000:1.0:1789760779.079503:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2fb4b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.079518:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=758s v=(0 0) 00010000:00010000:1.0:1789760779.079523:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760779.079529:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2fb4b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27881472->27885567] (req 27881472->27885567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.079542:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2fb4b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27881472->27885567] (req 27881472->27885567) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760779.113630:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27881472 -> 27885568 00010000:00010000:0.0:1789760779.113652:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2fb4b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27881472->27885567] (req 27881472->27885567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.113665:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2fb4b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27881472->27885567] (req 27881472->27885567) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.113675:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2fb4b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27881472->27885567] (req 27881472->27885567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.113682:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760779.113687:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2fb4b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27881472->27885567] (req 27881472->27885567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.113800:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2fb4b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27881472->27885567] (req 27881472->27885567) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.113823:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2fb4b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27881472->27885567] (req 27881472->27885567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.113831:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2fb4b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27881472->27885567] (req 27881472->27885567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760779.113913:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5305500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2358:x1876699512734592:12345-192.168.203.56@tcp:4:dd.0 Request processed in 34463us (34706us total) trans 4294974126 rc 0/0 00000100:00100000:0.0:1789760779.113924:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6837 00000100:00100000:0.0:1789760779.113933:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 758604804754 time 758643733049 state 0x2 ref 0 00000001:00080000:3.0:1789760779.114412:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974126 is committed 00000100:00100000:0.0:1789760779.117293:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760779.117378:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512734720 00000100:00100000:3.0:1789760779.117388:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512734720 00000100:00100000:3.0:1789760779.117444:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 758604804754 time 758643733049 state 0x0 ref 1 00000100:00100000:3.0:1789760779.117457:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 758643993465 time 758643733049 state 0x1 ref 1 00000100:00100000:3.0:1789760779.117486:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 758643993465 time 758681785815 state 0x0 ref 1 00000100:00100000:3.0:1789760779.117498:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5307b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512734720:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760779.117546:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27885568, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0xd807 00010000:00010000:3.0:1789760779.117554:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2fb52 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.117570:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=758s v=(0 0) 00010000:00010000:3.0:1789760779.117574:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760779.117579:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2fb52 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27885568->27889663] (req 27885568->27889663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.117593:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2fb52 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27885568->27889663] (req 27885568->27889663) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760779.146770:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974127 is committed 00002000:00010000:3.0:1789760779.147067:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27885568 -> 27889664 00010000:00010000:3.0:1789760779.147129:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2fb52 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27885568->27889663] (req 27885568->27889663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.147141:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2fb52 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27885568->27889663] (req 27885568->27889663) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.147151:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2fb52 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27885568->27889663] (req 27885568->27889663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.147158:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760779.147163:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2fb52 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27885568->27889663] (req 27885568->27889663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.147172:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2fb52 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27885568->27889663] (req 27885568->27889663) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.147191:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2fb52 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27885568->27889663] (req 27885568->27889663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.147199:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2fb52 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27885568->27889663] (req 27885568->27889663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760779.147293:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5307b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512734720:12345-192.168.203.56@tcp:4:dd.0 Request processed in 29792us (30006us total) trans 4294974127 rc 0/0 00000100:00100000:3.0:1789760779.147320:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6838 00000100:00100000:3.0:1789760779.147330:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 758643993465 time 758681785815 state 0x2 ref 0 00000100:00100000:1.0:1789760779.155710:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760779.155999:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512734848 00000100:00100000:3.0:1789760779.156011:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512734848 00000100:00100000:3.0:1789760779.156084:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 758643993465 time 758681785815 state 0x0 ref 1 00000100:00100000:3.0:1789760779.156099:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 758682046231 time 758681785815 state 0x1 ref 1 00000100:00100000:3.0:1789760779.156176:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 758682046231 time 758720475124 state 0x0 ref 1 00000100:00100000:3.0:1789760779.156197:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4133b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512734848:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760779.156246:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27889664, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0xd807 00010000:00010000:3.0:1789760779.156255:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2fb59 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.156271:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=758s v=(0 0) 00010000:00010000:3.0:1789760779.156275:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760779.156281:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2fb59 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27889664->27893759] (req 27889664->27893759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.156295:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2fb59 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27889664->27893759] (req 27889664->27893759) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760779.186559:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27889664 -> 27893760 00010000:00010000:0.0:1789760779.186582:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2fb59 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27889664->27893759] (req 27889664->27893759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.186596:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2fb59 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27889664->27893759] (req 27889664->27893759) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.186610:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2fb59 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27889664->27893759] (req 27889664->27893759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.186623:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760779.186627:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2fb59 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27889664->27893759] (req 27889664->27893759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760779.186630:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974128 is committed 00010000:00010000:0.0:1789760779.186636:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2fb59 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27889664->27893759] (req 27889664->27893759) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.186654:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2fb59 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27889664->27893759] (req 27889664->27893759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.186663:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2fb59 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27889664->27893759] (req 27889664->27893759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760779.186897:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4133b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512734848:12345-192.168.203.56@tcp:4:dd.0 Request processed in 30696us (31209us total) trans 4294974128 rc 0/0 00000100:00100000:0.0:1789760779.186908:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6839 00000100:00100000:0.0:1789760779.186917:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 758682046231 time 758720475124 state 0x2 ref 0 00000100:00100000:0.0:1789760779.189653:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760779.189801:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512734976 00000100:00100000:1.0:1789760779.189811:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512734976 00000100:00100000:1.0:1789760779.189866:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 758682046231 time 758720475124 state 0x0 ref 1 00000100:00100000:1.0:1789760779.189882:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 758720735540 time 758720475124 state 0x1 ref 1 00000100:00100000:1.0:1789760779.189912:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 758720735540 time 758754211745 state 0x0 ref 1 00000100:00100000:1.0:1789760779.189925:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5305f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512734976:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760779.189985:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27893760, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0xd807 00010000:00010000:1.0:1789760779.189995:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2fb60 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.190011:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=758s v=(0 0) 00010000:00010000:1.0:1789760779.190016:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760779.190023:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2fb60 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27893760->27897855] (req 27893760->27897855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.190037:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2fb60 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27893760->27897855] (req 27893760->27897855) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760779.232150:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27893760 -> 27897856 00010000:00010000:1.0:1789760779.232168:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2fb60 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27893760->27897855] (req 27893760->27897855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.232180:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2fb60 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27893760->27897855] (req 27893760->27897855) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.232190:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2fb60 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27893760->27897855] (req 27893760->27897855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.232197:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760779.232202:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2fb60 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27893760->27897855] (req 27893760->27897855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.232212:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2fb60 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27893760->27897855] (req 27893760->27897855) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.232232:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2fb60 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27893760->27897855] (req 27893760->27897855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.232241:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2fb60 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27893760->27897855] (req 27893760->27897855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760779.232331:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5305f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2360:x1876699512734976:12345-192.168.203.56@tcp:4:dd.0 Request processed in 42403us (42682us total) trans 4294974129 rc 0/0 00000100:00100000:1.0:1789760779.232340:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6840 00000100:00100000:1.0:1789760779.232349:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 758720735540 time 758754211745 state 0x2 ref 0 00000001:00080000:1.0:1789760779.232383:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974129 is committed 00000100:00100000:0.0:1789760779.235125:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760779.235287:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512735104 00000100:00100000:3.0:1789760779.235298:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512735104 00000100:00100000:3.0:1789760779.235350:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 758720735540 time 758754211745 state 0x0 ref 1 00000100:00100000:3.0:1789760779.235364:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 758754472161 time 758754211745 state 0x1 ref 1 00000100:00100000:3.0:1789760779.235395:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 758754472161 time 758799694587 state 0x0 ref 1 00000100:00100000:3.0:1789760779.235408:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5307480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512735104:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760779.235464:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27897856, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0xd807 00010000:00010000:3.0:1789760779.235472:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2fb67 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.235487:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=758s v=(0 0) 00010000:00010000:3.0:1789760779.235492:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760779.235498:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2fb67 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27897856->27901951] (req 27897856->27901951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.235511:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2fb67 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27897856->27901951] (req 27897856->27901951) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760779.263823:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974130 is committed 00002000:00010000:3.0:1789760779.263935:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27897856 -> 27901952 00010000:00010000:3.0:1789760779.263949:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2fb67 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27897856->27901951] (req 27897856->27901951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.263961:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2fb67 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27897856->27901951] (req 27897856->27901951) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.263970:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2fb67 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27897856->27901951] (req 27897856->27901951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.263979:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760779.263984:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2fb67 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27897856->27901951] (req 27897856->27901951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.263993:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2fb67 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27897856->27901951] (req 27897856->27901951) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.264012:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2fb67 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27897856->27901951] (req 27897856->27901951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.264020:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2fb67 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27897856->27901951] (req 27897856->27901951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760779.264118:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5307480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512735104:12345-192.168.203.56@tcp:4:dd.0 Request processed in 28708us (28996us total) trans 4294974130 rc 0/0 00000100:00100000:3.0:1789760779.264128:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6841 00000100:00100000:3.0:1789760779.264137:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 758754472161 time 758799694587 state 0x2 ref 0 00000100:00100000:3.0:1789760779.267089:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760779.267253:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512735232 00000100:00100000:0.0:1789760779.267262:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512735232 00000100:00100000:0.0:1789760779.267308:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 758754472161 time 758799694587 state 0x0 ref 1 00000100:00100000:0.0:1789760779.267322:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 758799955003 time 758799694587 state 0x1 ref 1 00000100:00100000:0.0:1789760779.267357:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 758799955003 time 758831657547 state 0x0 ref 1 00000100:00100000:0.0:1789760779.267368:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac531ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512735232:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760779.267409:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27901952, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0xd807 00010000:00010000:0.0:1789760779.267417:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2fb6e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.267432:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=758s v=(0 0) 00010000:00010000:0.0:1789760779.267436:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760779.267442:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2fb6e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27901952->27906047] (req 27901952->27906047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.267461:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2fb6e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27901952->27906047] (req 27901952->27906047) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760779.292844:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974131 is committed 00002000:00010000:3.0:1789760779.293012:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27901952 -> 27906048 00010000:00010000:3.0:1789760779.293027:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2fb6e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27901952->27906047] (req 27901952->27906047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.293049:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2fb6e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27901952->27906047] (req 27901952->27906047) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.293076:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2fb6e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27901952->27906047] (req 27901952->27906047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.293084:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760779.293090:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2fb6e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27901952->27906047] (req 27901952->27906047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.293100:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2fb6e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27901952->27906047] (req 27901952->27906047) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.293119:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2fb6e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27901952->27906047] (req 27901952->27906047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.293203:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2fb6e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27901952->27906047] (req 27901952->27906047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760779.293300:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac531ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512735232:12345-192.168.203.56@tcp:4:dd.0 Request processed in 25928us (26213us total) trans 4294974131 rc 0/0 00000100:00100000:3.0:1789760779.293311:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6842 00000100:00100000:3.0:1789760779.293319:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 758799955003 time 758831657547 state 0x2 ref 0 00000100:00100000:0.0:1789760779.296507:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760779.297296:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512735360 00000100:00100000:1.0:1789760779.297307:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512735360 00000100:00100000:1.0:1789760779.297365:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 758799955003 time 758831657547 state 0x0 ref 1 00000100:00100000:1.0:1789760779.297380:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 758831917963 time 758831657547 state 0x1 ref 1 00000100:00100000:2.0:1789760779.297674:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 758831917963 time 758861970852 state 0x0 ref 1 00000100:00100000:2.0:1789760779.297690:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5304700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512735360:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760779.297865:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27906048, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0xd807 00010000:00010000:2.0:1789760779.297875:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2fb75 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.297891:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=758s v=(0 0) 00010000:00010000:2.0:1789760779.297896:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760779.297902:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2fb75 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27906048->27910143] (req 27906048->27910143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.297917:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2fb75 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27906048->27910143] (req 27906048->27910143) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760779.328075:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27906048 -> 27910144 00010000:00010000:2.0:1789760779.328096:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2fb75 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27906048->27910143] (req 27906048->27910143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.328108:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2fb75 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27906048->27910143] (req 27906048->27910143) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.328118:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2fb75 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27906048->27910143] (req 27906048->27910143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.328127:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760779.328132:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2fb75 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27906048->27910143] (req 27906048->27910143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.328140:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2fb75 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27906048->27910143] (req 27906048->27910143) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.328160:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2fb75 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27906048->27910143] (req 27906048->27910143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.328165:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2fb75 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27906048->27910143] (req 27906048->27910143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760779.328239:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5304700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2361:x1876699512735360:12345-192.168.203.56@tcp:4:dd.0 Request processed in 30548us (31737us total) trans 4294974132 rc 0/0 00000100:00100000:2.0:1789760779.328249:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6843 00000100:00100000:2.0:1789760779.328258:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 758831917963 time 758861970852 state 0x2 ref 0 00000001:00080000:0.0:1789760779.328334:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974132 is committed 00000100:00100000:0.0:1789760779.330663:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760779.330736:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512735488 00000100:00100000:3.0:1789760779.330747:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512735488 00000100:00100000:3.0:1789760779.330810:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 758831917963 time 758861970852 state 0x0 ref 1 00000100:00100000:3.0:1789760779.330827:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 758862231268 time 758861970852 state 0x1 ref 1 00000100:00100000:3.0:1789760779.330864:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 758862231268 time 758895162781 state 0x0 ref 1 00000100:00100000:3.0:1789760779.330876:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5304380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512735488:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760779.330923:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27910144, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0xd807 00010000:00010000:3.0:1789760779.330932:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2fb7c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.330948:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=758s v=(0 0) 00010000:00010000:3.0:1789760779.330953:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760779.330959:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2fb7c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27910144->27914239] (req 27910144->27914239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.330974:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2fb7c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27910144->27914239] (req 27910144->27914239) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760779.362901:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974133 is committed 00002000:00010000:0.0:1789760779.363155:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27910144 -> 27914240 00010000:00010000:0.0:1789760779.363171:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2fb7c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27910144->27914239] (req 27910144->27914239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.363183:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2fb7c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27910144->27914239] (req 27910144->27914239) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.363198:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2fb7c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27910144->27914239] (req 27910144->27914239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.363206:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760779.363211:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2fb7c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27910144->27914239] (req 27910144->27914239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.363220:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2fb7c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27910144->27914239] (req 27910144->27914239) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.363239:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2fb7c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27910144->27914239] (req 27910144->27914239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.363247:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2fb7c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27910144->27914239] (req 27910144->27914239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760779.363338:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5304380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512735488:12345-192.168.203.56@tcp:4:dd.0 Request processed in 32460us (32677us total) trans 4294974133 rc 0/0 00000100:00100000:0.0:1789760779.363348:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6844 00000100:00100000:0.0:1789760779.363357:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 758862231268 time 758895162781 state 0x2 ref 0 00000100:00100000:3.0:1789760779.368625:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760779.368947:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512735616 00000100:00100000:0.0:1789760779.368960:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512735616 00000100:00100000:0.0:1789760779.369016:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 758862231268 time 758895162781 state 0x0 ref 1 00000100:00100000:0.0:1789760779.369033:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 758895423197 time 758895162781 state 0x1 ref 1 00000100:00100000:0.0:1789760779.369065:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 758895423197 time 758933365290 state 0x0 ref 1 00000100:00100000:0.0:1789760779.369078:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac531f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512735616:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760779.369124:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27914240, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0xd807 00010000:00010000:0.0:1789760779.369133:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2fb83 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.369158:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=758s v=(0 0) 00010000:00010000:0.0:1789760779.369162:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760779.369168:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2fb83 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27914240->27918335] (req 27914240->27918335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.369182:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2fb83 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27914240->27918335] (req 27914240->27918335) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760779.420754:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27914240 -> 27918336 00010000:00010000:1.0:1789760779.420773:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2fb83 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27914240->27918335] (req 27914240->27918335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.420786:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2fb83 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27914240->27918335] (req 27914240->27918335) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.420795:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2fb83 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27914240->27918335] (req 27914240->27918335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.420803:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760779.420807:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2fb83 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27914240->27918335] (req 27914240->27918335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.420816:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2fb83 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27914240->27918335] (req 27914240->27918335) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.420835:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2fb83 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27914240->27918335] (req 27914240->27918335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.420844:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2fb83 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27914240->27918335] (req 27914240->27918335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760779.420931:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac531f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2361:x1876699512735616:12345-192.168.203.56@tcp:4:dd.0 Request processed in 51851us (52312us total) trans 4294974134 rc 0/0 00000100:00100000:1.0:1789760779.420943:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6845 00000100:00100000:1.0:1789760779.420953:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 758895423197 time 758933365290 state 0x2 ref 0 00000001:00080000:3.0:1789760779.421507:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974134 is committed 00000100:00100000:3.0:1789760779.424209:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760779.424399:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512735744 00000100:00100000:2.0:1789760779.424409:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512735744 00000100:00100000:2.0:1789760779.424461:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 758895423197 time 758933365290 state 0x0 ref 1 00000100:00100000:2.0:1789760779.424476:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 758933625706 time 758933365290 state 0x1 ref 1 00000100:00100000:2.0:1789760779.424617:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 758933625706 time 758988914933 state 0x0 ref 1 00000100:00100000:2.0:1789760779.424634:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac531df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512735744:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760779.424684:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27918336, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0xd807 00010000:00010000:2.0:1789760779.424724:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2fb8a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.424748:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=758s v=(0 0) 00010000:00010000:2.0:1789760779.424752:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760779.424759:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2fb8a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27918336->27922431] (req 27918336->27922431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.424774:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2fb8a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27918336->27922431] (req 27918336->27922431) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760779.475426:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974135 is committed 00002000:00010000:1.0:1789760779.475529:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27918336 -> 27922432 00010000:00010000:1.0:1789760779.475545:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2fb8a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27918336->27922431] (req 27918336->27922431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.475558:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2fb8a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27918336->27922431] (req 27918336->27922431) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.475567:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2fb8a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27918336->27922431] (req 27918336->27922431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.475575:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760779.475579:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2fb8a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27918336->27922431] (req 27918336->27922431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.475588:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2fb8a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27918336->27922431] (req 27918336->27922431) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.475609:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2fb8a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27918336->27922431] (req 27918336->27922431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.475616:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2fb8a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27918336->27922431] (req 27918336->27922431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760779.475746:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac531df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512735744:12345-192.168.203.56@tcp:4:dd.0 Request processed in 51109us (51540us total) trans 4294974135 rc 0/0 00000100:00100000:1.0:1789760779.475757:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6846 00000100:00100000:1.0:1789760779.475765:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 758933625706 time 758988914933 state 0x2 ref 0 00000100:00100000:0.0:1789760779.482406:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760779.482479:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512735872 00000100:00100000:3.0:1789760779.482489:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512735872 00000100:00100000:3.0:1789760779.482541:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 758933625706 time 758988914933 state 0x0 ref 1 00000100:00100000:3.0:1789760779.482558:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 758989175349 time 758988914933 state 0x1 ref 1 00000100:00100000:3.0:1789760779.482587:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 758989175349 time 759046886086 state 0x0 ref 1 00000100:00100000:3.0:1789760779.482604:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5306680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512735872:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760779.482651:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27922432, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0xd807 00010000:00010000:3.0:1789760779.482661:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2fb91 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.482676:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=759s v=(0 0) 00010000:00010000:3.0:1789760779.482681:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760779.482687:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2fb91 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27922432->27926527] (req 27922432->27926527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.482738:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2fb91 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27922432->27926527] (req 27922432->27926527) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760779.528129:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974136 is committed 00002000:00010000:1.0:1789760779.528229:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27922432 -> 27926528 00010000:00010000:1.0:1789760779.528243:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2fb91 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27922432->27926527] (req 27922432->27926527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.528256:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2fb91 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27922432->27926527] (req 27922432->27926527) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.528265:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2fb91 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27922432->27926527] (req 27922432->27926527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.528272:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760779.528277:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2fb91 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27922432->27926527] (req 27922432->27926527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.528286:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2fb91 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27922432->27926527] (req 27922432->27926527) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.528305:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2fb91 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27922432->27926527] (req 27922432->27926527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.528318:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2fb91 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27922432->27926527] (req 27922432->27926527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760779.528425:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5306680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512735872:12345-192.168.203.56@tcp:4:dd.0 Request processed in 45820us (46023us total) trans 4294974136 rc 0/0 00000100:00100000:1.0:1789760779.528434:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6847 00000100:00100000:1.0:1789760779.528445:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 758989175349 time 759046886086 state 0x2 ref 0 00000100:00100000:3.0:1789760779.536201:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760779.536309:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512736000 00000100:00100000:1.0:1789760779.536318:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512736000 00000100:00100000:1.0:1789760779.536371:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 758989175349 time 759046886086 state 0x0 ref 1 00000100:00100000:1.0:1789760779.536388:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 759047146502 time 759046886086 state 0x1 ref 1 00000100:00100000:1.0:1789760779.536420:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 759047146502 time 759100718872 state 0x0 ref 1 00000100:00100000:1.0:1789760779.536433:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac531c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512736000:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760779.536480:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27926528, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0xd807 00010000:00010000:1.0:1789760779.536489:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2fb98 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.536505:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=759s v=(0 0) 00010000:00010000:1.0:1789760779.536509:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760779.536516:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2fb98 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27926528->27930623] (req 27926528->27930623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.536536:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2fb98 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27926528->27930623] (req 27926528->27930623) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760779.585995:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974137 is committed 00002000:00010000:0.0:1789760779.586047:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27926528 -> 27930624 00002000:00010000:0.0:1789760779.586063:0:6708:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760778 -> 1789760779 00002000:00010000:0.0:1789760779.586066:0:6708:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760778 -> 1789760779 00010000:00010000:0.0:1789760779.586080:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2fb98 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27926528->27930623] (req 27926528->27930623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.586093:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2fb98 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27926528->27930623] (req 27926528->27930623) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.586103:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2fb98 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27926528->27930623] (req 27926528->27930623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.586120:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760779.586129:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2fb98 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27926528->27930623] (req 27926528->27930623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.586139:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2fb98 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27926528->27930623] (req 27926528->27930623) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.586159:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2fb98 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27926528->27930623] (req 27926528->27930623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.586166:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2fb98 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27926528->27930623] (req 27926528->27930623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760779.586380:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac531c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512736000:12345-192.168.203.56@tcp:4:dd.0 Request processed in 49937us (50177us total) trans 4294974137 rc 0/0 00000100:00100000:0.0:1789760779.586391:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6848 00000100:00100000:0.0:1789760779.586407:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 759047146502 time 759100718872 state 0x2 ref 0 00000100:00100000:0.0:1789760779.598304:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760779.598623:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512736128 00000100:00100000:1.0:1789760779.598633:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512736128 00000100:00100000:1.0:1789760779.598836:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 759047146502 time 759100718872 state 0x0 ref 1 00000100:00100000:1.0:1789760779.598862:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 759100979288 time 759100718872 state 0x1 ref 1 00000100:00100000:1.0:1789760779.598900:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 759100979288 time 759163200372 state 0x0 ref 1 00000100:00100000:1.0:1789760779.598919:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5307100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512736128:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760779.598979:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27930624, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0xd807 00010000:00010000:1.0:1789760779.598994:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2fb9f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.599013:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=759s v=(0 0) 00010000:00010000:1.0:1789760779.599020:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760779.599027:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2fb9f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27930624->27934719] (req 27930624->27934719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.599045:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2fb9f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27930624->27934719] (req 27930624->27934719) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.635569:0:5835:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:0.0:1789760779.645075:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974138 is committed 00002000:00010000:2.0:1789760779.645191:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27930624 -> 27934720 00010000:00010000:2.0:1789760779.645208:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2fb9f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27930624->27934719] (req 27930624->27934719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.645221:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2fb9f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27930624->27934719] (req 27930624->27934719) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.645230:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2fb9f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27930624->27934719] (req 27930624->27934719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.645238:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760779.645243:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2fb9f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27930624->27934719] (req 27930624->27934719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.645252:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2fb9f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27930624->27934719] (req 27930624->27934719) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.645274:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2fb9f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27930624->27934719] (req 27930624->27934719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.645282:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2fb9f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27930624->27934719] (req 27930624->27934719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760779.645414:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5307100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512736128:12345-192.168.203.56@tcp:4:dd.0 Request processed in 46494us (47121us total) trans 4294974138 rc 0/0 00000100:00100000:2.0:1789760779.645424:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6849 00000100:00100000:2.0:1789760779.645433:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 759100979288 time 759163200372 state 0x2 ref 0 00000100:00100000:2.0:1789760779.648437:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760779.648581:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512736256 00000100:00100000:1.0:1789760779.648591:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512736256 00000100:00100000:1.0:1789760779.648639:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 759100979288 time 759163200372 state 0x0 ref 1 00000100:00100000:1.0:1789760779.648655:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 759163460788 time 759163200372 state 0x1 ref 1 00000100:00100000:1.0:1789760779.648758:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 759163460788 time 759213056438 state 0x0 ref 1 00000100:00100000:1.0:1789760779.648772:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac403a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512736256:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760779.648820:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27934720, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0xd807 00010000:00010000:1.0:1789760779.648829:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2fba6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.648843:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=759s v=(0 0) 00010000:00010000:1.0:1789760779.648848:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760779.648854:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2fba6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27934720->27938815] (req 27934720->27938815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.648871:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2fba6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27934720->27938815] (req 27934720->27938815) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760779.675444:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974139 is committed 00002000:00010000:1.0:1789760779.675596:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27934720 -> 27938816 00010000:00010000:1.0:1789760779.675611:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2fba6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27934720->27938815] (req 27934720->27938815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.675629:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2fba6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27934720->27938815] (req 27934720->27938815) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.675638:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2fba6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27934720->27938815] (req 27934720->27938815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.675646:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760779.675650:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2fba6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27934720->27938815] (req 27934720->27938815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.675659:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2fba6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27934720->27938815] (req 27934720->27938815) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.675676:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2fba6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27934720->27938815] (req 27934720->27938815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.675684:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2fba6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27934720->27938815] (req 27934720->27938815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760779.675844:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac403a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512736256:12345-192.168.203.56@tcp:4:dd.0 Request processed in 27068us (27412us total) trans 4294974139 rc 0/0 00000100:00100000:1.0:1789760779.675854:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6850 00000100:00100000:1.0:1789760779.675863:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 759163460788 time 759213056438 state 0x2 ref 0 00000100:00100000:2.0:1789760779.678445:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760779.678538:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512736384 00000100:00100000:0.0:1789760779.678549:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512736384 00000100:00100000:0.0:1789760779.678603:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 759163460788 time 759213056438 state 0x0 ref 1 00000100:00100000:0.0:1789760779.678623:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 759213316854 time 759213056438 state 0x1 ref 1 00000100:00100000:0.0:1789760779.678656:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 759213316854 time 759242955361 state 0x0 ref 1 00000100:00100000:0.0:1789760779.678668:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4038700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512736384:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760779.678756:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27938816, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0xd807 00010000:00010000:0.0:1789760779.678766:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2fbad lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.678782:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=759s v=(0 0) 00010000:00010000:0.0:1789760779.678786:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760779.678793:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2fbad lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27938816->27942911] (req 27938816->27942911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.678808:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2fbad lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27938816->27942911] (req 27938816->27942911) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760779.718661:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974140 is committed 00002000:00010000:0.0:1789760779.718944:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27938816 -> 27942912 00010000:00010000:0.0:1789760779.718964:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2fbad lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27938816->27942911] (req 27938816->27942911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.718978:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2fbad lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27938816->27942911] (req 27938816->27942911) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.718987:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2fbad lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27938816->27942911] (req 27938816->27942911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.718999:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760779.719004:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2fbad lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27938816->27942911] (req 27938816->27942911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.719012:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2fbad lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27938816->27942911] (req 27938816->27942911) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.719030:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2fbad lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27938816->27942911] (req 27938816->27942911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.719037:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2fbad lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27938816->27942911] (req 27938816->27942911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760779.719227:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4038700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512736384:12345-192.168.203.56@tcp:4:dd.0 Request processed in 40555us (40786us total) trans 4294974140 rc 0/0 00000100:00100000:0.0:1789760779.719238:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6851 00000100:00100000:0.0:1789760779.719247:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 759213316854 time 759242955361 state 0x2 ref 0 00000100:00100000:0.0:1789760779.721994:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760779.722222:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512736512 00000100:00100000:2.0:1789760779.722234:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512736512 00000100:00100000:2.0:1789760779.722288:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 759213316854 time 759242955361 state 0x0 ref 1 00000100:00100000:2.0:1789760779.722305:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 759243215777 time 759242955361 state 0x1 ref 1 00000100:00100000:2.0:1789760779.722370:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 759243215777 time 759286668656 state 0x0 ref 1 00000100:00100000:2.0:1789760779.722382:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5304a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512736512:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760779.722428:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27942912, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0xd807 00010000:00010000:2.0:1789760779.722436:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2fbb4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.722450:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=759s v=(0 0) 00010000:00010000:2.0:1789760779.722454:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760779.722460:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2fbb4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27942912->27947007] (req 27942912->27947007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.722473:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2fbb4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27942912->27947007] (req 27942912->27947007) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760779.751613:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974141 is committed 00002000:00010000:1.0:1789760779.751772:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27942912 -> 27947008 00010000:00010000:1.0:1789760779.751788:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2fbb4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27942912->27947007] (req 27942912->27947007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.751801:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2fbb4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27942912->27947007] (req 27942912->27947007) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.751809:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2fbb4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27942912->27947007] (req 27942912->27947007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.751816:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760779.751821:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2fbb4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27942912->27947007] (req 27942912->27947007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.751829:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2fbb4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27942912->27947007] (req 27942912->27947007) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.751849:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2fbb4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27942912->27947007] (req 27942912->27947007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.751857:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2fbb4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27942912->27947007] (req 27942912->27947007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760779.751982:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5304a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512736512:12345-192.168.203.56@tcp:4:dd.0 Request processed in 29596us (29991us total) trans 4294974141 rc 0/0 00000100:00100000:1.0:1789760779.751991:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6852 00000100:00100000:1.0:1789760779.752000:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 759243215777 time 759286668656 state 0x2 ref 0 00000100:00100000:2.0:1789760779.755639:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760779.755759:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512736640 00000100:00100000:1.0:1789760779.755768:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512736640 00000100:00100000:1.0:1789760779.755819:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 759243215777 time 759286668656 state 0x0 ref 1 00000100:00100000:1.0:1789760779.755833:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 759286929072 time 759286668656 state 0x1 ref 1 00000100:00100000:1.0:1789760779.755856:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 759286929072 time 759320156291 state 0x0 ref 1 00000100:00100000:1.0:1789760779.755869:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4039f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512736640:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760779.755921:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27947008, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0xd807 00010000:00010000:1.0:1789760779.755930:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2fbbb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.755946:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=759s v=(0 0) 00010000:00010000:1.0:1789760779.755950:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760779.755957:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2fbbb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27947008->27951103] (req 27947008->27951103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.755971:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2fbbb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27947008->27951103] (req 27947008->27951103) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760779.788854:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974142 is committed 00002000:00010000:1.0:1789760779.789184:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27947008 -> 27951104 00010000:00010000:1.0:1789760779.789201:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2fbbb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27947008->27951103] (req 27947008->27951103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.789215:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2fbbb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27947008->27951103] (req 27947008->27951103) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.789226:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2fbbb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27947008->27951103] (req 27947008->27951103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.789234:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760779.789240:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2fbbb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27947008->27951103] (req 27947008->27951103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.789250:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2fbbb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27947008->27951103] (req 27947008->27951103) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.789269:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2fbbb lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27947008->27951103] (req 27947008->27951103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.789277:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2fbbb lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27947008->27951103] (req 27947008->27951103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760779.789376:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4039f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512736640:12345-192.168.203.56@tcp:4:dd.0 Request processed in 33504us (33739us total) trans 4294974142 rc 0/0 00000100:00100000:1.0:1789760779.789386:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6853 00000100:00100000:1.0:1789760779.789396:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 759286929072 time 759320156291 state 0x2 ref 0 00000100:00100000:2.0:1789760779.792326:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760779.792411:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512736768 00000100:00100000:1.0:1789760779.792421:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512736768 00000100:00100000:1.0:1789760779.792480:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 759286929072 time 759320156291 state 0x0 ref 1 00000100:00100000:1.0:1789760779.792496:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 759320416707 time 759320156291 state 0x1 ref 1 00000100:00100000:1.0:1789760779.792529:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 759320416707 time 759356828124 state 0x0 ref 1 00000100:00100000:1.0:1789760779.792541:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4038000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512736768:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760779.792594:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27951104, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0xd807 00010000:00010000:1.0:1789760779.792602:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2fbc2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.792618:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=759s v=(0 0) 00010000:00010000:1.0:1789760779.792622:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760779.792629:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2fbc2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27951104->27955199] (req 27951104->27955199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.792642:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2fbc2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27951104->27955199] (req 27951104->27955199) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760779.840365:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974143 is committed 00002000:00010000:3.0:1789760779.840475:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27951104 -> 27955200 00010000:00010000:3.0:1789760779.840491:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2fbc2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27951104->27955199] (req 27951104->27955199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.840503:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2fbc2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27951104->27955199] (req 27951104->27955199) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.840512:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2fbc2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27951104->27955199] (req 27951104->27955199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.840519:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760779.840524:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2fbc2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27951104->27955199] (req 27951104->27955199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.840532:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2fbc2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27951104->27955199] (req 27951104->27955199) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.840548:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2fbc2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27951104->27955199] (req 27951104->27955199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.840555:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2fbc2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27951104->27955199] (req 27951104->27955199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760779.840634:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4038000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512736768:12345-192.168.203.56@tcp:4:dd.0 Request processed in 48091us (48314us total) trans 4294974143 rc 0/0 00000100:00100000:3.0:1789760779.840644:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6854 00000100:00100000:3.0:1789760779.840653:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 759320416707 time 759356828124 state 0x2 ref 0 00000100:00100000:3.0:1789760779.843511:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760779.843623:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512736896 00000100:00100000:1.0:1789760779.843634:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512736896 00000100:00100000:1.0:1789760779.843690:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 759320416707 time 759356828124 state 0x0 ref 1 00000100:00100000:1.0:1789760779.843746:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 759357088540 time 759356828124 state 0x1 ref 1 00000100:00100000:1.0:1789760779.843766:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 759357088540 time 759408066398 state 0x0 ref 1 00000100:00100000:1.0:1789760779.843778:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac531d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512736896:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760779.843839:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27955200, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0xd807 00010000:00010000:1.0:1789760779.843848:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2fbc9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.843864:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=759s v=(0 0) 00010000:00010000:1.0:1789760779.843868:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760779.843875:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2fbc9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27955200->27959295] (req 27955200->27959295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.843889:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2fbc9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27955200->27959295] (req 27955200->27959295) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760779.888449:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974144 is committed 00002000:00010000:1.0:1789760779.888582:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27955200 -> 27959296 00010000:00010000:1.0:1789760779.888596:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2fbc9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27955200->27959295] (req 27955200->27959295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.888609:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2fbc9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27955200->27959295] (req 27955200->27959295) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.888618:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2fbc9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27955200->27959295] (req 27955200->27959295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.888625:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760779.888630:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2fbc9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27955200->27959295] (req 27955200->27959295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.888639:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2fbc9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27955200->27959295] (req 27955200->27959295) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.888659:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2fbc9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27955200->27959295] (req 27955200->27959295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.888667:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2fbc9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27955200->27959295] (req 27955200->27959295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760779.888891:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac531d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512736896:12345-192.168.203.56@tcp:4:dd.0 Request processed in 45110us (45385us total) trans 4294974144 rc 0/0 00000100:00100000:1.0:1789760779.888901:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6855 00000100:00100000:1.0:1789760779.888911:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 759357088540 time 759408066398 state 0x2 ref 0 00000100:00100000:0.0:1789760779.897091:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760779.897146:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512737024 00000100:00100000:1.0:1789760779.897156:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512737024 00000100:00100000:1.0:1789760779.897210:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 759357088540 time 759408066398 state 0x0 ref 1 00000100:00100000:1.0:1789760779.898148:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 759408326814 time 759408066398 state 0x1 ref 1 00000100:00100000:1.0:1789760779.898243:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 759408326814 time 759462539777 state 0x0 ref 1 00000100:00100000:1.0:1789760779.898258:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5305880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512737024:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760779.898308:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27959296, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0xd807 00010000:00010000:1.0:1789760779.898317:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2fbd0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.898332:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=759s v=(0 0) 00010000:00010000:1.0:1789760779.898338:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760779.898345:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2fbd0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27959296->27963391] (req 27959296->27963391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.898358:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2fbd0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27959296->27963391] (req 27959296->27963391) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760779.954622:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974145 is committed 00002000:00010000:1.0:1789760779.954741:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27959296 -> 27963392 00010000:00010000:1.0:1789760779.954758:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2fbd0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27959296->27963391] (req 27959296->27963391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.954788:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2fbd0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27959296->27963391] (req 27959296->27963391) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.954798:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2fbd0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27959296->27963391] (req 27959296->27963391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.954805:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760779.954810:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2fbd0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27959296->27963391] (req 27959296->27963391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.954820:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2fbd0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27959296->27963391] (req 27959296->27963391) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.954840:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2fbd0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27959296->27963391] (req 27959296->27963391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.954854:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2fbd0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27959296->27963391] (req 27959296->27963391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760779.954954:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5305880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512737024:12345-192.168.203.56@tcp:4:dd.0 Request processed in 56696us (57868us total) trans 4294974145 rc 0/0 00000100:00100000:1.0:1789760779.954963:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6856 00000100:00100000:1.0:1789760779.954975:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 759408326814 time 759462539777 state 0x2 ref 0 00000100:00100000:3.0:1789760779.958671:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760779.958805:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512737152 00000100:00100000:1.0:1789760779.958815:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512737152 00000100:00100000:1.0:1789760779.958868:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 759408326814 time 759462539777 state 0x0 ref 1 00000100:00100000:1.0:1789760779.958883:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 759462800193 time 759462539777 state 0x1 ref 1 00000100:00100000:1.0:1789760779.958903:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 759462800193 time 759523203536 state 0x0 ref 1 00000100:00100000:1.0:1789760779.958917:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac531e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512737152:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760779.958968:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27963392, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0xd807 00010000:00010000:1.0:1789760779.958976:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2fbd7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.958993:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=759s v=(0 0) 00010000:00010000:1.0:1789760779.958997:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760779.959003:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2fbd7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27963392->27967487] (req 27963392->27967487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.959023:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2fbd7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27963392->27967487] (req 27963392->27967487) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760780.006405:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974146 is committed 00002000:00010000:1.0:1789760780.006482:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27963392 -> 27967488 00010000:00010000:1.0:1789760780.006499:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2fbd7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27963392->27967487] (req 27963392->27967487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.006511:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2fbd7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27963392->27967487] (req 27963392->27967487) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.006521:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2fbd7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27963392->27967487] (req 27963392->27967487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.006529:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760780.006533:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2fbd7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27963392->27967487] (req 27963392->27967487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.006543:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2fbd7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27963392->27967487] (req 27963392->27967487) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.006565:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2fbd7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27963392->27967487] (req 27963392->27967487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.006572:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2fbd7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27963392->27967487] (req 27963392->27967487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760780.006745:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac531e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512737152:12345-192.168.203.56@tcp:4:dd.0 Request processed in 47827us (48076us total) trans 4294974146 rc 0/0 00000100:00100000:1.0:1789760780.006756:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6857 00000100:00100000:1.0:1789760780.007288:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 759462800193 time 759523203536 state 0x2 ref 0 00000100:00100000:0.0:1789760780.011484:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760780.012872:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512737280 00000100:00100000:1.0:1789760780.012885:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512737280 00000100:00100000:1.0:1789760780.012943:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 759462800193 time 759523203536 state 0x0 ref 1 00000100:00100000:1.0:1789760780.012961:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 759523463952 time 759523203536 state 0x1 ref 1 00000100:00100000:1.0:1789760780.013040:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 759523463952 time 759577339023 state 0x0 ref 1 00000100:00100000:1.0:1789760780.013067:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5306300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512737280:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760780.013115:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27967488, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0xd807 00010000:00010000:1.0:1789760780.013125:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2fbde lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.013149:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=759s v=(0 0) 00010000:00010000:1.0:1789760780.013154:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760780.013159:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2fbde lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27967488->27971583] (req 27967488->27971583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.013173:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2fbde lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27967488->27971583] (req 27967488->27971583) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760780.061072:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27967488 -> 27971584 00010000:00010000:1.0:1789760780.061092:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2fbde lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27967488->27971583] (req 27967488->27971583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.061104:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2fbde lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27967488->27971583] (req 27967488->27971583) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.061113:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2fbde lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27967488->27971583] (req 27967488->27971583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.061119:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760780.061124:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2fbde lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27967488->27971583] (req 27967488->27971583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.061133:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2fbde lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27967488->27971583] (req 27967488->27971583) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760780.061269:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974147 is committed 00010000:00010000:1.0:1789760780.061419:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2fbde lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27967488->27971583] (req 27967488->27971583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.061430:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2fbde lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27967488->27971583] (req 27967488->27971583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760780.061553:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5306300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512737280:12345-192.168.203.56@tcp:4:dd.0 Request processed in 48484us (50077us total) trans 4294974147 rc 0/0 00000100:00100000:1.0:1789760780.061567:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6858 00000100:00100000:1.0:1789760780.061576:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 759523463952 time 759577339023 state 0x2 ref 0 00000100:00100000:0.0:1789760780.064124:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760780.064179:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512737408 00000100:00100000:1.0:1789760780.064188:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512737408 00000100:00100000:1.0:1789760780.064246:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 759523463952 time 759577339023 state 0x0 ref 1 00000100:00100000:1.0:1789760780.064262:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 759577599439 time 759577339023 state 0x1 ref 1 00000100:00100000:1.0:1789760780.064292:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 759577599439 time 759628591767 state 0x0 ref 1 00000100:00100000:1.0:1789760780.064305:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5307800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512737408:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760780.064579:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27971584, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0xd807 00010000:00010000:1.0:1789760780.064588:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2fbe5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.064603:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=759s v=(0 0) 00010000:00010000:1.0:1789760780.064607:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760780.064613:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2fbe5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27971584->27975679] (req 27971584->27975679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.064626:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2fbe5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27971584->27975679] (req 27971584->27975679) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760780.100776:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27971584 -> 27975680 00010000:00010000:3.0:1789760780.100794:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2fbe5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27971584->27975679] (req 27971584->27975679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.100807:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2fbe5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27971584->27975679] (req 27971584->27975679) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.100816:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2fbe5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27971584->27975679] (req 27971584->27975679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.100823:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760780.100828:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2fbe5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27971584->27975679] (req 27971584->27975679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.100836:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2fbe5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27971584->27975679] (req 27971584->27975679) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.100866:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2fbe5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27971584->27975679] (req 27971584->27975679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.100875:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2fbe5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27971584->27975679] (req 27971584->27975679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760780.100972:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5307800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2359:x1876699512737408:12345-192.168.203.56@tcp:4:dd.0 Request processed in 36665us (36853us total) trans 4294974148 rc 0/0 00000100:00100000:3.0:1789760780.100982:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6859 00000100:00100000:3.0:1789760780.100992:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 759577599439 time 759628591767 state 0x2 ref 0 00000001:00080000:0.0:1789760780.101217:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974148 is committed 00000100:00100000:3.0:1789760780.104890:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760780.105192:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512737536 00000100:00100000:1.0:1789760780.105202:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512737536 00000100:00100000:1.0:1789760780.105258:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 759577599439 time 759628591767 state 0x0 ref 1 00000100:00100000:1.0:1789760780.105272:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 759628852183 time 759628591767 state 0x1 ref 1 00000100:00100000:1.0:1789760780.105301:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 759628852183 time 759669600796 state 0x0 ref 1 00000100:00100000:1.0:1789760780.105323:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac531ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512737536:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760780.105372:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27975680, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0xd807 00010000:00010000:1.0:1789760780.105381:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2fbec lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.105397:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=759s v=(0 0) 00010000:00010000:1.0:1789760780.105403:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760780.105410:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2fbec lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27975680->27979775] (req 27975680->27979775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.105426:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2fbec lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27975680->27979775] (req 27975680->27979775) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760780.134982:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974149 is committed 00002000:00010000:0.0:1789760780.135043:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27975680 -> 27979776 00010000:00010000:0.0:1789760780.135066:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2fbec lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27975680->27979775] (req 27975680->27979775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.135079:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2fbec lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27975680->27979775] (req 27975680->27979775) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.135088:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2fbec lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27975680->27979775] (req 27975680->27979775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.135095:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760780.135100:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2fbec lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27975680->27979775] (req 27975680->27979775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.135109:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2fbec lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27975680->27979775] (req 27975680->27979775) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.135127:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2fbec lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27975680->27979775] (req 27975680->27979775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.135135:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2fbec lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27975680->27979775] (req 27975680->27979775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760780.135222:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac531ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512737536:12345-192.168.203.56@tcp:4:dd.0 Request processed in 29906us (30337us total) trans 4294974149 rc 0/0 00000100:00100000:0.0:1789760780.135232:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6860 00000100:00100000:0.0:1789760780.135251:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 759628852183 time 759669600796 state 0x2 ref 0 00000100:00100000:1.0:1789760780.139569:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760780.139744:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512737664 00000100:00100000:2.0:1789760780.139753:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512737664 00000100:00100000:2.0:1789760780.139807:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 759628852183 time 759669600796 state 0x0 ref 1 00000100:00100000:2.0:1789760780.139822:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 759669861212 time 759669600796 state 0x1 ref 1 00000100:00100000:2.0:1789760780.139842:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 759669861212 time 759704141880 state 0x0 ref 1 00000100:00100000:2.0:1789760780.139852:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4133800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512737664:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760780.139898:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27979776, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0xd807 00010000:00010000:2.0:1789760780.139911:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2fbf3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.139937:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=759s v=(0 0) 00010000:00010000:2.0:1789760780.139942:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760780.139949:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2fbf3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27979776->27983871] (req 27979776->27983871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.139962:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2fbf3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27979776->27983871] (req 27979776->27983871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760780.191540:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974150 is committed 00002000:00010000:1.0:1789760780.191855:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27979776 -> 27983872 00010000:00010000:1.0:1789760780.191870:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2fbf3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27979776->27983871] (req 27979776->27983871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.191883:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2fbf3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27979776->27983871] (req 27979776->27983871) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.191898:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2fbf3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27979776->27983871] (req 27979776->27983871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.191906:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760780.191910:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2fbf3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27979776->27983871] (req 27979776->27983871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.191919:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2fbf3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27979776->27983871] (req 27979776->27983871) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.191935:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2fbf3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27979776->27983871] (req 27979776->27983871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.191942:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2fbf3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27979776->27983871] (req 27979776->27983871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760780.192032:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4133800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512737664:12345-192.168.203.56@tcp:4:dd.0 Request processed in 52177us (52468us total) trans 4294974150 rc 0/0 00000100:00100000:1.0:1789760780.192041:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6861 00000100:00100000:1.0:1789760780.192060:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 759669861212 time 759704141880 state 0x2 ref 0 00000100:00100000:1.0:1789760780.198659:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760780.198947:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512737792 00000100:00100000:3.0:1789760780.198959:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512737792 00000100:00100000:3.0:1789760780.199011:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 759669861212 time 759704141880 state 0x0 ref 1 00000100:00100000:3.0:1789760780.199027:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 759704402296 time 759704141880 state 0x1 ref 1 00000100:00100000:3.0:1789760780.199069:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 759704402296 time 759763368828 state 0x0 ref 1 00000100:00100000:3.0:1789760780.199081:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4132680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512737792:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760780.199126:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27983872, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0xd807 00010000:00010000:3.0:1789760780.199135:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2fbfa lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.199151:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=759s v=(0 0) 00010000:00010000:3.0:1789760780.199155:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760780.199161:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2fbfa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27983872->27987967] (req 27983872->27987967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.199175:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2fbfa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27983872->27987967] (req 27983872->27987967) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760780.227731:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974151 is committed 00002000:00010000:1.0:1789760780.227837:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27983872 -> 27987968 00010000:00010000:1.0:1789760780.227854:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2fbfa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27983872->27987967] (req 27983872->27987967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.227871:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2fbfa lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27983872->27987967] (req 27983872->27987967) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.227880:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2fbfa lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27983872->27987967] (req 27983872->27987967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.227888:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760780.227893:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2fbfa lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27983872->27987967] (req 27983872->27987967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.227902:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2fbfa lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27983872->27987967] (req 27983872->27987967) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.227922:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2fbfa lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27983872->27987967] (req 27983872->27987967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.227930:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2fbfa lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27983872->27987967] (req 27983872->27987967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760780.228025:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4132680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512737792:12345-192.168.203.56@tcp:4:dd.0 Request processed in 28940us (29371us total) trans 4294974151 rc 0/0 00000100:00100000:1.0:1789760780.228035:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6862 00000100:00100000:1.0:1789760780.228056:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 759704402296 time 759763368828 state 0x2 ref 0 00000100:00100000:0.0:1789760780.235659:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760780.236005:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512737920 00000100:00100000:1.0:1789760780.236017:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512737920 00000100:00100000:1.0:1789760780.236088:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 759704402296 time 759763368828 state 0x0 ref 1 00000100:00100000:1.0:1789760780.236114:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 759763629244 time 759763368828 state 0x1 ref 1 00000100:00100000:1.0:1789760780.236145:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 759763629244 time 759800444508 state 0x0 ref 1 00000100:00100000:1.0:1789760780.236157:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febfb374380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512737920:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760780.236225:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27987968, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0xd807 00010000:00010000:1.0:1789760780.236236:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2fc01 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.236254:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=759s v=(0 0) 00010000:00010000:1.0:1789760780.236259:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760780.236267:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2fc01 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27987968->27992063] (req 27987968->27992063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.236283:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2fc01 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27987968->27992063] (req 27987968->27992063) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760780.268003:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974152 is committed 00002000:00010000:1.0:1789760780.268146:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27987968 -> 27992064 00010000:00010000:1.0:1789760780.268162:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2fc01 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27987968->27992063] (req 27987968->27992063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.268174:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2fc01 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27987968->27992063] (req 27987968->27992063) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.268187:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2fc01 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27987968->27992063] (req 27987968->27992063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.268194:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760780.268199:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2fc01 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27987968->27992063] (req 27987968->27992063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.268208:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2fc01 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27987968->27992063] (req 27987968->27992063) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.268229:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2fc01 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27987968->27992063] (req 27987968->27992063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.268237:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2fc01 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27987968->27992063] (req 27987968->27992063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760780.268334:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febfb374380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512737920:12345-192.168.203.56@tcp:4:dd.0 Request processed in 32173us (32678us total) trans 4294974152 rc 0/0 00000100:00100000:1.0:1789760780.268344:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6863 00000100:00100000:1.0:1789760780.268353:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 759763629244 time 759800444508 state 0x2 ref 0 00000100:00100000:0.0:1789760780.271071:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760780.271159:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512738048 00000100:00100000:2.0:1789760780.271168:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512738048 00000100:00100000:2.0:1789760780.271214:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 759763629244 time 759800444508 state 0x0 ref 1 00000100:00100000:2.0:1789760780.271229:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 759800704924 time 759800444508 state 0x1 ref 1 00000100:00100000:2.0:1789760780.271259:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 759800704924 time 759835559008 state 0x0 ref 1 00000100:00100000:2.0:1789760780.271270:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4059880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512738048:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760780.271317:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27992064, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0xd807 00010000:00010000:2.0:1789760780.271325:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2fc08 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.271343:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=759s v=(0 0) 00010000:00010000:2.0:1789760780.271349:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760780.271355:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2fc08 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27992064->27996159] (req 27992064->27996159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.271368:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2fc08 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27992064->27996159] (req 27992064->27996159) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760780.301314:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974153 is committed 00002000:00010000:1.0:1789760780.301523:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27992064 -> 27996160 00010000:00010000:1.0:1789760780.301540:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2fc08 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27992064->27996159] (req 27992064->27996159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.301567:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2fc08 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27992064->27996159] (req 27992064->27996159) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.301575:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2fc08 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27992064->27996159] (req 27992064->27996159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.301582:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760780.301588:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2fc08 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27992064->27996159] (req 27992064->27996159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.301596:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2fc08 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27992064->27996159] (req 27992064->27996159) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.301615:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2fc08 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27992064->27996159] (req 27992064->27996159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.301622:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c2fc08 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27992064->27996159] (req 27992064->27996159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760780.301745:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4059880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512738048:12345-192.168.203.56@tcp:4:dd.0 Request processed in 30471us (30680us total) trans 4294974153 rc 0/0 00000100:00100000:1.0:1789760780.301753:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6864 00000100:00100000:1.0:1789760780.301760:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 759800704924 time 759835559008 state 0x2 ref 0 00000100:00100000:0.0:1789760780.305064:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760780.305418:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512738176 00000100:00100000:1.0:1789760780.305429:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512738176 00000100:00100000:1.0:1789760780.305482:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 759800704924 time 759835559008 state 0x0 ref 1 00000100:00100000:1.0:1789760780.305496:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 759835819424 time 759835559008 state 0x1 ref 1 00000100:00100000:1.0:1789760780.305573:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 759835819424 time 759869872565 state 0x0 ref 1 00000100:00100000:1.0:1789760780.305586:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac405bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512738176:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760780.305635:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27996160, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0xd807 00010000:00010000:1.0:1789760780.305644:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2fc0f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.305661:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=759s v=(0 0) 00010000:00010000:1.0:1789760780.305664:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760780.305671:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2fc0f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27996160->28000255] (req 27996160->28000255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.305685:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2fc0f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27996160->28000255] (req 27996160->28000255) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760780.332418:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974154 is committed 00002000:00010000:1.0:1789760780.332514:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27996160 -> 28000256 00010000:00010000:1.0:1789760780.332529:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2fc0f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27996160->28000255] (req 27996160->28000255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.332543:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2fc0f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27996160->28000255] (req 27996160->28000255) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.332560:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2fc0f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27996160->28000255] (req 27996160->28000255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.332568:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760780.332573:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2fc0f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27996160->28000255] (req 27996160->28000255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.332582:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2fc0f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27996160->28000255] (req 27996160->28000255) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.332602:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2fc0f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27996160->28000255] (req 27996160->28000255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.332611:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2fc0f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27996160->28000255] (req 27996160->28000255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760780.332817:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac405bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512738176:12345-192.168.203.56@tcp:4:dd.0 Request processed in 27108us (27653us total) trans 4294974154 rc 0/0 00000100:00100000:1.0:1789760780.332829:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6865 00000100:00100000:1.0:1789760780.332838:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 759835819424 time 759869872565 state 0x2 ref 0 00000100:00100000:0.0:1789760780.338632:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760780.338784:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512738304 00000100:00100000:1.0:1789760780.338795:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512738304 00000100:00100000:1.0:1789760780.338847:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 759835819424 time 759869872565 state 0x0 ref 1 00000100:00100000:1.0:1789760780.338863:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 759870132981 time 759869872565 state 0x1 ref 1 00000100:00100000:1.0:1789760780.338916:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 759870132981 time 759903215221 state 0x0 ref 1 00000100:00100000:1.0:1789760780.338933:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febea489450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512738304:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760780.338985:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28000256, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0xd807 00010000:00010000:1.0:1789760780.338995:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2fc16 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.339011:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=759s v=(0 0) 00010000:00010000:1.0:1789760780.339015:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760780.339022:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2fc16 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28000256->28004351] (req 28000256->28004351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.339037:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2fc16 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28000256->28004351] (req 28000256->28004351) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760780.365602:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974155 is committed 00002000:00010000:1.0:1789760780.365691:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28000256 -> 28004352 00010000:00010000:1.0:1789760780.365728:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2fc16 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28000256->28004351] (req 28000256->28004351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.365748:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2fc16 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28000256->28004351] (req 28000256->28004351) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.365757:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2fc16 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28000256->28004351] (req 28000256->28004351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.365765:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760780.365770:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2fc16 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28000256->28004351] (req 28000256->28004351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.365778:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2fc16 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28000256->28004351] (req 28000256->28004351) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.365796:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2fc16 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28000256->28004351] (req 28000256->28004351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.365803:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2fc16 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28000256->28004351] (req 28000256->28004351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760780.365889:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febea489450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512738304:12345-192.168.203.56@tcp:4:dd.0 Request processed in 26953us (27262us total) trans 4294974155 rc 0/0 00000100:00100000:1.0:1789760780.365898:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6866 00000100:00100000:1.0:1789760780.365907:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 759870132981 time 759903215221 state 0x2 ref 0 00000100:00100000:0.0:1789760780.368656:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760780.368755:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512738432 00000100:00100000:1.0:1789760780.368764:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512738432 00000100:00100000:1.0:1789760780.368813:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 759870132981 time 759903215221 state 0x0 ref 1 00000100:00100000:1.0:1789760780.368827:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 759903475637 time 759903215221 state 0x1 ref 1 00000100:00100000:1.0:1789760780.368844:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 759903475637 time 759933144583 state 0x0 ref 1 00000100:00100000:1.0:1789760780.368855:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4058380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512738432:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760780.368899:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28004352, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0xd807 00010000:00010000:1.0:1789760780.368907:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2fc1d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.368922:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=759s v=(0 0) 00010000:00010000:1.0:1789760780.368926:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760780.368933:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2fc1d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28004352->28008447] (req 28004352->28008447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.368945:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2fc1d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28004352->28008447] (req 28004352->28008447) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760780.399633:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974156 is committed 00002000:00010000:1.0:1789760780.399813:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28004352 -> 28008448 00010000:00010000:1.0:1789760780.399831:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2fc1d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28004352->28008447] (req 28004352->28008447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.399844:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2fc1d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28004352->28008447] (req 28004352->28008447) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.399853:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2fc1d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28004352->28008447] (req 28004352->28008447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.399860:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760780.399865:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2fc1d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28004352->28008447] (req 28004352->28008447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.399874:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2fc1d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28004352->28008447] (req 28004352->28008447) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.399898:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2fc1d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28004352->28008447] (req 28004352->28008447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.399907:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2fc1d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28004352->28008447] (req 28004352->28008447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760780.400041:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4058380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512738432:12345-192.168.203.56@tcp:4:dd.0 Request processed in 31182us (31387us total) trans 4294974156 rc 0/0 00000100:00100000:1.0:1789760780.400060:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6867 00000100:00100000:1.0:1789760780.400070:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 759903475637 time 759933144583 state 0x2 ref 0 00000100:00100000:0.0:1789760780.403567:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760780.403641:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512738560 00000100:00100000:1.0:1789760780.403652:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512738560 00000100:00100000:1.0:1789760780.403728:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 759903475637 time 759933144583 state 0x0 ref 1 00000100:00100000:1.0:1789760780.403744:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 759933404999 time 759933144583 state 0x1 ref 1 00000100:00100000:1.0:1789760780.403764:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 759933404999 time 759968063920 state 0x0 ref 1 00000100:00100000:1.0:1789760780.403776:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4058a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512738560:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760780.403823:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28008448, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0xd807 00010000:00010000:1.0:1789760780.403832:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2fc24 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.403847:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=759s v=(0 0) 00010000:00010000:1.0:1789760780.403852:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760780.403858:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2fc24 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28008448->28012543] (req 28008448->28012543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.403872:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2fc24 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28008448->28012543] (req 28008448->28012543) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760780.436806:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28008448 -> 28012544 00010000:00010000:3.0:1789760780.436835:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2fc24 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28008448->28012543] (req 28008448->28012543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.436847:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2fc24 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28008448->28012543] (req 28008448->28012543) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.436855:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2fc24 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28008448->28012543] (req 28008448->28012543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.436862:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760780.436867:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2fc24 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28008448->28012543] (req 28008448->28012543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.436876:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2fc24 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28008448->28012543] (req 28008448->28012543) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.436898:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2fc24 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28008448->28012543] (req 28008448->28012543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.436907:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c2fc24 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28008448->28012543] (req 28008448->28012543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760780.436999:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4058a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2359:x1876699512738560:12345-192.168.203.56@tcp:4:dd.0 Request processed in 33221us (33438us total) trans 4294974157 rc 0/0 00000100:00100000:3.0:1789760780.437010:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6868 00000100:00100000:3.0:1789760780.437019:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 759933404999 time 759968063920 state 0x2 ref 0 00000001:00080000:0.0:1789760780.437236:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974157 is committed 00000100:00100000:2.0:1789760780.446333:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760780.446519:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512738688 00000100:00100000:1.0:1789760780.446529:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512738688 00000100:00100000:1.0:1789760780.446581:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 759933404999 time 759968063920 state 0x0 ref 1 00000100:00100000:1.0:1789760780.446596:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 759968324336 time 759968063920 state 0x1 ref 1 00000100:00100000:1.0:1789760780.446718:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 759968324336 time 760010993274 state 0x0 ref 1 00000100:00100000:1.0:1789760780.446732:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac403bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512738688:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760780.446780:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28012544, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0xd807 00010000:00010000:1.0:1789760780.446790:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2fc2b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.446807:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=760s v=(0 0) 00010000:00010000:1.0:1789760780.446812:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760780.446818:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2fc2b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28012544->28016639] (req 28012544->28016639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.446833:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2fc2b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28012544->28016639] (req 28012544->28016639) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760780.474582:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974158 is committed 00002000:00010000:0.0:1789760780.474942:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28012544 -> 28016640 00010000:00010000:0.0:1789760780.474959:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2fc2b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28012544->28016639] (req 28012544->28016639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.474972:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2fc2b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28012544->28016639] (req 28012544->28016639) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.474981:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2fc2b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28012544->28016639] (req 28012544->28016639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.474988:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760780.474993:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2fc2b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28012544->28016639] (req 28012544->28016639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.475002:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2fc2b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28012544->28016639] (req 28012544->28016639) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.475026:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2fc2b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28012544->28016639] (req 28012544->28016639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.475034:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2fc2b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28012544->28016639] (req 28012544->28016639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760780.475183:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac403bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512738688:12345-192.168.203.56@tcp:4:dd.0 Request processed in 28448us (28853us total) trans 4294974158 rc 0/0 00000100:00100000:0.0:1789760780.475194:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6869 00000100:00100000:0.0:1789760780.475204:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 759968324336 time 760010993274 state 0x2 ref 0 00000100:00100000:1.0:1789760780.479031:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760780.479139:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512738816 00000100:00100000:2.0:1789760780.479149:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512738816 00000100:00100000:2.0:1789760780.479202:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 759968324336 time 760010993274 state 0x0 ref 1 00000100:00100000:2.0:1789760780.479219:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 760011253690 time 760010993274 state 0x1 ref 1 00000100:00100000:2.0:1789760780.479278:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 760011253690 time 760043576957 state 0x0 ref 1 00000100:00100000:2.0:1789760780.479291:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4130000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512738816:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760780.479337:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28016640, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0xd807 00010000:00010000:2.0:1789760780.479346:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2fc32 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.479361:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=760s v=(0 0) 00010000:00010000:2.0:1789760780.479365:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760780.479370:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2fc32 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28016640->28020735] (req 28016640->28020735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.479383:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2fc32 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28016640->28020735] (req 28016640->28020735) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760780.510152:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28016640 -> 28020736 00010000:00010000:1.0:1789760780.510172:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2fc32 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28016640->28020735] (req 28016640->28020735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.510183:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2fc32 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28016640->28020735] (req 28016640->28020735) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.510192:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2fc32 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28016640->28020735] (req 28016640->28020735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.510199:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760780.510203:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2fc32 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28016640->28020735] (req 28016640->28020735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.510212:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2fc32 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28016640->28020735] (req 28016640->28020735) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.510232:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2fc32 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28016640->28020735] (req 28016640->28020735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.510240:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2fc32 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28016640->28020735] (req 28016640->28020735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760780.510341:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4130000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2361:x1876699512738816:12345-192.168.203.56@tcp:4:dd.0 Request processed in 31047us (31315us total) trans 4294974159 rc 0/0 00000100:00100000:1.0:1789760780.510353:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6870 00000100:00100000:1.0:1789760780.510361:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 760011253690 time 760043576957 state 0x2 ref 0 00000001:00080000:3.0:1789760780.510454:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974159 is committed 00000100:00100000:1.0:1789760780.513674:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760780.513834:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512738944 00000100:00100000:3.0:1789760780.513845:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512738944 00000100:00100000:3.0:1789760780.513895:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 760011253690 time 760043576957 state 0x0 ref 1 00000100:00100000:3.0:1789760780.513909:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 760043837373 time 760043576957 state 0x1 ref 1 00000100:00100000:3.0:1789760780.513942:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 760043837373 time 760078241637 state 0x0 ref 1 00000100:00100000:3.0:1789760780.513954:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4130700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512738944:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760780.514001:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28020736, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0xd807 00010000:00010000:3.0:1789760780.514009:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2fc39 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.514026:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=760s v=(0 0) 00010000:00010000:3.0:1789760780.514031:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760780.514037:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2fc39 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28020736->28024831] (req 28020736->28024831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.514065:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2fc39 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28020736->28024831] (req 28020736->28024831) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760780.543057:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974160 is committed 00002000:00010000:3.0:1789760780.543151:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28020736 -> 28024832 00002000:00010000:3.0:1789760780.543156:0:13062:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760779 -> 1789760780 00002000:00010000:3.0:1789760780.543159:0:13062:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760779 -> 1789760780 00010000:00010000:3.0:1789760780.543172:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2fc39 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28020736->28024831] (req 28020736->28024831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.543185:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2fc39 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28020736->28024831] (req 28020736->28024831) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.543194:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2fc39 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28020736->28024831] (req 28020736->28024831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.543202:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760780.543207:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2fc39 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28020736->28024831] (req 28020736->28024831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.543216:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2fc39 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28020736->28024831] (req 28020736->28024831) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.543236:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2fc39 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28020736->28024831] (req 28020736->28024831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.543244:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2fc39 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28020736->28024831] (req 28020736->28024831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760780.543341:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4130700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512738944:12345-192.168.203.56@tcp:4:dd.0 Request processed in 29383us (29671us total) trans 4294974160 rc 0/0 00000100:00100000:3.0:1789760780.543351:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6871 00000100:00100000:3.0:1789760780.543361:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 760043837373 time 760078241637 state 0x2 ref 0 00000100:00100000:2.0:1789760780.547530:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760780.549050:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760780.549090:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760780.549163:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760780.549331:0:5848:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512739328 00000100:00100000:3.0:1789760780.549342:0:5848:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512739328 00000100:00100000:3.0:1789760780.549397:0:5848:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512739456 00000100:00100000:3.0:1789760780.549400:0:5848:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512739456 00000100:00100000:3.0:1789760780.549430:0:5848:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 509 00000100:00100000:3.0:1789760780.549449:0:5848:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4131180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+8:2359:x1876699512739328:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:3.0:1789760780.549526:0:5848:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4131180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+8:2359:x1876699512739328:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 78us (490us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760780.549534:0:5848:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 509 00000100:00100000:3.0:1789760780.549552:0:5848:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 510 00000100:00100000:3.0:1789760780.549559:0:5848:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4132300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:d2c40150-e97b-40da-952b-cb9f18bf7e96+5:2358:x1876699512739456:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:3.0:1789760780.549584:0:5848:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4132300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:d2c40150-e97b-40da-952b-cb9f18bf7e96+5:2358:x1876699512739456:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 25us (421us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760780.549590:0:5848:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 510 00000100:00080000:3.0:1789760780.549610:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789760780.550087:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512739072 00000100:00100000:0.0:1789760780.550092:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512739072 00000100:00100000:0.0:1789760780.550131:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 760043837373 time 760078241637 state 0x0 ref 1 00000100:00100000:0.0:1789760780.550145:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 760078502053 time 760078241637 state 0x1 ref 1 00000100:00100000:0.0:1789760780.550168:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 760078502053 time 760114468322 state 0x0 ref 1 00000100:00100000:0.0:1789760780.550180:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac403b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512739072:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760780.550223:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28024832, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0xd807 00010000:00010000:0.0:1789760780.550232:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2fc40 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.550252:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=760s v=(0 0) 00010000:00010000:0.0:1789760780.550256:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760780.550262:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2fc40 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28024832->28028927] (req 28024832->28028927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.550276:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2fc40 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28024832->28028927] (req 28024832->28028927) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760780.550996:0:5842:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512739200 00000100:00100000:2.0:1789760780.551002:0:5842:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512739200 00000100:00100000:2.0:1789760780.551057:0:5842:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 412 00000100:00100000:2.0:1789760780.551066:0:5842:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febe00c5850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:e6cf7267-9b19-462a-a263-14115df15dab+13:2358:x1876699512739200:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:2.0:1789760780.551117:0:5842:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febe00c5850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:e6cf7267-9b19-462a-a263-14115df15dab+13:2358:x1876699512739200:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 51us (2026us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760780.551124:0:5842:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 412 00000001:00080000:1.0:1789760780.593261:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974161 is committed 00002000:00010000:3.0:1789760780.593353:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28024832 -> 28028928 00010000:00010000:3.0:1789760780.593369:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2fc40 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28024832->28028927] (req 28024832->28028927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.593382:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2fc40 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28024832->28028927] (req 28024832->28028927) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.593392:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2fc40 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28024832->28028927] (req 28024832->28028927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.593400:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760780.593405:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2fc40 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28024832->28028927] (req 28024832->28028927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.593414:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2fc40 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28024832->28028927] (req 28024832->28028927) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.593434:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2fc40 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28024832->28028927] (req 28024832->28028927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.593442:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2fc40 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28024832->28028927] (req 28024832->28028927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760780.593539:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac403b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512739072:12345-192.168.203.56@tcp:4:dd.0 Request processed in 43356us (46012us total) trans 4294974161 rc 0/0 00000100:00100000:3.0:1789760780.593549:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6872 00000100:00100000:3.0:1789760780.593559:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 760078502053 time 760114468322 state 0x2 ref 0 00000100:00100000:3.0:1789760780.597431:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760780.597538:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512739584 00000100:00100000:1.0:1789760780.597549:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512739584 00000100:00100000:1.0:1789760780.597602:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 760078502053 time 760114468322 state 0x0 ref 1 00000100:00100000:1.0:1789760780.597618:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 760114728738 time 760114468322 state 0x1 ref 1 00000100:00100000:1.0:1789760780.597731:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 760114728738 time 760161996653 state 0x0 ref 1 00000100:00100000:1.0:1789760780.597745:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac531dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512739584:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760780.597793:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28028928, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0xd807 00010000:00010000:1.0:1789760780.597802:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2fc47 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.597817:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=760s v=(0 0) 00010000:00010000:1.0:1789760780.597822:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760780.597829:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2fc47 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28028928->28033023] (req 28028928->28033023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.597849:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2fc47 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28028928->28033023] (req 28028928->28033023) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760780.626447:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974162 is committed 00002000:00010000:3.0:1789760780.627932:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28028928 -> 28033024 00010000:00010000:3.0:1789760780.627950:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2fc47 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28028928->28033023] (req 28028928->28033023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.627963:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2fc47 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28028928->28033023] (req 28028928->28033023) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.627973:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2fc47 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28028928->28033023] (req 28028928->28033023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.627980:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760780.627986:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2fc47 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28028928->28033023] (req 28028928->28033023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.627994:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2fc47 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28028928->28033023] (req 28028928->28033023) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.628013:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2fc47 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28028928->28033023] (req 28028928->28033023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.628036:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2fc47 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28028928->28033023] (req 28028928->28033023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760780.628281:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac531dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512739584:12345-192.168.203.56@tcp:4:dd.0 Request processed in 30534us (30854us total) trans 4294974162 rc 0/0 00000100:00100000:3.0:1789760780.628292:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6873 00000100:00100000:3.0:1789760780.628302:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 760114728738 time 760161996653 state 0x2 ref 0 00000100:00100000:1.0:1789760780.634110:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760780.634164:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512739712 00000100:00100000:3.0:1789760780.634174:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512739712 00000100:00100000:3.0:1789760780.634230:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 760114728738 time 760161996653 state 0x0 ref 1 00000100:00100000:3.0:1789760780.634245:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 760162257069 time 760161996653 state 0x1 ref 1 00000100:00100000:3.0:1789760780.634490:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 760162257069 time 760198788556 state 0x0 ref 1 00000100:00100000:3.0:1789760780.634504:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4131f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512739712:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760780.634552:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28033024, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0xd807 00010000:00010000:3.0:1789760780.634561:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2fc4e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.634578:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=760s v=(0 0) 00010000:00010000:3.0:1789760780.634582:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760780.634589:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2fc4e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28033024->28037119] (req 28033024->28037119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.634607:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2fc4e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28033024->28037119] (req 28033024->28037119) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.658805:0:7647:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:3.0:1789760780.670082:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28033024 -> 28037120 00010000:00010000:3.0:1789760780.670102:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2fc4e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28033024->28037119] (req 28033024->28037119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.670115:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2fc4e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28033024->28037119] (req 28033024->28037119) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.670125:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2fc4e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28033024->28037119] (req 28033024->28037119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.670132:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760780.670137:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2fc4e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28033024->28037119] (req 28033024->28037119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.670145:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2fc4e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28033024->28037119] (req 28033024->28037119) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.670162:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2fc4e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28033024->28037119] (req 28033024->28037119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.670169:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2fc4e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28033024->28037119] (req 28033024->28037119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760780.670330:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974163 is committed 00000100:00100000:3.0:1789760780.670384:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4131f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512739712:12345-192.168.203.56@tcp:4:dd.0 Request processed in 35876us (36277us total) trans 4294974163 rc 0/0 00000100:00100000:3.0:1789760780.670395:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6874 00000100:00100000:3.0:1789760780.670404:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 760162257069 time 760198788556 state 0x2 ref 0 00000100:00100000:2.0:1789760780.673317:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760780.673401:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512739840 00000100:00100000:3.0:1789760780.673410:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512739840 00000100:00100000:3.0:1789760780.673520:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 760162257069 time 760198788556 state 0x0 ref 1 00000100:00100000:3.0:1789760780.673536:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 760199048972 time 760198788556 state 0x1 ref 1 00000100:00100000:3.0:1789760780.673604:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 760199048972 time 760237902825 state 0x0 ref 1 00000100:00100000:3.0:1789760780.673617:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4038380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512739840:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760780.673666:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28037120, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0xd807 00010000:00010000:3.0:1789760780.673674:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2fc55 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.673690:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=760s v=(0 0) 00010000:00010000:3.0:1789760780.673710:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760780.673733:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2fc55 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28037120->28041215] (req 28037120->28041215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.673747:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2fc55 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28037120->28041215] (req 28037120->28041215) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760780.700441:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974164 is committed 00002000:00010000:3.0:1789760780.700723:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28037120 -> 28041216 00010000:00010000:3.0:1789760780.700740:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2fc55 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28037120->28041215] (req 28037120->28041215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.700754:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2fc55 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28037120->28041215] (req 28037120->28041215) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.700763:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2fc55 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28037120->28041215] (req 28037120->28041215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.700771:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760780.700776:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2fc55 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28037120->28041215] (req 28037120->28041215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.700784:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2fc55 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28037120->28041215] (req 28037120->28041215) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.700802:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2fc55 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28037120->28041215] (req 28037120->28041215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.700809:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2fc55 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28037120->28041215] (req 28037120->28041215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760780.700904:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4038380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512739840:12345-192.168.203.56@tcp:4:dd.0 Request processed in 27284us (27590us total) trans 4294974164 rc 0/0 00000100:00100000:3.0:1789760780.700918:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6875 00000100:00100000:3.0:1789760780.700928:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 760199048972 time 760237902825 state 0x2 ref 0 00000100:00100000:2.0:1789760780.706082:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760780.706552:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512739968 00000100:00100000:3.0:1789760780.706562:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512739968 00000100:00100000:3.0:1789760780.706624:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 760199048972 time 760237902825 state 0x0 ref 1 00000100:00100000:3.0:1789760780.706639:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 760238163241 time 760237902825 state 0x1 ref 1 00000100:00100000:3.0:1789760780.706672:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 760238163241 time 760270971420 state 0x0 ref 1 00000100:00100000:3.0:1789760780.706689:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4039500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512739968:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760780.706781:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28041216, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0xd807 00010000:00010000:3.0:1789760780.706790:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2fc5c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.706808:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=760s v=(0 0) 00010000:00010000:3.0:1789760780.706813:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760780.706819:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2fc5c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28041216->28045311] (req 28041216->28045311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.706833:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2fc5c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28041216->28045311] (req 28041216->28045311) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760780.760526:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28041216 -> 28045312 00010000:00010000:2.0:1789760780.760547:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2fc5c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28041216->28045311] (req 28041216->28045311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.760561:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2fc5c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28041216->28045311] (req 28041216->28045311) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.760569:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2fc5c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28041216->28045311] (req 28041216->28045311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.760576:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760780.760582:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2fc5c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28041216->28045311] (req 28041216->28045311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.760597:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2fc5c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28041216->28045311] (req 28041216->28045311) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.760615:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2fc5c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28041216->28045311] (req 28041216->28045311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.760624:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2fc5c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28041216->28045311] (req 28041216->28045311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760780.760815:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4039500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2360:x1876699512739968:12345-192.168.203.56@tcp:4:dd.0 Request processed in 54127us (54815us total) trans 4294974165 rc 0/0 00000100:00100000:2.0:1789760780.760826:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6876 00000100:00100000:2.0:1789760780.760835:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 760238163241 time 760270971420 state 0x2 ref 0 00000001:00080000:1.0:1789760780.761588:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974165 is committed 00000100:00100000:1.0:1789760780.764090:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760780.766332:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512740096 00000100:00100000:3.0:1789760780.766350:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512740096 00000100:00100000:3.0:1789760780.766482:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 760238163241 time 760270971420 state 0x0 ref 1 00000100:00100000:3.0:1789760780.766498:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 760271231836 time 760270971420 state 0x1 ref 1 00000100:00100000:3.0:1789760780.766569:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 760271231836 time 760330868331 state 0x0 ref 1 00000100:00100000:3.0:1789760780.766585:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4130380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512740096:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760780.766633:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28045312, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0xd807 00010000:00010000:3.0:1789760780.766642:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2fc63 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.766656:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=760s v=(0 0) 00010000:00010000:3.0:1789760780.766661:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760780.766666:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2fc63 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28045312->28049407] (req 28045312->28049407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.766679:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2fc63 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28045312->28049407] (req 28045312->28049407) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760780.824430:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974166 is committed 00002000:00010000:3.0:1789760780.824565:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28045312 -> 28049408 00010000:00010000:3.0:1789760780.824581:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2fc63 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28045312->28049407] (req 28045312->28049407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.824594:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2fc63 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28045312->28049407] (req 28045312->28049407) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.824603:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2fc63 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28045312->28049407] (req 28045312->28049407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.824611:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760780.824617:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2fc63 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28045312->28049407] (req 28045312->28049407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.824626:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2fc63 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28045312->28049407] (req 28045312->28049407) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.824643:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2fc63 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28045312->28049407] (req 28045312->28049407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.824651:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2fc63 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28045312->28049407] (req 28045312->28049407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760780.824792:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4130380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512740096:12345-192.168.203.56@tcp:4:dd.0 Request processed in 58207us (60706us total) trans 4294974166 rc 0/0 00000100:00100000:3.0:1789760780.824802:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6877 00000100:00100000:3.0:1789760780.824810:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 760271231836 time 760330868331 state 0x2 ref 0 00000100:00100000:3.0:1789760780.827940:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760780.828192:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512740224 00000100:00100000:0.0:1789760780.828203:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512740224 00000100:00100000:0.0:1789760780.828255:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 760271231836 time 760330868331 state 0x0 ref 1 00000100:00100000:0.0:1789760780.828269:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 760331128747 time 760330868331 state 0x1 ref 1 00000100:00100000:0.0:1789760780.828300:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 760331128747 time 760392599859 state 0x0 ref 1 00000100:00100000:0.0:1789760780.828317:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac531d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512740224:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760780.828365:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28049408, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0xd807 00010000:00010000:0.0:1789760780.828375:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2fc6a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.828389:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=760s v=(0 0) 00010000:00010000:0.0:1789760780.828394:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760780.828400:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2fc6a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28049408->28053503] (req 28049408->28053503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.828420:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2fc6a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28049408->28053503] (req 28049408->28053503) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760780.865499:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974167 is committed 00002000:00010000:3.0:1789760780.865638:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28049408 -> 28053504 00010000:00010000:3.0:1789760780.865653:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2fc6a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28049408->28053503] (req 28049408->28053503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.865666:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2fc6a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28049408->28053503] (req 28049408->28053503) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.865680:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2fc6a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28049408->28053503] (req 28049408->28053503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.865688:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760780.865693:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2fc6a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28049408->28053503] (req 28049408->28053503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.865729:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2fc6a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28049408->28053503] (req 28049408->28053503) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.865749:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2fc6a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28049408->28053503] (req 28049408->28053503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.865757:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2fc6a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28049408->28053503] (req 28049408->28053503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760780.865855:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac531d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512740224:12345-192.168.203.56@tcp:4:dd.0 Request processed in 37539us (37918us total) trans 4294974167 rc 0/0 00000100:00100000:3.0:1789760780.865867:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6878 00000100:00100000:3.0:1789760780.865878:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 760331128747 time 760392599859 state 0x2 ref 0 00000100:00100000:1.0:1789760780.869076:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760780.869174:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512740352 00000100:00100000:2.0:1789760780.869185:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512740352 00000100:00100000:2.0:1789760780.869237:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 760331128747 time 760392599859 state 0x0 ref 1 00000100:00100000:2.0:1789760780.869252:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 760392860275 time 760392599859 state 0x1 ref 1 00000100:00100000:2.0:1789760780.869285:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 760392860275 time 760433584232 state 0x0 ref 1 00000100:00100000:2.0:1789760780.869297:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4133b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512740352:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760780.869343:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28053504, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0xd807 00010000:00010000:2.0:1789760780.869352:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2fc71 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.869368:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=760s v=(0 0) 00010000:00010000:2.0:1789760780.869372:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760780.869378:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2fc71 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28053504->28057599] (req 28053504->28057599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.869390:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2fc71 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28053504->28057599] (req 28053504->28057599) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760780.967778:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974168 is committed 00002000:00010000:3.0:1789760780.967954:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28053504 -> 28057600 00010000:00010000:3.0:1789760780.967971:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2fc71 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28053504->28057599] (req 28053504->28057599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.967990:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2fc71 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28053504->28057599] (req 28053504->28057599) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.967999:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2fc71 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28053504->28057599] (req 28053504->28057599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.968006:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760780.968010:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2fc71 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28053504->28057599] (req 28053504->28057599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.968019:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2fc71 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28053504->28057599] (req 28053504->28057599) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.968055:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2fc71 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28053504->28057599] (req 28053504->28057599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.968069:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2fc71 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28053504->28057599] (req 28053504->28057599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760780.968204:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4133b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512740352:12345-192.168.203.56@tcp:4:dd.0 Request processed in 98904us (99211us total) trans 4294974168 rc 0/0 00000100:00100000:3.0:1789760780.968213:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6879 00000100:00100000:3.0:1789760780.968223:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 760392860275 time 760433584232 state 0x2 ref 0 00000100:00100000:0.0:1789760780.973599:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760780.974776:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512740480 00000100:00100000:3.0:1789760780.974788:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512740480 00000100:00100000:3.0:1789760780.974842:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 760392860275 time 760433584232 state 0x0 ref 1 00000100:00100000:3.0:1789760780.974856:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 760433844648 time 760433584232 state 0x1 ref 1 00000100:00100000:3.0:1789760780.974878:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 760433844648 time 760539178524 state 0x0 ref 1 00000100:00100000:3.0:1789760780.974890:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4059500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512740480:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760780.974934:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28057600, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0xd807 00010000:00010000:3.0:1789760780.974943:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2fc78 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.974964:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=760s v=(0 0) 00010000:00010000:3.0:1789760780.974968:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760780.974974:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2fc78 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28057600->28061695] (req 28057600->28061695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.974987:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2fc78 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28057600->28061695] (req 28057600->28061695) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760781.026359:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28057600 -> 28061696 00010000:00010000:1.0:1789760781.026379:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2fc78 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28057600->28061695] (req 28057600->28061695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.026391:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2fc78 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28057600->28061695] (req 28057600->28061695) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.026399:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2fc78 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28057600->28061695] (req 28057600->28061695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.026405:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760781.026409:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2fc78 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28057600->28061695] (req 28057600->28061695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.026422:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2fc78 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28057600->28061695] (req 28057600->28061695) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.026440:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2fc78 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28057600->28061695] (req 28057600->28061695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.026446:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2fc78 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28057600->28061695] (req 28057600->28061695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760781.026548:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4059500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2361:x1876699512740480:12345-192.168.203.56@tcp:4:dd.0 Request processed in 51654us (52964us total) trans 4294974169 rc 0/0 00000100:00100000:1.0:1789760781.026558:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6880 00000100:00100000:1.0:1789760781.026577:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 760433844648 time 760539178524 state 0x2 ref 0 00000001:00080000:2.0:1789760781.027306:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974169 is committed 00000100:00100000:1.0:1789760781.030365:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760781.030449:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512740608 00000100:00100000:3.0:1789760781.030459:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512740608 00000100:00100000:3.0:1789760781.030532:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 760433844648 time 760539178524 state 0x0 ref 1 00000100:00100000:3.0:1789760781.030546:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 760539438940 time 760539178524 state 0x1 ref 1 00000100:00100000:3.0:1789760781.030582:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 760539438940 time 760594882014 state 0x0 ref 1 00000100:00100000:3.0:1789760781.030599:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4133800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512740608:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760781.030646:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28061696, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0xd807 00010000:00010000:3.0:1789760781.030658:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2fc7f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.030674:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=760s v=(0 0) 00010000:00010000:3.0:1789760781.030679:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760781.030685:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2fc7f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28061696->28065791] (req 28061696->28065791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.030908:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2fc7f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28061696->28065791] (req 28061696->28065791) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1789760781.034917:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:2.0:1789760781.062189:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974170 is committed 00002000:00010000:0.0:1789760781.062285:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28061696 -> 28065792 00010000:00010000:0.0:1789760781.062301:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2fc7f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28061696->28065791] (req 28061696->28065791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.062314:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2fc7f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28061696->28065791] (req 28061696->28065791) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.062324:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2fc7f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28061696->28065791] (req 28061696->28065791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.062332:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760781.062337:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2fc7f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28061696->28065791] (req 28061696->28065791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.062345:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2fc7f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28061696->28065791] (req 28061696->28065791) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.062364:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2fc7f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28061696->28065791] (req 28061696->28065791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.062372:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c2fc7f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28061696->28065791] (req 28061696->28065791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760781.062465:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4133800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512740608:12345-192.168.203.56@tcp:4:dd.0 Request processed in 31865us (32113us total) trans 4294974170 rc 0/0 00000100:00100000:0.0:1789760781.062476:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6881 00000100:00100000:0.0:1789760781.062485:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 760539438940 time 760594882014 state 0x2 ref 0 00000100:00100000:0.0:1789760781.065743:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760781.066001:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512740736 00000100:00100000:3.0:1789760781.066010:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512740736 00000100:00100000:3.0:1789760781.066077:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 760539438940 time 760594882014 state 0x0 ref 1 00000100:00100000:3.0:1789760781.066092:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 760595142430 time 760594882014 state 0x1 ref 1 00000100:00100000:3.0:1789760781.066123:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 760595142430 time 760630422756 state 0x0 ref 1 00000100:00100000:3.0:1789760781.066135:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac405ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512740736:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760781.066182:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28065792, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0xd807 00010000:00010000:3.0:1789760781.066191:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2fc86 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.066211:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=760s v=(0 0) 00010000:00010000:3.0:1789760781.066215:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760781.066221:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2fc86 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28065792->28069887] (req 28065792->28069887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.066235:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2fc86 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28065792->28069887] (req 28065792->28069887) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760781.099004:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974171 is committed 00002000:00010000:0.0:1789760781.099121:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28065792 -> 28069888 00010000:00010000:0.0:1789760781.099136:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2fc86 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28065792->28069887] (req 28065792->28069887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.099149:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2fc86 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28065792->28069887] (req 28065792->28069887) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.099159:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2fc86 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28065792->28069887] (req 28065792->28069887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.099166:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760781.099172:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2fc86 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28065792->28069887] (req 28065792->28069887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.099181:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2fc86 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28065792->28069887] (req 28065792->28069887) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.099200:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2fc86 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28065792->28069887] (req 28065792->28069887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.099213:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c2fc86 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28065792->28069887] (req 28065792->28069887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760781.099304:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac405ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512740736:12345-192.168.203.56@tcp:4:dd.0 Request processed in 33167us (33603us total) trans 4294974171 rc 0/0 00000100:00100000:0.0:1789760781.099314:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6882 00000100:00100000:0.0:1789760781.099323:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 760595142430 time 760630422756 state 0x2 ref 0 00000100:00100000:1.0:1789760781.101500:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760781.101580:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512740864 00000100:00100000:3.0:1789760781.101588:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512740864 00000100:00100000:3.0:1789760781.101628:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 760595142430 time 760630422756 state 0x0 ref 1 00000100:00100000:3.0:1789760781.101640:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 760630683172 time 760630422756 state 0x1 ref 1 00000100:00100000:3.0:1789760781.102443:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 760630683172 time 760665995957 state 0x0 ref 1 00000100:00100000:2.0:1789760781.102534:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4132680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512740864:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760781.102580:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28069888, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0xd807 00010000:00010000:2.0:1789760781.102588:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2fc8d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.102603:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=760s v=(0 0) 00010000:00010000:2.0:1789760781.102607:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760781.102613:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2fc8d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28069888->28073983] (req 28069888->28073983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.102626:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2fc8d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28069888->28073983] (req 28069888->28073983) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760781.138479:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974172 is committed 00002000:00010000:2.0:1789760781.138586:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28069888 -> 28073984 00010000:00010000:2.0:1789760781.138601:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2fc8d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28069888->28073983] (req 28069888->28073983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.138614:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2fc8d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28069888->28073983] (req 28069888->28073983) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.138623:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2fc8d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28069888->28073983] (req 28069888->28073983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.138629:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760781.138634:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2fc8d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28069888->28073983] (req 28069888->28073983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.138642:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2fc8d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28069888->28073983] (req 28069888->28073983) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.138659:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2fc8d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28069888->28073983] (req 28069888->28073983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.138666:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2fc8d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28069888->28073983] (req 28069888->28073983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760781.138907:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4132680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512740864:12345-192.168.203.56@tcp:4:dd.0 Request processed in 36446us (37411us total) trans 4294974172 rc 0/0 00000100:00100000:2.0:1789760781.138918:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6883 00000100:00100000:2.0:1789760781.138927:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 760630683172 time 760665995957 state 0x2 ref 0 00000100:00100000:1.0:1789760781.142326:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760781.142422:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512740992 00000100:00100000:0.0:1789760781.142490:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512740992 00000100:00100000:0.0:1789760781.142545:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 760630683172 time 760665995957 state 0x0 ref 1 00000100:00100000:0.0:1789760781.142560:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 760666256373 time 760665995957 state 0x1 ref 1 00000100:00100000:0.0:1789760781.142632:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 760666256373 time 760706930829 state 0x0 ref 1 00000100:00100000:0.0:1789760781.142645:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febc7b32680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512740992:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760781.142692:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28073984, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0xd807 00010000:00010000:0.0:1789760781.142735:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2fc94 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.142750:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=760s v=(0 0) 00010000:00010000:0.0:1789760781.142755:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760781.142762:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2fc94 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28073984->28078079] (req 28073984->28078079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.142781:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2fc94 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28073984->28078079] (req 28073984->28078079) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760781.174078:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974173 is committed 00002000:00010000:2.0:1789760781.174418:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28073984 -> 28078080 00010000:00010000:2.0:1789760781.174433:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2fc94 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28073984->28078079] (req 28073984->28078079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.174445:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2fc94 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28073984->28078079] (req 28073984->28078079) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.174454:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2fc94 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28073984->28078079] (req 28073984->28078079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.174462:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760781.174466:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2fc94 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28073984->28078079] (req 28073984->28078079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.174476:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2fc94 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28073984->28078079] (req 28073984->28078079) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.174494:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2fc94 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28073984->28078079] (req 28073984->28078079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.174502:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2fc94 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28073984->28078079] (req 28073984->28078079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760781.174590:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febc7b32680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512740992:12345-192.168.203.56@tcp:4:dd.0 Request processed in 31942us (32270us total) trans 4294974173 rc 0/0 00000100:00100000:2.0:1789760781.174600:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6884 00000100:00100000:2.0:1789760781.174609:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 760666256373 time 760706930829 state 0x2 ref 0 00000100:00100000:0.0:1789760781.178380:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760781.178464:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512741120 00000100:00100000:2.0:1789760781.178474:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512741120 00000100:00100000:2.0:1789760781.178529:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 760666256373 time 760706930829 state 0x0 ref 1 00000100:00100000:2.0:1789760781.178544:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 760707191245 time 760706930829 state 0x1 ref 1 00000100:00100000:2.0:1789760781.178578:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 760707191245 time 760742877460 state 0x0 ref 1 00000100:00100000:2.0:1789760781.178590:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4058e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512741120:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760781.178636:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28078080, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0xd807 00010000:00010000:2.0:1789760781.178645:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2fc9b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.178662:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=760s v=(0 0) 00010000:00010000:2.0:1789760781.178667:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760781.178673:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2fc9b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28078080->28082175] (req 28078080->28082175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.178687:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2fc9b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28078080->28082175] (req 28078080->28082175) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760781.223241:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28078080 -> 28082176 00010000:00010000:3.0:1789760781.223262:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2fc9b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28078080->28082175] (req 28078080->28082175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.223275:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2fc9b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28078080->28082175] (req 28078080->28082175) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.223282:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2fc9b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28078080->28082175] (req 28078080->28082175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.223294:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760781.223299:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2fc9b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28078080->28082175] (req 28078080->28082175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.223308:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2fc9b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28078080->28082175] (req 28078080->28082175) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.223326:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2fc9b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28078080->28082175] (req 28078080->28082175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.223334:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2fc9b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28078080->28082175] (req 28078080->28082175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760781.223440:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4058e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2360:x1876699512741120:12345-192.168.203.56@tcp:4:dd.0 Request processed in 44847us (45064us total) trans 4294974174 rc 0/0 00000100:00100000:3.0:1789760781.223450:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6885 00000100:00100000:3.0:1789760781.223460:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 760707191245 time 760742877460 state 0x2 ref 0 00000001:00080000:3.0:1789760781.223505:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974174 is committed 00000100:00100000:1.0:1789760781.226921:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760781.227051:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512741248 00000100:00100000:2.0:1789760781.227068:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512741248 00000100:00100000:2.0:1789760781.227129:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 760707191245 time 760742877460 state 0x0 ref 1 00000100:00100000:2.0:1789760781.227145:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 760743137876 time 760742877460 state 0x1 ref 1 00000100:00100000:2.0:1789760781.227174:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 760743137876 time 760791473876 state 0x0 ref 1 00000100:00100000:2.0:1789760781.227187:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febc7b33100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512741248:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760781.227235:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28082176, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0xd807 00010000:00010000:2.0:1789760781.227244:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2fca2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.227260:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=760s v=(0 0) 00010000:00010000:2.0:1789760781.227264:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760781.227270:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2fca2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28082176->28086271] (req 28082176->28086271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.227283:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2fca2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28082176->28086271] (req 28082176->28086271) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760781.268650:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974175 is committed 00002000:00010000:1.0:1789760781.268871:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28082176 -> 28086272 00010000:00010000:1.0:1789760781.268887:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2fca2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28082176->28086271] (req 28082176->28086271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.268901:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2fca2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28082176->28086271] (req 28082176->28086271) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.268915:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2fca2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28082176->28086271] (req 28082176->28086271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.268923:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760781.268928:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2fca2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28082176->28086271] (req 28082176->28086271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.268937:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2fca2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28082176->28086271] (req 28082176->28086271) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.268956:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2fca2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28082176->28086271] (req 28082176->28086271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.268963:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2fca2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28082176->28086271] (req 28082176->28086271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760781.269162:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febc7b33100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512741248:12345-192.168.203.56@tcp:4:dd.0 Request processed in 41972us (42246us total) trans 4294974175 rc 0/0 00000100:00100000:1.0:1789760781.269173:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6886 00000100:00100000:1.0:1789760781.269183:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 760743137876 time 760791473876 state 0x2 ref 0 00000100:00100000:0.0:1789760781.272291:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760781.272868:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512741376 00000100:00100000:2.0:1789760781.272879:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512741376 00000100:00100000:2.0:1789760781.272935:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 760743137876 time 760791473876 state 0x0 ref 1 00000100:00100000:2.0:1789760781.272950:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 760791734292 time 760791473876 state 0x1 ref 1 00000100:00100000:2.0:1789760781.272981:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 760791734292 time 760837281030 state 0x0 ref 1 00000100:00100000:2.0:1789760781.272994:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4059180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512741376:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760781.274172:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28086272, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0xd807 00010000:00010000:2.0:1789760781.274184:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2fca9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.274201:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=760s v=(0 0) 00010000:00010000:2.0:1789760781.274205:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760781.274211:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2fca9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28086272->28090367] (req 28086272->28090367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.274226:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2fca9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28086272->28090367] (req 28086272->28090367) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760781.313968:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28086272 -> 28090368 00010000:00010000:3.0:1789760781.314000:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2fca9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28086272->28090367] (req 28086272->28090367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.314014:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2fca9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28086272->28090367] (req 28086272->28090367) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.314024:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2fca9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28086272->28090367] (req 28086272->28090367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.314032:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760781.314047:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2fca9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28086272->28090367] (req 28086272->28090367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.314056:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2fca9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28086272->28090367] (req 28086272->28090367) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.314077:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2fca9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28086272->28090367] (req 28086272->28090367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.314086:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c2fca9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28086272->28090367] (req 28086272->28090367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760781.314191:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4059180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2360:x1876699512741376:12345-192.168.203.56@tcp:4:dd.0 Request processed in 41194us (41903us total) trans 4294974176 rc 0/0 00000100:00100000:3.0:1789760781.314201:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6887 00000100:00100000:3.0:1789760781.314211:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 760791734292 time 760837281030 state 0x2 ref 0 00000001:00080000:1.0:1789760781.314319:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974176 is committed 00000100:00100000:2.0:1789760781.323081:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760781.323377:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512741504 00000100:00100000:3.0:1789760781.323388:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512741504 00000100:00100000:3.0:1789760781.323445:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 760791734292 time 760837281030 state 0x0 ref 1 00000100:00100000:3.0:1789760781.323462:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 760837541446 time 760837281030 state 0x1 ref 1 00000100:00100000:3.0:1789760781.323548:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 760837541446 time 760887847381 state 0x0 ref 1 00000100:00100000:3.0:1789760781.323560:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac403a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512741504:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760781.323609:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28090368, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0xd807 00010000:00010000:3.0:1789760781.323618:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2fcb0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.323634:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=760s v=(0 0) 00010000:00010000:3.0:1789760781.323638:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760781.323645:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2fcb0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28090368->28094463] (req 28090368->28094463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.323659:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2fcb0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28090368->28094463] (req 28090368->28094463) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760781.367587:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974177 is committed 00002000:00010000:1.0:1789760781.367684:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28090368 -> 28094464 00010000:00010000:1.0:1789760781.367731:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2fcb0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28090368->28094463] (req 28090368->28094463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.367751:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2fcb0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28090368->28094463] (req 28090368->28094463) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.367760:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2fcb0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28090368->28094463] (req 28090368->28094463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.367767:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760781.367773:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2fcb0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28090368->28094463] (req 28090368->28094463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.367781:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2fcb0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28090368->28094463] (req 28090368->28094463) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.367802:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2fcb0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28090368->28094463] (req 28090368->28094463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.367809:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c2fcb0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28090368->28094463] (req 28090368->28094463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760781.367892:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac403a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512741504:12345-192.168.203.56@tcp:4:dd.0 Request processed in 44330us (44815us total) trans 4294974177 rc 0/0 00000100:00100000:1.0:1789760781.367902:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6888 00000100:00100000:1.0:1789760781.367912:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 760837541446 time 760887847381 state 0x2 ref 0 00000100:00100000:1.0:1789760781.375607:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760781.375729:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512741632 00000100:00100000:2.0:1789760781.375739:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512741632 00000100:00100000:2.0:1789760781.375790:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 760837541446 time 760887847381 state 0x0 ref 1 00000100:00100000:2.0:1789760781.375805:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 760888107797 time 760887847381 state 0x1 ref 1 00000100:00100000:2.0:1789760781.375830:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 760888107797 time 760940130353 state 0x0 ref 1 00000100:00100000:2.0:1789760781.375842:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febc7b33480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512741632:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760781.375890:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28094464, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0xd807 00010000:00010000:2.0:1789760781.375899:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2fcb7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.375914:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=760s v=(0 0) 00010000:00010000:2.0:1789760781.375919:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760781.375924:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2fcb7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28094464->28098559] (req 28094464->28098559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.375938:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2fcb7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28094464->28098559] (req 28094464->28098559) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760781.418688:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974178 is committed 00002000:00010000:2.0:1789760781.419293:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28094464 -> 28098560 00010000:00010000:2.0:1789760781.419310:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2fcb7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28094464->28098559] (req 28094464->28098559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.419322:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2fcb7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28094464->28098559] (req 28094464->28098559) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.419330:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2fcb7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28094464->28098559] (req 28094464->28098559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.419336:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760781.419341:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2fcb7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28094464->28098559] (req 28094464->28098559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.419349:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2fcb7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28094464->28098559] (req 28094464->28098559) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.419368:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2fcb7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28094464->28098559] (req 28094464->28098559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.419381:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2fcb7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28094464->28098559] (req 28094464->28098559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760781.419477:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febc7b33480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512741632:12345-192.168.203.56@tcp:4:dd.0 Request processed in 43631us (43874us total) trans 4294974178 rc 0/0 00000100:00100000:2.0:1789760781.419487:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6889 00000100:00100000:2.0:1789760781.419497:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 760888107797 time 760940130353 state 0x2 ref 0 00000100:00100000:2.0:1789760781.437246:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760781.438275:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512741760 00000100:00100000:3.0:1789760781.438288:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512741760 00000100:00100000:3.0:1789760781.438349:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 760888107797 time 760940130353 state 0x0 ref 1 00000100:00100000:3.0:1789760781.438365:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 760940390769 time 760940130353 state 0x1 ref 1 00000100:00100000:3.0:1789760781.438407:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 760940390769 time 761002705984 state 0x0 ref 1 00000100:00100000:3.0:1789760781.438420:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4039c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512741760:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760781.438478:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28098560, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0xd807 00010000:00010000:3.0:1789760781.438488:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2fcbe lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.438503:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=761s v=(0 0) 00010000:00010000:3.0:1789760781.438508:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760781.438514:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2fcbe lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28098560->28102655] (req 28098560->28102655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.438528:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2fcbe lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28098560->28102655] (req 28098560->28102655) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760781.486670:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974179 is committed 00002000:00010000:2.0:1789760781.487076:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28098560 -> 28102656 00010000:00010000:2.0:1789760781.487094:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2fcbe lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28098560->28102655] (req 28098560->28102655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.487107:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2fcbe lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28098560->28102655] (req 28098560->28102655) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.487117:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2fcbe lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28098560->28102655] (req 28098560->28102655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.487126:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760781.487130:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2fcbe lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28098560->28102655] (req 28098560->28102655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.487140:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2fcbe lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28098560->28102655] (req 28098560->28102655) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.487165:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2fcbe lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28098560->28102655] (req 28098560->28102655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.487174:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2fcbe lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28098560->28102655] (req 28098560->28102655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760781.487268:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4039c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512741760:12345-192.168.203.56@tcp:4:dd.0 Request processed in 48845us (50027us total) trans 4294974179 rc 0/0 00000100:00100000:2.0:1789760781.487279:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6890 00000100:00100000:2.0:1789760781.487288:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 760940390769 time 761002705984 state 0x2 ref 0 00000100:00100000:0.0:1789760781.490790:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760781.491046:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512741888 00000100:00100000:1.0:1789760781.491072:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512741888 00000100:00100000:1.0:1789760781.491124:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 760940390769 time 761002705984 state 0x0 ref 1 00000100:00100000:1.0:1789760781.491138:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 761002966400 time 761002705984 state 0x1 ref 1 00000100:00100000:1.0:1789760781.491169:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 761002966400 time 761055469180 state 0x0 ref 1 00000100:00100000:1.0:1789760781.491181:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4059880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512741888:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760781.491232:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28102656, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0xd807 00010000:00010000:1.0:1789760781.491241:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2fcc5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.491259:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=761s v=(0 0) 00010000:00010000:1.0:1789760781.491264:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760781.491272:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2fcc5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28102656->28106751] (req 28102656->28106751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.491287:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2fcc5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28102656->28106751] (req 28102656->28106751) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760781.533414:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974180 is committed 00002000:00010000:3.0:1789760781.533599:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28102656 -> 28106752 00002000:00010000:3.0:1789760781.533604:0:6709:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760780 -> 1789760781 00002000:00010000:3.0:1789760781.533607:0:6709:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760780 -> 1789760781 00010000:00010000:3.0:1789760781.533621:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2fcc5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28102656->28106751] (req 28102656->28106751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.533635:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2fcc5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28102656->28106751] (req 28102656->28106751) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.533642:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2fcc5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28102656->28106751] (req 28102656->28106751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.533647:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760781.533651:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2fcc5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28102656->28106751] (req 28102656->28106751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.533661:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2fcc5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28102656->28106751] (req 28102656->28106751) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.533676:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2fcc5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28102656->28106751] (req 28102656->28106751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.533680:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2fcc5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28102656->28106751] (req 28102656->28106751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760781.533993:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4059880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512741888:12345-192.168.203.56@tcp:4:dd.0 Request processed in 42809us (43208us total) trans 4294974180 rc 0/0 00000100:00100000:0.0:1789760781.534002:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6891 00000100:00100000:0.0:1789760781.534012:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 761002966400 time 761055469180 state 0x2 ref 0 00000100:00100000:1.0:1789760781.538485:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760781.539165:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512742016 00000100:00100000:2.0:1789760781.539176:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512742016 00000100:00100000:2.0:1789760781.539229:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 761002966400 time 761055469180 state 0x0 ref 1 00000100:00100000:2.0:1789760781.539246:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 761055729596 time 761055469180 state 0x1 ref 1 00000100:00100000:2.0:1789760781.539279:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 761055729596 time 761103578803 state 0x0 ref 1 00000100:00100000:2.0:1789760781.539291:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac287b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512742016:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760781.539337:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28106752, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0xd807 00010000:00010000:2.0:1789760781.539347:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2fccc lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.539369:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=761s v=(0 0) 00010000:00010000:2.0:1789760781.539374:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760781.539381:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2fccc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28106752->28110847] (req 28106752->28110847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.539394:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2fccc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28106752->28110847] (req 28106752->28110847) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760781.569922:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974181 is committed 00002000:00010000:2.0:1789760781.570404:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28106752 -> 28110848 00010000:00010000:2.0:1789760781.570423:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2fccc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28106752->28110847] (req 28106752->28110847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.570437:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2fccc lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28106752->28110847] (req 28106752->28110847) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.570447:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2fccc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28106752->28110847] (req 28106752->28110847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.570455:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760781.570460:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2fccc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28106752->28110847] (req 28106752->28110847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.570470:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2fccc lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28106752->28110847] (req 28106752->28110847) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.570492:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2fccc lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28106752->28110847] (req 28106752->28110847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.570499:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2fccc lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28106752->28110847] (req 28106752->28110847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760781.570627:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac287b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512742016:12345-192.168.203.56@tcp:4:dd.0 Request processed in 31332us (32144us total) trans 4294974181 rc 0/0 00000100:00100000:2.0:1789760781.570639:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6892 00000100:00100000:2.0:1789760781.570649:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 761055729596 time 761103578803 state 0x2 ref 0 00000100:00100000:2.0:1789760781.575322:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760781.575416:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512742144 00000100:00100000:3.0:1789760781.575426:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512742144 00000100:00100000:3.0:1789760781.575478:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 761055729596 time 761103578803 state 0x0 ref 1 00000100:00100000:3.0:1789760781.575493:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 761103839219 time 761103578803 state 0x1 ref 1 00000100:00100000:3.0:1789760781.575567:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 761103839219 time 761139866428 state 0x0 ref 1 00000100:00100000:3.0:1789760781.575578:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4038e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512742144:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760781.575626:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28110848, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0xd807 00010000:00010000:3.0:1789760781.575634:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2fcd3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.575650:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=761s v=(0 0) 00010000:00010000:3.0:1789760781.575655:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760781.575661:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2fcd3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28110848->28114943] (req 28110848->28114943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.575676:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2fcd3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28110848->28114943] (req 28110848->28114943) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760781.614599:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974182 is committed 00002000:00010000:0.0:1789760781.614898:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28110848 -> 28114944 00010000:00010000:0.0:1789760781.614914:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2fcd3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28110848->28114943] (req 28110848->28114943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.614926:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2fcd3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28110848->28114943] (req 28110848->28114943) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.614934:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2fcd3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28110848->28114943] (req 28110848->28114943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.614941:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760781.614946:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2fcd3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28110848->28114943] (req 28110848->28114943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.614960:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2fcd3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28110848->28114943] (req 28110848->28114943) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.614980:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2fcd3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28110848->28114943] (req 28110848->28114943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.614988:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2fcd3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28110848->28114943] (req 28110848->28114943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760781.615091:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4038e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512742144:12345-192.168.203.56@tcp:4:dd.0 Request processed in 39510us (39773us total) trans 4294974182 rc 0/0 00000100:00100000:0.0:1789760781.615101:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6893 00000100:00100000:0.0:1789760781.615111:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 761103839219 time 761139866428 state 0x2 ref 0 00000100:00100000:0.0:1789760781.617814:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760781.621855:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512742272 00000100:00100000:2.0:1789760781.621871:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512742272 00000100:00100000:2.0:1789760781.621931:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 761103839219 time 761139866428 state 0x0 ref 1 00000100:00100000:2.0:1789760781.621947:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 761140126844 time 761139866428 state 0x1 ref 1 00000100:00100000:2.0:1789760781.621983:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 761140126844 time 761186282837 state 0x0 ref 1 00000100:00100000:2.0:1789760781.621997:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac405bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512742272:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760781.622055:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28114944, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0xd807 00010000:00010000:2.0:1789760781.622064:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2fcda lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.622084:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=761s v=(0 0) 00010000:00010000:2.0:1789760781.622089:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760781.622094:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2fcda lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28114944->28119039] (req 28114944->28119039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.622108:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2fcda lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28114944->28119039] (req 28114944->28119039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760781.650045:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974183 is committed 00002000:00010000:0.0:1789760781.650136:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28114944 -> 28119040 00010000:00010000:0.0:1789760781.650152:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2fcda lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28114944->28119039] (req 28114944->28119039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.650166:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2fcda lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28114944->28119039] (req 28114944->28119039) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.650175:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2fcda lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28114944->28119039] (req 28114944->28119039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.650187:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760781.650192:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2fcda lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28114944->28119039] (req 28114944->28119039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.650201:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2fcda lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28114944->28119039] (req 28114944->28119039) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.650220:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2fcda lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28114944->28119039] (req 28114944->28119039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.650228:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2fcda lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28114944->28119039] (req 28114944->28119039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760781.650356:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac405bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512742272:12345-192.168.203.56@tcp:4:dd.0 Request processed in 28356us (32546us total) trans 4294974183 rc 0/0 00000100:00100000:0.0:1789760781.650366:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6894 00000100:00100000:0.0:1789760781.650375:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 761140126844 time 761186282837 state 0x2 ref 0 00000100:00100000:1.0:1789760781.653068:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760781.653148:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512742400 00000100:00100000:2.0:1789760781.653158:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512742400 00000100:00100000:2.0:1789760781.653497:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 761140126844 time 761186282837 state 0x0 ref 1 00000100:00100000:2.0:1789760781.653513:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 761186543253 time 761186282837 state 0x1 ref 1 00000100:00100000:2.0:1789760781.653543:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 761186543253 time 761217843071 state 0x0 ref 1 00000100:00100000:2.0:1789760781.653555:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2878000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512742400:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760781.653601:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28119040, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0xd807 00010000:00010000:2.0:1789760781.653616:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2fce1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.653726:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=761s v=(0 0) 00010000:00010000:2.0:1789760781.653731:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760781.653737:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2fce1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28119040->28123135] (req 28119040->28123135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.653752:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2fce1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28119040->28123135] (req 28119040->28123135) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760781.680323:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28119040 -> 28123136 00010000:00010000:1.0:1789760781.680342:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2fce1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28119040->28123135] (req 28119040->28123135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.680356:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2fce1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28119040->28123135] (req 28119040->28123135) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.680366:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2fce1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28119040->28123135] (req 28119040->28123135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.680374:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760781.680378:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2fce1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28119040->28123135] (req 28119040->28123135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.680387:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2fce1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28119040->28123135] (req 28119040->28123135) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.680406:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2fce1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28119040->28123135] (req 28119040->28123135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.680414:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c2fce1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28119040->28123135] (req 28119040->28123135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760781.680496:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2878000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2361:x1876699512742400:12345-192.168.203.56@tcp:4:dd.0 Request processed in 26939us (27433us total) trans 4294974184 rc 0/0 00000100:00100000:1.0:1789760781.680506:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6895 00000100:00100000:1.0:1789760781.680516:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 761186543253 time 761217843071 state 0x2 ref 0 00000001:00080000:0.0:1789760781.681232:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974184 is committed 00000100:00100000:0.0:1789760781.683873:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760781.683988:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512742528 00000100:00100000:3.0:1789760781.683998:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512742528 00000100:00100000:3.0:1789760781.684059:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 761186543253 time 761217843071 state 0x0 ref 1 00000100:00100000:3.0:1789760781.684073:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 761218103487 time 761217843071 state 0x1 ref 1 00000100:00100000:0.0:1789760781.684193:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 761218103487 time 761248490074 state 0x0 ref 1 00000100:00100000:0.0:1789760781.684207:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac38b1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512742528:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760781.684255:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28123136, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0xd807 00010000:00010000:0.0:1789760781.684265:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2fce8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.684280:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=761s v=(0 0) 00010000:00010000:0.0:1789760781.684285:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760781.684291:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2fce8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28123136->28127231] (req 28123136->28127231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.684304:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2fce8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28123136->28127231] (req 28123136->28127231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.688668:0:5836:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:2.0:1789760781.719317:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974185 is committed 00002000:00010000:1.0:1789760781.719499:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28123136 -> 28127232 00010000:00010000:1.0:1789760781.719515:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2fce8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28123136->28127231] (req 28123136->28127231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.719529:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2fce8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28123136->28127231] (req 28123136->28127231) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.719539:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2fce8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28123136->28127231] (req 28123136->28127231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.719547:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760781.719553:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2fce8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28123136->28127231] (req 28123136->28127231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.719563:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2fce8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28123136->28127231] (req 28123136->28127231) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.719582:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2fce8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28123136->28127231] (req 28123136->28127231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.719591:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2fce8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28123136->28127231] (req 28123136->28127231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760781.719869:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac38b1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512742528:12345-192.168.203.56@tcp:4:dd.0 Request processed in 35660us (36000us total) trans 4294974185 rc 0/0 00000100:00100000:1.0:1789760781.719881:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6896 00000100:00100000:1.0:1789760781.719893:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 761218103487 time 761248490074 state 0x2 ref 0 00000100:00100000:2.0:1789760781.722646:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760781.722752:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512742656 00000100:00100000:0.0:1789760781.722762:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512742656 00000100:00100000:0.0:1789760781.722815:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 761218103487 time 761248490074 state 0x0 ref 1 00000100:00100000:0.0:1789760781.722830:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 761248750490 time 761248490074 state 0x1 ref 1 00000100:00100000:0.0:1789760781.722858:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 761248750490 time 761287157929 state 0x0 ref 1 00000100:00100000:0.0:1789760781.722872:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac403b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512742656:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760781.722919:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28127232, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0xd807 00010000:00010000:0.0:1789760781.722928:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2fcef lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.722942:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=761s v=(0 0) 00010000:00010000:0.0:1789760781.722947:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760781.722953:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2fcef lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28127232->28131327] (req 28127232->28131327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.722968:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2fcef lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28127232->28131327] (req 28127232->28131327) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760781.756917:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974186 is committed 00002000:00010000:1.0:1789760781.757021:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28127232 -> 28131328 00010000:00010000:1.0:1789760781.757046:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2fcef lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28127232->28131327] (req 28127232->28131327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.757059:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2fcef lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28127232->28131327] (req 28127232->28131327) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.757068:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2fcef lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28127232->28131327] (req 28127232->28131327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.757076:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760781.757080:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2fcef lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28127232->28131327] (req 28127232->28131327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.757088:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2fcef lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28127232->28131327] (req 28127232->28131327) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.757109:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2fcef lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28127232->28131327] (req 28127232->28131327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.757117:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2fcef lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28127232->28131327] (req 28127232->28131327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760781.757242:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac403b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512742656:12345-192.168.203.56@tcp:4:dd.0 Request processed in 34368us (34600us total) trans 4294974186 rc 0/0 00000100:00100000:1.0:1789760781.757253:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6897 00000100:00100000:1.0:1789760781.757262:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 761248750490 time 761287157929 state 0x2 ref 0 00000100:00100000:2.0:1789760781.764123:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760781.764201:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512742784 00000100:00100000:1.0:1789760781.764211:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512742784 00000100:00100000:1.0:1789760781.764263:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 761248750490 time 761287157929 state 0x0 ref 1 00000100:00100000:1.0:1789760781.764278:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 761287418345 time 761287157929 state 0x1 ref 1 00000100:00100000:1.0:1789760781.764307:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 761287418345 time 761328606010 state 0x0 ref 1 00000100:00100000:1.0:1789760781.764319:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac403a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512742784:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760781.764367:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28131328, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0xd807 00010000:00010000:1.0:1789760781.764377:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2fcf6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.764392:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=761s v=(0 0) 00010000:00010000:1.0:1789760781.764397:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760781.764403:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2fcf6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28131328->28135423] (req 28131328->28135423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.764417:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2fcf6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28131328->28135423] (req 28131328->28135423) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760781.814243:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974187 is committed 00002000:00010000:1.0:1789760781.814328:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28131328 -> 28135424 00010000:00010000:1.0:1789760781.814343:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2fcf6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28131328->28135423] (req 28131328->28135423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.814362:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2fcf6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28131328->28135423] (req 28131328->28135423) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.814371:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2fcf6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28131328->28135423] (req 28131328->28135423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.814377:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760781.814382:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2fcf6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28131328->28135423] (req 28131328->28135423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.814391:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2fcf6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28131328->28135423] (req 28131328->28135423) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.814410:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2fcf6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28131328->28135423] (req 28131328->28135423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.814418:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2fcf6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28131328->28135423] (req 28131328->28135423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760781.814505:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac403a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512742784:12345-192.168.203.56@tcp:4:dd.0 Request processed in 50184us (50387us total) trans 4294974187 rc 0/0 00000100:00100000:1.0:1789760781.814515:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6898 00000100:00100000:1.0:1789760781.814524:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 761287418345 time 761328606010 state 0x2 ref 0 00000100:00100000:0.0:1789760781.819207:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760781.819498:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512742912 00000100:00100000:1.0:1789760781.819508:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512742912 00000100:00100000:1.0:1789760781.819559:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 761287418345 time 761328606010 state 0x0 ref 1 00000100:00100000:1.0:1789760781.819575:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 761328866426 time 761328606010 state 0x1 ref 1 00000100:00100000:1.0:1789760781.819597:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 761328866426 time 761383897528 state 0x0 ref 1 00000100:00100000:1.0:1789760781.819609:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac38b0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512742912:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760781.819660:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28135424, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0xd807 00010000:00010000:1.0:1789760781.819669:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2fcfd lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.819684:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=761s v=(0 0) 00010000:00010000:1.0:1789760781.819688:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760781.819726:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2fcfd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28135424->28139519] (req 28135424->28139519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.819740:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2fcfd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28135424->28139519] (req 28135424->28139519) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760781.865224:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974188 is committed 00002000:00010000:3.0:1789760781.865328:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28135424 -> 28139520 00010000:00010000:3.0:1789760781.865344:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2fcfd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28135424->28139519] (req 28135424->28139519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.865357:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2fcfd lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28135424->28139519] (req 28135424->28139519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.865366:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2fcfd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28135424->28139519] (req 28135424->28139519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.865375:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760781.865379:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2fcfd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28135424->28139519] (req 28135424->28139519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.865389:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2fcfd lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28135424->28139519] (req 28135424->28139519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.865416:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2fcfd lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28135424->28139519] (req 28135424->28139519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.865424:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2fcfd lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28135424->28139519] (req 28135424->28139519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760781.865507:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac38b0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512742912:12345-192.168.203.56@tcp:4:dd.0 Request processed in 45895us (46304us total) trans 4294974188 rc 0/0 00000100:00100000:3.0:1789760781.865518:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6899 00000100:00100000:3.0:1789760781.865527:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 761328866426 time 761383897528 state 0x2 ref 0 00000100:00100000:2.0:1789760781.870976:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760781.871660:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512743040 00000100:00100000:1.0:1789760781.871672:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512743040 00000100:00100000:1.0:1789760781.871761:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 761328866426 time 761383897528 state 0x0 ref 1 00000100:00100000:1.0:1789760781.871777:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 761384157944 time 761383897528 state 0x1 ref 1 00000100:00100000:1.0:1789760781.871810:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 761384157944 time 761436109114 state 0x0 ref 1 00000100:00100000:1.0:1789760781.871826:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4038700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512743040:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760781.871874:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28139520, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0xd807 00010000:00010000:1.0:1789760781.871883:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2fd04 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.871899:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=761s v=(0 0) 00010000:00010000:1.0:1789760781.871904:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760781.871910:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2fd04 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28139520->28143615] (req 28139520->28143615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.871924:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2fd04 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28139520->28143615] (req 28139520->28143615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760781.906860:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974189 is committed 00002000:00010000:0.0:1789760781.906888:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28139520 -> 28143616 00010000:00010000:0.0:1789760781.906903:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2fd04 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28139520->28143615] (req 28139520->28143615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.906914:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2fd04 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28139520->28143615] (req 28139520->28143615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.906923:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2fd04 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28139520->28143615] (req 28139520->28143615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.906930:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760781.906935:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2fd04 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28139520->28143615] (req 28139520->28143615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.906945:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2fd04 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28139520->28143615] (req 28139520->28143615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.906965:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2fd04 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28139520->28143615] (req 28139520->28143615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.906978:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2fd04 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28139520->28143615] (req 28139520->28143615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760781.907195:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4038700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512743040:12345-192.168.203.56@tcp:4:dd.0 Request processed in 35366us (36222us total) trans 4294974189 rc 0/0 00000100:00100000:0.0:1789760781.907205:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6900 00000100:00100000:0.0:1789760781.907215:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 761384157944 time 761436109114 state 0x2 ref 0 00000100:00100000:2.0:1789760781.911411:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760781.911973:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512743168 00000100:00100000:1.0:1789760781.911983:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512743168 00000100:00100000:1.0:1789760781.912052:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 761384157944 time 761436109114 state 0x0 ref 1 00000100:00100000:1.0:1789760781.912067:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 761436369530 time 761436109114 state 0x1 ref 1 00000100:00100000:1.0:1789760781.912131:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 761436369530 time 761476429888 state 0x0 ref 1 00000100:00100000:1.0:1789760781.912143:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4039f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512743168:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760781.912190:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28143616, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0xd807 00010000:00010000:1.0:1789760781.912198:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2fd0b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.912219:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=761s v=(0 0) 00010000:00010000:1.0:1789760781.912223:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760781.912229:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2fd0b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28143616->28147711] (req 28143616->28147711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.912247:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2fd0b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28143616->28147711] (req 28143616->28147711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760781.941836:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974190 is committed 00002000:00010000:1.0:1789760781.941973:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28143616 -> 28147712 00010000:00010000:1.0:1789760781.941989:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2fd0b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28143616->28147711] (req 28143616->28147711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.942002:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2fd0b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28143616->28147711] (req 28143616->28147711) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.942011:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2fd0b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28143616->28147711] (req 28143616->28147711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.942019:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760781.942025:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2fd0b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28143616->28147711] (req 28143616->28147711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.942045:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2fd0b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28143616->28147711] (req 28143616->28147711) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.942066:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2fd0b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28143616->28147711] (req 28143616->28147711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.942074:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2fd0b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28143616->28147711] (req 28143616->28147711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760781.942163:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4039f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512743168:12345-192.168.203.56@tcp:4:dd.0 Request processed in 30017us (30768us total) trans 4294974190 rc 0/0 00000100:00100000:1.0:1789760781.942174:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6901 00000100:00100000:1.0:1789760781.942183:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 761436369530 time 761476429888 state 0x2 ref 0 00000100:00100000:1.0:1789760781.945294:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760781.945411:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512743296 00000100:00100000:3.0:1789760781.945422:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512743296 00000100:00100000:3.0:1789760781.945580:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 761436369530 time 761476429888 state 0x0 ref 1 00000100:00100000:3.0:1789760781.945596:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 761476690304 time 761476429888 state 0x1 ref 1 00000100:00100000:3.0:1789760781.945829:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 761476690304 time 761510127116 state 0x0 ref 1 00000100:00100000:3.0:1789760781.945845:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2879180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512743296:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760781.945890:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28147712, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0xd807 00010000:00010000:3.0:1789760781.945899:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2fd12 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.945914:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=761s v=(0 0) 00010000:00010000:3.0:1789760781.945919:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760781.945926:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2fd12 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28147712->28151807] (req 28147712->28151807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.945940:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2fd12 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28147712->28151807] (req 28147712->28151807) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760781.987771:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28147712 -> 28151808 00010000:00010000:2.0:1789760781.987790:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2fd12 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28147712->28151807] (req 28147712->28151807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.987801:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2fd12 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28147712->28151807] (req 28147712->28151807) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.987809:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2fd12 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28147712->28151807] (req 28147712->28151807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.987816:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760781.987821:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2fd12 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28147712->28151807] (req 28147712->28151807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.987830:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2fd12 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28147712->28151807] (req 28147712->28151807) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.987851:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2fd12 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28147712->28151807] (req 28147712->28151807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.987864:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2fd12 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28147712->28151807] (req 28147712->28151807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760781.987980:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2879180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2360:x1876699512743296:12345-192.168.203.56@tcp:4:dd.0 Request processed in 42133us (42688us total) trans 4294974191 rc 0/0 00000100:00100000:2.0:1789760781.987990:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6902 00000100:00100000:2.0:1789760781.988009:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 761476690304 time 761510127116 state 0x2 ref 0 00000001:00080000:1.0:1789760781.988839:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974191 is committed 00000100:00100000:2.0:1789760781.994556:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760781.994653:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512743424 00000100:00100000:0.0:1789760781.994663:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512743424 00000100:00100000:0.0:1789760781.994817:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 761476690304 time 761510127116 state 0x0 ref 1 00000100:00100000:0.0:1789760781.994832:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 761510387532 time 761510127116 state 0x1 ref 1 00000100:00100000:0.0:1789760781.994854:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 761510387532 time 761559154020 state 0x0 ref 1 00000100:00100000:0.0:1789760781.994865:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4038000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512743424:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760781.994911:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28151808, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0xd807 00010000:00010000:0.0:1789760781.994920:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2fd19 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.994935:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=761s v=(0 0) 00010000:00010000:0.0:1789760781.994940:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760781.994946:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2fd19 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28151808->28155903] (req 28151808->28155903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.994959:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2fd19 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28151808->28155903] (req 28151808->28155903) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760782.030217:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28151808 -> 28155904 00010000:00010000:0.0:1789760782.030237:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2fd19 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28151808->28155903] (req 28151808->28155903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.030252:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2fd19 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28151808->28155903] (req 28151808->28155903) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.030261:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2fd19 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28151808->28155903] (req 28151808->28155903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.030270:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760782.030275:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2fd19 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28151808->28155903] (req 28151808->28155903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.030284:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2fd19 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28151808->28155903] (req 28151808->28155903) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.030301:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2fd19 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28151808->28155903] (req 28151808->28155903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.030308:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2fd19 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28151808->28155903] (req 28151808->28155903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760782.030402:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4038000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2361:x1876699512743424:12345-192.168.203.56@tcp:4:dd.0 Request processed in 35533us (35850us total) trans 4294974192 rc 0/0 00000100:00100000:0.0:1789760782.030412:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6903 00000100:00100000:0.0:1789760782.030421:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 761510387532 time 761559154020 state 0x2 ref 0 00000001:00080000:1.0:1789760782.030488:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974192 is committed 00000100:00100000:0.0:1789760782.033577:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760782.033680:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512743552 00000100:00100000:1.0:1789760782.033689:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512743552 00000100:00100000:1.0:1789760782.033770:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 761510387532 time 761559154020 state 0x0 ref 1 00000100:00100000:1.0:1789760782.033784:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 761559414436 time 761559154020 state 0x1 ref 1 00000100:00100000:1.0:1789760782.033807:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 761559414436 time 761598106971 state 0x0 ref 1 00000100:00100000:1.0:1789760782.033819:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac38b2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512743552:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760782.033866:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28155904, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0xd807 00010000:00010000:1.0:1789760782.033874:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2fd20 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.033891:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=761s v=(0 0) 00010000:00010000:1.0:1789760782.033896:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760782.033903:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2fd20 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28155904->28159999] (req 28155904->28159999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.033928:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2fd20 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28155904->28159999] (req 28155904->28159999) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760782.074102:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974193 is committed 00002000:00010000:0.0:1789760782.074206:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28155904 -> 28160000 00010000:00010000:0.0:1789760782.074222:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2fd20 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28155904->28159999] (req 28155904->28159999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.074236:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2fd20 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28155904->28159999] (req 28155904->28159999) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.074245:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2fd20 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28155904->28159999] (req 28155904->28159999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.074253:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760782.074259:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2fd20 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28155904->28159999] (req 28155904->28159999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.074268:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2fd20 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28155904->28159999] (req 28155904->28159999) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.074286:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2fd20 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28155904->28159999] (req 28155904->28159999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.074298:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2fd20 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28155904->28159999] (req 28155904->28159999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760782.074390:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac38b2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512743552:12345-192.168.203.56@tcp:4:dd.0 Request processed in 40567us (40817us total) trans 4294974193 rc 0/0 00000100:00100000:0.0:1789760782.074400:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6904 00000100:00100000:0.0:1789760782.074410:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 761559414436 time 761598106971 state 0x2 ref 0 00000100:00100000:2.0:1789760782.080544:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760782.080673:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512743680 00000100:00100000:3.0:1789760782.080683:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512743680 00000100:00100000:3.0:1789760782.080753:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 761559414436 time 761598106971 state 0x0 ref 1 00000100:00100000:3.0:1789760782.080769:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 761598367387 time 761598106971 state 0x1 ref 1 00000100:00100000:3.0:1789760782.080801:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 761598367387 time 761645100684 state 0x0 ref 1 00000100:00100000:3.0:1789760782.080814:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febff932a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512743680:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760782.080857:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28160000, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0xd807 00010000:00010000:3.0:1789760782.080867:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2fd27 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.080883:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=761s v=(0 0) 00010000:00010000:3.0:1789760782.080887:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760782.080894:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2fd27 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28160000->28164095] (req 28160000->28164095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.080908:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2fd27 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28160000->28164095] (req 28160000->28164095) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760782.130458:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974194 is committed 00002000:00010000:0.0:1789760782.130920:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28160000 -> 28164096 00010000:00010000:0.0:1789760782.130938:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2fd27 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28160000->28164095] (req 28160000->28164095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.130951:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2fd27 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28160000->28164095] (req 28160000->28164095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.130960:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2fd27 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28160000->28164095] (req 28160000->28164095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.130968:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760782.130973:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2fd27 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28160000->28164095] (req 28160000->28164095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.130982:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2fd27 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28160000->28164095] (req 28160000->28164095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.131007:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2fd27 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28160000->28164095] (req 28160000->28164095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.131015:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2fd27 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28160000->28164095] (req 28160000->28164095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760782.131189:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febff932a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512743680:12345-192.168.203.56@tcp:4:dd.0 Request processed in 50371us (50645us total) trans 4294974194 rc 0/0 00000100:00100000:0.0:1789760782.131200:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6905 00000100:00100000:0.0:1789760782.131211:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 761598367387 time 761645100684 state 0x2 ref 0 00000100:00100000:2.0:1789760782.136021:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760782.136088:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512743808 00000100:00100000:0.0:1789760782.136099:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512743808 00000100:00100000:0.0:1789760782.136158:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 761598367387 time 761645100684 state 0x0 ref 1 00000100:00100000:0.0:1789760782.136186:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 761645361100 time 761645100684 state 0x1 ref 1 00000100:00100000:0.0:1789760782.136220:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 761645361100 time 761700519325 state 0x0 ref 1 00000100:00100000:0.0:1789760782.136233:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febff930e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512743808:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760782.136283:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28164096, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0xd807 00010000:00010000:0.0:1789760782.136292:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2fd2e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.136308:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=761s v=(0 0) 00010000:00010000:0.0:1789760782.136314:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760782.136320:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2fd2e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28164096->28168191] (req 28164096->28168191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.136334:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2fd2e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28164096->28168191] (req 28164096->28168191) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760782.188434:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28164096 -> 28168192 00010000:00010000:0.0:1789760782.188457:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2fd2e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28164096->28168191] (req 28164096->28168191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.188470:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2fd2e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28164096->28168191] (req 28164096->28168191) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.188491:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2fd2e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28164096->28168191] (req 28164096->28168191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.188499:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760782.188505:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2fd2e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28164096->28168191] (req 28164096->28168191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.188515:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2fd2e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28164096->28168191] (req 28164096->28168191) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.188534:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2fd2e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28164096->28168191] (req 28164096->28168191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.188542:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2fd2e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28164096->28168191] (req 28164096->28168191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760782.188633:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febff930e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2358:x1876699512743808:12345-192.168.203.56@tcp:4:dd.0 Request processed in 52397us (52616us total) trans 4294974195 rc 0/0 00000100:00100000:0.0:1789760782.188642:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6906 00000100:00100000:0.0:1789760782.188651:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 761645361100 time 761700519325 state 0x2 ref 0 00000001:00080000:3.0:1789760782.189414:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974195 is committed 00000100:00100000:2.0:1789760782.193037:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760782.193197:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512743936 00000100:00100000:0.0:1789760782.193207:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512743936 00000100:00100000:0.0:1789760782.193265:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 761645361100 time 761700519325 state 0x0 ref 1 00000100:00100000:0.0:1789760782.193281:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 761700779741 time 761700519325 state 0x1 ref 1 00000100:00100000:0.0:1789760782.193300:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 761700779741 time 761757600666 state 0x0 ref 1 00000100:00100000:0.0:1789760782.193312:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac3351180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512743936:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760782.193358:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28168192, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0xd807 00010000:00010000:0.0:1789760782.193367:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2fd35 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.193381:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=761s v=(0 0) 00010000:00010000:0.0:1789760782.193386:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760782.193392:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2fd35 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28168192->28172287] (req 28168192->28172287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.193406:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2fd35 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28168192->28172287] (req 28168192->28172287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760782.223441:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974196 is committed 00002000:00010000:0.0:1789760782.223564:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28168192 -> 28172288 00010000:00010000:0.0:1789760782.223580:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2fd35 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28168192->28172287] (req 28168192->28172287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.223593:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2fd35 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28168192->28172287] (req 28168192->28172287) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.223603:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2fd35 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28168192->28172287] (req 28168192->28172287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.223610:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760782.223616:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2fd35 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28168192->28172287] (req 28168192->28172287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.223625:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2fd35 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28168192->28172287] (req 28168192->28172287) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.223645:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2fd35 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28168192->28172287] (req 28168192->28172287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.223653:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2fd35 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28168192->28172287] (req 28168192->28172287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760782.223876:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac3351180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512743936:12345-192.168.203.56@tcp:4:dd.0 Request processed in 30559us (30850us total) trans 4294974196 rc 0/0 00000100:00100000:0.0:1789760782.223893:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6907 00000100:00100000:0.0:1789760782.223906:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 761700779741 time 761757600666 state 0x2 ref 0 00000100:00100000:2.0:1789760782.228333:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760782.228410:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512744064 00000100:00100000:0.0:1789760782.228419:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512744064 00000100:00100000:0.0:1789760782.228474:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 761700779741 time 761757600666 state 0x0 ref 1 00000100:00100000:0.0:1789760782.228489:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 761757861082 time 761757600666 state 0x1 ref 1 00000100:00100000:0.0:1789760782.228525:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 761757861082 time 761792823968 state 0x0 ref 1 00000100:00100000:0.0:1789760782.228538:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac3352a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512744064:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760782.228586:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28172288, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0xd807 00010000:00010000:0.0:1789760782.228596:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2fd3c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.228611:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=761s v=(0 0) 00010000:00010000:0.0:1789760782.228616:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760782.228622:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2fd3c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28172288->28176383] (req 28172288->28176383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.228636:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2fd3c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28172288->28176383] (req 28172288->28176383) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760782.259990:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974197 is committed 00002000:00010000:0.0:1789760782.260160:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28172288 -> 28176384 00010000:00010000:0.0:1789760782.260176:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2fd3c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28172288->28176383] (req 28172288->28176383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.260189:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2fd3c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28172288->28176383] (req 28172288->28176383) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.260199:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2fd3c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28172288->28176383] (req 28172288->28176383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.260206:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760782.260212:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2fd3c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28172288->28176383] (req 28172288->28176383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.260221:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2fd3c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28172288->28176383] (req 28172288->28176383) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.260240:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2fd3c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28172288->28176383] (req 28172288->28176383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.260247:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2fd3c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28172288->28176383] (req 28172288->28176383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760782.260343:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac3352a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512744064:12345-192.168.203.56@tcp:4:dd.0 Request processed in 31803us (32013us total) trans 4294974197 rc 0/0 00000100:00100000:0.0:1789760782.260353:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6908 00000100:00100000:0.0:1789760782.260363:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 761757861082 time 761792823968 state 0x2 ref 0 00000100:00100000:2.0:1789760782.260989:0:3310:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8feac2879500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3310:1876699540231296:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789760782.261054:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760782.261159:0:6706:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540231296 00000100:00100000:1.0:1789760782.261168:0:6706:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540231296 00000100:00100000:1.0:1789760782.261226:0:6706:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 212 00000100:00100000:1.0:1789760782.261235:0:6706:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac3353480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3310:x1876699540231296:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1789760782.261256:0:6706:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 526712832 pending 0 free 3830448128 avail 3786407936 00000100:00100000:1.0:1789760782.261296:0:6706:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac3353480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3310:x1876699540231296:12345-0@lo:13:osp-pre-0-0.0 Request processed in 63us (242us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760782.261303:0:6706:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 212 00000100:00080000:2.0:1789760782.261363:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1789760782.261399:0:3310:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8feac2879500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3310:1876699540231296:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1789760782.261884:0:3309:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8feac2878380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3309:1876699540231680:0@lo:400:kworker.0 00000100:00100000:1.0:1789760782.261921:0:3309:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760782.261948:0:3309:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1789760782.261954:0:3309:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8feac287a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3309:1876699540231808:0@lo:400:kworker.0 00000100:00100000:1.0:1789760782.261975:0:3309:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760782.262011:0:5848:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540231680 00000100:00100000:1.0:1789760782.262016:0:5848:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540231680 00000100:00100000:1.0:1789760782.262059:0:5848:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540231808 00000100:00100000:1.0:1789760782.262062:0:5848:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540231808 00000100:00100000:1.0:1789760782.262085:0:5848:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 511 00000100:00100000:1.0:1789760782.262093:0:5848:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2878e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3309:x1876699540231680:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760782.262132:0:5848:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2878e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3309:x1876699540231680:12345-0@lo:400:kworker.0 Request processed in 40us (210us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760782.262139:0:5848:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 511 00000100:00100000:1.0:1789760782.262152:0:5848:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 512 00000100:00100000:1.0:1789760782.262158:0:5848:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2879500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3309:x1876699540231808:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760782.262185:0:5848:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2879500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3309:x1876699540231808:12345-0@lo:400:kworker.0 Request processed in 27us (209us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760782.262190:0:5848:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 512 00000100:00100000:1.0:1789760782.262229:0:3309:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8feac287a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3309:1876699540231808:0@lo:400:kworker.0 00000100:00100000:1.0:1789760782.262240:0:3309:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8feac2878380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3309:1876699540231680:0@lo:400:kworker.0 00000100:00100000:0.0:1789760782.262300:0:3312:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8feac287bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3312:1876699540231552:0@lo:400:kworker.0 00000100:00100000:0.0:1789760782.262337:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760782.262364:0:3312:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [2->3] 00000100:00100000:0.0:1789760782.262369:0:3312:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8feac287b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:6df98d54-eb52-4c5c-8fc9-ee9d74f90f98:3312:1876699540231424:0@lo:400:kworker.0 00000100:00100000:0.0:1789760782.262398:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760782.262418:0:3312:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8feac2879c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3312:1876699540231936:0@lo:400:kworker.0 00000100:00100000:0.0:1789760782.262440:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760782.262540:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540231424 00000100:00100000:1.0:1789760782.262545:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540231424 00000100:00100000:1.0:1789760782.262583:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 413 00000100:00100000:1.0:1789760782.262591:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febe00c5c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:6df98d54-eb52-4c5c-8fc9-ee9d74f90f98+19:3312:x1876699540231424:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760782.262633:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febe00c5c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:6df98d54-eb52-4c5c-8fc9-ee9d74f90f98+19:3312:x1876699540231424:12345-0@lo:400:kworker.0 Request processed in 42us (233us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760782.262639:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 413 00000100:00100000:1.0:1789760782.262663:0:5848:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540231552 00000100:00100000:1.0:1789760782.262667:0:5848:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540231552 00000100:00100000:1.0:1789760782.262688:0:5848:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 513 00000100:00100000:0.0:1789760782.262800:0:3312:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8feac287b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:6df98d54-eb52-4c5c-8fc9-ee9d74f90f98:3312:1876699540231424:0@lo:400:kworker.0 00000100:00100000:1.0:1789760782.262878:0:5848:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febfc0a0700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3312:x1876699540231552:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760782.262924:0:5848:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febfc0a0700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3312:x1876699540231552:12345-0@lo:400:kworker.0 Request processed in 230us (586us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760782.262931:0:5848:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 513 00000100:00100000:0.0:1789760782.262990:0:3312:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8feac287bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3312:1876699540231552:0@lo:400:kworker.0 00000100:00100000:2.0:1789760782.263066:0:6704:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540231936 00000100:00100000:2.0:1789760782.263071:0:6704:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540231936 00000100:00100000:2.0:1789760782.263142:0:6704:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 193 00000100:00100000:2.0:1789760782.263150:0:6704:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febfc0a3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3312:x1876699540231936:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760782.263197:0:6704:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febfc0a3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3312:x1876699540231936:12345-0@lo:400:kworker.0 Request processed in 49us (755us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760782.263204:0:6704:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 193 00000100:00100000:0.0:1789760782.263251:0:3312:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8feac2879c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3312:1876699540231936:0@lo:400:kworker.0 00000100:00080000:2.0:1789760782.263312:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:2.0:1789760782.263317:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789760782.267904:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760782.267985:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512744192 00000100:00100000:0.0:1789760782.267994:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512744192 00000100:00100000:0.0:1789760782.268064:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 761757861082 time 761792823968 state 0x0 ref 1 00000100:00100000:0.0:1789760782.268080:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 761793084384 time 761792823968 state 0x1 ref 1 00000100:00100000:0.0:1789760782.268112:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 761793084384 time 761832411582 state 0x0 ref 1 00000100:00100000:0.0:1789760782.268124:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac3352300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512744192:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760782.268170:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28176384, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0xd807 00010000:00010000:0.0:1789760782.268179:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2fd43 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.268195:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=761s v=(0 0) 00010000:00010000:0.0:1789760782.268199:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760782.268206:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2fd43 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28176384->28180479] (req 28176384->28180479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.268221:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2fd43 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28176384->28180479] (req 28176384->28180479) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760782.322676:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974198 is committed 00002000:00010000:1.0:1789760782.322884:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28176384 -> 28180480 00010000:00010000:1.0:1789760782.322900:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2fd43 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28176384->28180479] (req 28176384->28180479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.322913:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2fd43 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28176384->28180479] (req 28176384->28180479) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.322922:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2fd43 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28176384->28180479] (req 28176384->28180479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.322929:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760782.322934:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2fd43 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28176384->28180479] (req 28176384->28180479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.322944:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2fd43 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28176384->28180479] (req 28176384->28180479) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.322964:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2fd43 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28176384->28180479] (req 28176384->28180479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.322972:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2fd43 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28176384->28180479] (req 28176384->28180479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760782.323077:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac3352300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512744192:12345-192.168.203.56@tcp:4:dd.0 Request processed in 54950us (55178us total) trans 4294974198 rc 0/0 00000100:00100000:1.0:1789760782.323087:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6909 00000100:00100000:1.0:1789760782.323096:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 761793084384 time 761832411582 state 0x2 ref 0 00000100:00100000:2.0:1789760782.330629:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760782.330973:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512744320 00000100:00100000:0.0:1789760782.330983:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512744320 00000100:00100000:0.0:1789760782.331049:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 761793084384 time 761832411582 state 0x0 ref 1 00000100:00100000:0.0:1789760782.331063:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 761832671998 time 761832411582 state 0x1 ref 1 00000100:00100000:0.0:1789760782.331136:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 761832671998 time 761895434119 state 0x0 ref 1 00000100:00100000:0.0:1789760782.331150:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac3353800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512744320:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760782.331212:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28180480, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0xd807 00010000:00010000:0.0:1789760782.331221:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2fd4a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.331237:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=761s v=(0 0) 00010000:00010000:0.0:1789760782.331243:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760782.331250:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2fd4a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28180480->28184575] (req 28180480->28184575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.331263:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2fd4a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28180480->28184575] (req 28180480->28184575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760782.365366:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974199 is committed 00002000:00010000:1.0:1789760782.365477:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28180480 -> 28184576 00010000:00010000:1.0:1789760782.365492:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2fd4a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28180480->28184575] (req 28180480->28184575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.365505:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2fd4a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28180480->28184575] (req 28180480->28184575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.365513:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2fd4a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28180480->28184575] (req 28180480->28184575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.365520:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760782.365525:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2fd4a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28180480->28184575] (req 28180480->28184575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.365533:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2fd4a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28180480->28184575] (req 28180480->28184575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.365551:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2fd4a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28180480->28184575] (req 28180480->28184575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.365559:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2fd4a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28180480->28184575] (req 28180480->28184575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760782.365682:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac3353800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512744320:12345-192.168.203.56@tcp:4:dd.0 Request processed in 34530us (35058us total) trans 4294974199 rc 0/0 00000100:00100000:1.0:1789760782.365692:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6910 00000100:00100000:1.0:1789760782.365754:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 761832671998 time 761895434119 state 0x2 ref 0 00000100:00100000:2.0:1789760782.370351:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760782.370437:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512744448 00000100:00100000:0.0:1789760782.370447:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512744448 00000100:00100000:0.0:1789760782.370513:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 761832671998 time 761895434119 state 0x0 ref 1 00000100:00100000:0.0:1789760782.370529:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 761895694535 time 761895434119 state 0x1 ref 1 00000100:00100000:0.0:1789760782.370558:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 761895694535 time 761934857059 state 0x0 ref 1 00000100:00100000:0.0:1789760782.370570:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac38b6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512744448:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760782.370627:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28184576, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0xd807 00010000:00010000:0.0:1789760782.370635:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2fd51 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.370650:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=761s v=(0 0) 00010000:00010000:0.0:1789760782.370654:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760782.370660:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2fd51 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28184576->28188671] (req 28184576->28188671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.370673:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2fd51 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28184576->28188671] (req 28184576->28188671) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760782.409422:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974200 is committed 00002000:00010000:2.0:1789760782.409605:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28184576 -> 28188672 00010000:00010000:2.0:1789760782.409621:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2fd51 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28184576->28188671] (req 28184576->28188671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.409634:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2fd51 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28184576->28188671] (req 28184576->28188671) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.409642:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2fd51 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28184576->28188671] (req 28184576->28188671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.409650:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760782.409655:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2fd51 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28184576->28188671] (req 28184576->28188671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.409664:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2fd51 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28184576->28188671] (req 28184576->28188671) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.409683:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2fd51 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28184576->28188671] (req 28184576->28188671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.409820:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2fd51 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28184576->28188671] (req 28184576->28188671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760782.409947:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac38b6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512744448:12345-192.168.203.56@tcp:4:dd.0 Request processed in 39375us (39600us total) trans 4294974200 rc 0/0 00000100:00100000:2.0:1789760782.409957:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6911 00000100:00100000:2.0:1789760782.409967:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 761895694535 time 761934857059 state 0x2 ref 0 00000100:00100000:2.0:1789760782.413311:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760782.413478:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512744576 00000100:00100000:1.0:1789760782.413489:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512744576 00000100:00100000:1.0:1789760782.413540:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 761895694535 time 761934857059 state 0x0 ref 1 00000100:00100000:1.0:1789760782.413555:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 761935117475 time 761934857059 state 0x1 ref 1 00000100:00100000:1.0:1789760782.413668:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 761935117475 time 761977966248 state 0x0 ref 1 00000100:00100000:1.0:1789760782.413681:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac38b4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512744576:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760782.413876:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28188672, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0xd807 00010000:00010000:1.0:1789760782.413886:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2fd58 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.413902:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=761s v=(0 0) 00010000:00010000:1.0:1789760782.413906:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760782.413912:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2fd58 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28188672->28192767] (req 28188672->28192767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.413925:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2fd58 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28188672->28192767] (req 28188672->28192767) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760782.440661:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974201 is committed 00002000:00010000:2.0:1789760782.440860:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28188672 -> 28192768 00010000:00010000:2.0:1789760782.440876:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2fd58 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28188672->28192767] (req 28188672->28192767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.440889:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2fd58 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28188672->28192767] (req 28188672->28192767) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.440897:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2fd58 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28188672->28192767] (req 28188672->28192767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.440905:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760782.440910:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2fd58 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28188672->28192767] (req 28188672->28192767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.440919:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2fd58 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28188672->28192767] (req 28188672->28192767) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.440944:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2fd58 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28188672->28192767] (req 28188672->28192767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.440952:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2fd58 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28188672->28192767] (req 28188672->28192767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760782.441233:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac38b4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512744576:12345-192.168.203.56@tcp:4:dd.0 Request processed in 27549us (27924us total) trans 4294974201 rc 0/0 00000100:00100000:2.0:1789760782.441244:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6912 00000100:00100000:2.0:1789760782.441253:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 761935117475 time 761977966248 state 0x2 ref 0 00000100:00100000:0.0:1789760782.447861:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760782.447994:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512744704 00000100:00100000:1.0:1789760782.448008:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512744704 00000100:00100000:1.0:1789760782.448072:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 761935117475 time 761977966248 state 0x0 ref 1 00000100:00100000:1.0:1789760782.448087:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 761978226664 time 761977966248 state 0x1 ref 1 00000100:00100000:1.0:1789760782.448128:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 761978226664 time 762012420779 state 0x0 ref 1 00000100:00100000:1.0:1789760782.448140:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac28f2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512744704:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760782.448191:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28192768, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0xd807 00010000:00010000:1.0:1789760782.448200:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2fd5f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.448216:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=762s v=(0 0) 00010000:00010000:1.0:1789760782.448220:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760782.448226:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2fd5f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28192768->28196863] (req 28192768->28196863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.448238:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2fd5f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28192768->28196863] (req 28192768->28196863) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760782.477989:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28192768 -> 28196864 00000001:00080000:2.0:1789760782.477999:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974202 is committed 00010000:00010000:0.0:1789760782.478010:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2fd5f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28192768->28196863] (req 28192768->28196863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.478024:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2fd5f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28192768->28196863] (req 28192768->28196863) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.478049:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2fd5f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28192768->28196863] (req 28192768->28196863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.478056:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760782.478061:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2fd5f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28192768->28196863] (req 28192768->28196863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.478070:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2fd5f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28192768->28196863] (req 28192768->28196863) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.478091:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2fd5f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28192768->28196863] (req 28192768->28196863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.478099:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2fd5f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28192768->28196863] (req 28192768->28196863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760782.478201:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac28f2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512744704:12345-192.168.203.56@tcp:4:dd.0 Request processed in 30058us (30345us total) trans 4294974202 rc 0/0 00000100:00100000:0.0:1789760782.478210:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6913 00000100:00100000:0.0:1789760782.478222:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 761978226664 time 762012420779 state 0x2 ref 0 00000100:00100000:1.0:1789760782.484325:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760782.484490:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512744832 00000100:00100000:2.0:1789760782.484500:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512744832 00000100:00100000:2.0:1789760782.484551:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 761978226664 time 762012420779 state 0x0 ref 1 00000100:00100000:2.0:1789760782.484566:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 762012681195 time 762012420779 state 0x1 ref 1 00000100:00100000:2.0:1789760782.484594:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 762012681195 time 762048893819 state 0x0 ref 1 00000100:00100000:2.0:1789760782.484607:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2878380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512744832:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760782.484654:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28196864, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0xd807 00010000:00010000:2.0:1789760782.484663:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2fd66 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.484683:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=762s v=(0 0) 00010000:00010000:2.0:1789760782.484687:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760782.484693:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2fd66 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28196864->28200959] (req 28196864->28200959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.484744:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2fd66 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28196864->28200959] (req 28196864->28200959) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760782.520867:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974203 is committed 00002000:00010000:1.0:1789760782.520968:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28196864 -> 28200960 00010000:00010000:1.0:1789760782.520983:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2fd66 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28196864->28200959] (req 28196864->28200959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.520995:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2fd66 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28196864->28200959] (req 28196864->28200959) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.521003:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2fd66 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28196864->28200959] (req 28196864->28200959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.521015:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760782.521020:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2fd66 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28196864->28200959] (req 28196864->28200959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.521038:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2fd66 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28196864->28200959] (req 28196864->28200959) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.521056:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2fd66 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28196864->28200959] (req 28196864->28200959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.521064:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2fd66 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28196864->28200959] (req 28196864->28200959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760782.521149:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2878380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512744832:12345-192.168.203.56@tcp:4:dd.0 Request processed in 36540us (36829us total) trans 4294974203 rc 0/0 00000100:00100000:1.0:1789760782.521159:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6914 00000100:00100000:1.0:1789760782.521169:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 762012681195 time 762048893819 state 0x2 ref 0 00000100:00100000:2.0:1789760782.525920:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760782.526105:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512744960 00000100:00100000:1.0:1789760782.526114:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512744960 00000100:00100000:1.0:1789760782.526165:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 762012681195 time 762048893819 state 0x0 ref 1 00000100:00100000:1.0:1789760782.526178:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 762049154235 time 762048893819 state 0x1 ref 1 00000100:00100000:1.0:1789760782.526197:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 762049154235 time 762090497785 state 0x0 ref 1 00000100:00100000:1.0:1789760782.526211:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac38b5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512744960:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760782.526257:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28200960, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0xd807 00010000:00010000:1.0:1789760782.526265:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2fd6d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.526281:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=762s v=(0 0) 00010000:00010000:1.0:1789760782.526285:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760782.526290:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2fd6d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28200960->28205055] (req 28200960->28205055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.526302:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2fd6d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28200960->28205055] (req 28200960->28205055) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760782.555132:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974204 is committed 00002000:00010000:0.0:1789760782.555241:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28200960 -> 28205056 00002000:00010000:0.0:1789760782.555246:0:6709:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760781 -> 1789760782 00002000:00010000:0.0:1789760782.555249:0:6709:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760781 -> 1789760782 00010000:00010000:0.0:1789760782.555261:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2fd6d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28200960->28205055] (req 28200960->28205055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.555275:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2fd6d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28200960->28205055] (req 28200960->28205055) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.555285:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2fd6d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28200960->28205055] (req 28200960->28205055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.555293:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760782.555298:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2fd6d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28200960->28205055] (req 28200960->28205055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.555307:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2fd6d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28200960->28205055] (req 28200960->28205055) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.555325:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2fd6d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28200960->28205055] (req 28200960->28205055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.555333:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c2fd6d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28200960->28205055] (req 28200960->28205055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760782.555426:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac38b5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512744960:12345-192.168.203.56@tcp:4:dd.0 Request processed in 29210us (29510us total) trans 4294974204 rc 0/0 00000100:00100000:0.0:1789760782.555437:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6915 00000100:00100000:0.0:1789760782.555447:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 762049154235 time 762090497785 state 0x2 ref 0 00000100:00100000:2.0:1789760782.559763:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760782.559866:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512745088 00000100:00100000:0.0:1789760782.559877:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512745088 00000100:00100000:0.0:1789760782.559932:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 762049154235 time 762090497785 state 0x0 ref 1 00000100:00100000:0.0:1789760782.559948:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 762090758201 time 762090497785 state 0x1 ref 1 00000100:00100000:0.0:1789760782.559989:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 762090758201 time 762124279988 state 0x0 ref 1 00000100:00100000:0.0:1789760782.560004:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac38b5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512745088:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760782.560064:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28205056, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0xd807 00010000:00010000:0.0:1789760782.560073:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2fd74 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.560088:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=762s v=(0 0) 00010000:00010000:0.0:1789760782.560093:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760782.560099:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2fd74 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28205056->28209151] (req 28205056->28209151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.560112:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2fd74 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28205056->28209151] (req 28205056->28209151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760782.581798:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974205 is committed 00002000:00010000:0.0:1789760782.581967:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28205056 -> 28209152 00010000:00010000:0.0:1789760782.582050:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2fd74 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28205056->28209151] (req 28205056->28209151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.582064:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2fd74 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28205056->28209151] (req 28205056->28209151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.582081:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2fd74 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28205056->28209151] (req 28205056->28209151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.582088:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760782.582093:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2fd74 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28205056->28209151] (req 28205056->28209151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.582102:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2fd74 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28205056->28209151] (req 28205056->28209151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.582122:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2fd74 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28205056->28209151] (req 28205056->28209151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.582131:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2fd74 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28205056->28209151] (req 28205056->28209151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760782.582222:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac38b5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512745088:12345-192.168.203.56@tcp:4:dd.0 Request processed in 22217us (22463us total) trans 4294974205 rc 0/0 00000100:00100000:0.0:1789760782.582232:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6916 00000100:00100000:0.0:1789760782.582241:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 762090758201 time 762124279988 state 0x2 ref 0 00000100:00100000:2.0:1789760782.585588:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760782.585879:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512745216 00000100:00100000:1.0:1789760782.585891:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512745216 00000100:00100000:1.0:1789760782.585946:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 762090758201 time 762124279988 state 0x0 ref 1 00000100:00100000:1.0:1789760782.585968:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 762124540404 time 762124279988 state 0x1 ref 1 00000100:00100000:1.0:1789760782.586095:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 762124540404 time 762150393259 state 0x0 ref 1 00000100:00100000:1.0:1789760782.586108:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac38b6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512745216:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760782.586157:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28209152, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0xd807 00010000:00010000:1.0:1789760782.586166:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2fd7b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.586186:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=762s v=(0 0) 00010000:00010000:1.0:1789760782.586191:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760782.586198:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2fd7b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28209152->28213247] (req 28209152->28213247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.586211:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2fd7b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28209152->28213247] (req 28209152->28213247) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760782.614010:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974206 is committed 00002000:00010000:1.0:1789760782.614128:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28209152 -> 28213248 00010000:00010000:1.0:1789760782.614149:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2fd7b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28209152->28213247] (req 28209152->28213247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.614163:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2fd7b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28209152->28213247] (req 28209152->28213247) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.614172:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2fd7b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28209152->28213247] (req 28209152->28213247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.614178:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760782.614183:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2fd7b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28209152->28213247] (req 28209152->28213247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.614192:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2fd7b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28209152->28213247] (req 28209152->28213247) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.614210:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2fd7b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28209152->28213247] (req 28209152->28213247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.614220:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2fd7b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28209152->28213247] (req 28209152->28213247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760782.614305:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac38b6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512745216:12345-192.168.203.56@tcp:4:dd.0 Request processed in 28195us (28721us total) trans 4294974206 rc 0/0 00000100:00100000:1.0:1789760782.614314:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6917 00000100:00100000:1.0:1789760782.614323:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 762124540404 time 762150393259 state 0x2 ref 0 00000100:00100000:2.0:1789760782.617019:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760782.617130:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512745344 00000100:00100000:0.0:1789760782.617146:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512745344 00000100:00100000:0.0:1789760782.617196:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 762124540404 time 762150393259 state 0x0 ref 1 00000100:00100000:0.0:1789760782.617214:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 762150653675 time 762150393259 state 0x1 ref 1 00000100:00100000:0.0:1789760782.617238:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 762150653675 time 762181537889 state 0x0 ref 1 00000100:00100000:0.0:1789760782.617250:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febea651500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512745344:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760782.617295:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28213248, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0xd807 00010000:00010000:0.0:1789760782.617304:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2fd82 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.617320:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=762s v=(0 0) 00010000:00010000:0.0:1789760782.617324:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760782.617331:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2fd82 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28213248->28217343] (req 28213248->28217343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.617345:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2fd82 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28213248->28217343] (req 28213248->28217343) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760782.654142:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974207 is committed 00002000:00010000:0.0:1789760782.654399:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28213248 -> 28217344 00010000:00010000:0.0:1789760782.654414:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2fd82 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28213248->28217343] (req 28213248->28217343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.654426:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2fd82 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28213248->28217343] (req 28213248->28217343) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.654434:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2fd82 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28213248->28217343] (req 28213248->28217343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.654446:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760782.654451:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2fd82 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28213248->28217343] (req 28213248->28217343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.654460:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2fd82 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28213248->28217343] (req 28213248->28217343) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.654479:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2fd82 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28213248->28217343] (req 28213248->28217343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.654487:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2fd82 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28213248->28217343] (req 28213248->28217343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760782.654578:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febea651500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512745344:12345-192.168.203.56@tcp:4:dd.0 Request processed in 37325us (37563us total) trans 4294974207 rc 0/0 00000100:00100000:0.0:1789760782.654589:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6918 00000100:00100000:0.0:1789760782.654599:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 762150653675 time 762181537889 state 0x2 ref 0 00000100:00100000:2.0:1789760782.660481:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760782.660584:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512745472 00000100:00100000:0.0:1789760782.660594:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512745472 00000100:00100000:0.0:1789760782.660648:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 762150653675 time 762181537889 state 0x0 ref 1 00000100:00100000:0.0:1789760782.660662:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 762181798305 time 762181537889 state 0x1 ref 1 00000100:00100000:0.0:1789760782.660765:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 762181798305 time 762225063968 state 0x0 ref 1 00000100:00100000:0.0:1789760782.660782:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febea650380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512745472:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760782.660830:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28217344, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0xd807 00010000:00010000:0.0:1789760782.660839:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2fd89 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.660854:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=762s v=(0 0) 00010000:00010000:0.0:1789760782.660859:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760782.660866:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2fd89 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28217344->28221439] (req 28217344->28221439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.660879:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2fd89 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28217344->28221439] (req 28217344->28221439) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760782.684992:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974208 is committed 00002000:00010000:0.0:1789760782.685099:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28217344 -> 28221440 00010000:00010000:0.0:1789760782.685114:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2fd89 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28217344->28221439] (req 28217344->28221439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.685127:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2fd89 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28217344->28221439] (req 28217344->28221439) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.685137:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2fd89 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28217344->28221439] (req 28217344->28221439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.685144:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760782.685149:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2fd89 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28217344->28221439] (req 28217344->28221439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.685158:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2fd89 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28217344->28221439] (req 28217344->28221439) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.685177:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2fd89 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28217344->28221439] (req 28217344->28221439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.685185:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2fd89 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28217344->28221439] (req 28217344->28221439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760782.685336:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febea650380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512745472:12345-192.168.203.56@tcp:4:dd.0 Request processed in 24552us (24860us total) trans 4294974208 rc 0/0 00000100:00100000:0.0:1789760782.685346:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6919 00000100:00100000:0.0:1789760782.685355:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 762181798305 time 762225063968 state 0x2 ref 0 00000100:00100000:2.0:1789760782.687881:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760782.687976:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512745600 00000100:00100000:1.0:1789760782.687986:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512745600 00000100:00100000:1.0:1789760782.688054:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 762181798305 time 762225063968 state 0x0 ref 1 00000100:00100000:1.0:1789760782.688419:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 762225324384 time 762225063968 state 0x1 ref 1 00000100:00100000:1.0:1789760782.688448:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 762225324384 time 762252747982 state 0x0 ref 1 00000100:00100000:1.0:1789760782.688463:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2745880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512745600:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760782.688508:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28221440, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0xd807 00010000:00010000:1.0:1789760782.688516:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2fd90 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.688532:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=762s v=(0 0) 00010000:00010000:1.0:1789760782.688536:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760782.688542:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2fd90 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28221440->28225535] (req 28221440->28225535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.688566:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2fd90 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28221440->28225535] (req 28221440->28225535) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.707320:0:5835:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:3.0:1789760782.727465:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974209 is committed 00002000:00010000:0.0:1789760782.727571:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28221440 -> 28225536 00010000:00010000:0.0:1789760782.727586:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2fd90 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28221440->28225535] (req 28221440->28225535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.727599:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2fd90 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28221440->28225535] (req 28221440->28225535) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.727608:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2fd90 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28221440->28225535] (req 28221440->28225535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.727616:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760782.727622:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2fd90 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28221440->28225535] (req 28221440->28225535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.727631:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2fd90 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28221440->28225535] (req 28221440->28225535) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.727649:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2fd90 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28221440->28225535] (req 28221440->28225535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.727658:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c2fd90 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28221440->28225535] (req 28221440->28225535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760782.727869:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2745880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512745600:12345-192.168.203.56@tcp:4:dd.0 Request processed in 39404us (39992us total) trans 4294974209 rc 0/0 00000100:00100000:0.0:1789760782.727881:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6920 00000100:00100000:0.0:1789760782.727891:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 762225324384 time 762252747982 state 0x2 ref 0 00000100:00100000:2.0:1789760782.732150:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760782.732298:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512745728 00000100:00100000:0.0:1789760782.732308:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512745728 00000100:00100000:0.0:1789760782.732361:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 762225324384 time 762252747982 state 0x0 ref 1 00000100:00100000:0.0:1789760782.732376:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 762253008398 time 762252747982 state 0x1 ref 1 00000100:00100000:0.0:1789760782.732440:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 762253008398 time 762296739096 state 0x0 ref 1 00000100:00100000:0.0:1789760782.732452:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2745180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512745728:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760782.732517:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28225536, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0xd807 00010000:00010000:0.0:1789760782.732527:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2fd97 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.732544:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=762s v=(0 0) 00010000:00010000:0.0:1789760782.732548:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760782.732556:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2fd97 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28225536->28229631] (req 28225536->28229631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.732569:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2fd97 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28225536->28229631] (req 28225536->28229631) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760782.768637:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28225536 -> 28229632 00010000:00010000:0.0:1789760782.768656:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2fd97 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28225536->28229631] (req 28225536->28229631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.768669:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2fd97 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28225536->28229631] (req 28225536->28229631) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.768677:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2fd97 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28225536->28229631] (req 28225536->28229631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.768684:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760782.768689:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2fd97 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28225536->28229631] (req 28225536->28229631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.768725:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2fd97 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28225536->28229631] (req 28225536->28229631) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.768749:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2fd97 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28225536->28229631] (req 28225536->28229631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.768757:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2fd97 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28225536->28229631] (req 28225536->28229631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760782.768847:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2745180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2359:x1876699512745728:12345-192.168.203.56@tcp:4:dd.0 Request processed in 36392us (36702us total) trans 4294974210 rc 0/0 00000100:00100000:0.0:1789760782.768857:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6921 00000100:00100000:0.0:1789760782.768867:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 762253008398 time 762296739096 state 0x2 ref 0 00000001:00080000:1.0:1789760782.769530:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974210 is committed 00000100:00100000:1.0:1789760782.771296:0:3312:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8feac2744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3312:1876699540232064:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1789760782.771363:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760782.771488:0:6706:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540232064 00000100:00100000:3.0:1789760782.771495:0:6706:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540232064 00000100:00100000:3.0:1789760782.771546:0:6706:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 213 00000100:00100000:3.0:1789760782.771553:0:6706:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac287a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3312:x1876699540232064:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1789760782.771585:0:6706:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3859808256 avail 3857711104 00000100:00100000:3.0:1789760782.774812:0:6706:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac287a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3312:x1876699540232064:12345-0@lo:13:osp-pre-1-0.0 Request processed in 3249us (3438us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760782.774826:0:6706:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 213 00000100:00100000:1.0:1789760782.777143:0:3312:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8feac2744a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3312:1876699540232064:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789760782.777632:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760782.777769:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512745856 00000100:00100000:0.0:1789760782.777776:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512745856 00000100:00100000:0.0:1789760782.777817:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 762253008398 time 762296739096 state 0x0 ref 1 00000100:00100000:0.0:1789760782.777830:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 762296999512 time 762296739096 state 0x1 ref 1 00000100:00100000:0.0:1789760782.777848:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 762296999512 time 762342148150 state 0x0 ref 1 00000100:00100000:0.0:1789760782.777858:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2746300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512745856:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760782.777905:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28229632, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0xd807 00010000:00010000:0.0:1789760782.777914:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2fd9e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.777938:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=762s v=(0 0) 00010000:00010000:0.0:1789760782.777943:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760782.777949:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2fd9e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28229632->28233727] (req 28229632->28233727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.777961:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2fd9e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28229632->28233727] (req 28229632->28233727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760782.808102:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974211 is committed 00002000:00010000:0.0:1789760782.809204:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28229632 -> 28233728 00010000:00010000:0.0:1789760782.809221:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2fd9e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28229632->28233727] (req 28229632->28233727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.809235:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2fd9e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28229632->28233727] (req 28229632->28233727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.809244:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2fd9e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28229632->28233727] (req 28229632->28233727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.809252:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760782.809257:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2fd9e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28229632->28233727] (req 28229632->28233727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.809267:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2fd9e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28229632->28233727] (req 28229632->28233727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.809284:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2fd9e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28229632->28233727] (req 28229632->28233727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.809312:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2fd9e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28229632->28233727] (req 28229632->28233727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760782.809413:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2746300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512745856:12345-192.168.203.56@tcp:4:dd.0 Request processed in 31551us (31779us total) trans 4294974211 rc 0/0 00000100:00100000:0.0:1789760782.809423:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6922 00000100:00100000:0.0:1789760782.809432:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 762296999512 time 762342148150 state 0x2 ref 0 00000100:00100000:3.0:1789760782.813762:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760782.813826:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512745984 00000100:00100000:0.0:1789760782.813835:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512745984 00000100:00100000:0.0:1789760782.813890:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 762296999512 time 762342148150 state 0x0 ref 1 00000100:00100000:0.0:1789760782.813907:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 762342408566 time 762342148150 state 0x1 ref 1 00000100:00100000:0.0:1789760782.813937:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 762342408566 time 762378236507 state 0x0 ref 1 00000100:00100000:0.0:1789760782.813950:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febea489050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512745984:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760782.813997:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28233728, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0xd807 00010000:00010000:0.0:1789760782.814006:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2fda5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.814023:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=762s v=(0 0) 00010000:00010000:0.0:1789760782.814037:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760782.814047:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2fda5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28233728->28237823] (req 28233728->28237823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.814066:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2fda5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28233728->28237823] (req 28233728->28237823) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760782.861975:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974212 is committed 00002000:00010000:1.0:1789760782.862013:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28233728 -> 28237824 00010000:00010000:1.0:1789760782.862044:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2fda5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28233728->28237823] (req 28233728->28237823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.862058:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2fda5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28233728->28237823] (req 28233728->28237823) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.862067:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2fda5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28233728->28237823] (req 28233728->28237823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.862074:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760782.862079:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2fda5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28233728->28237823] (req 28233728->28237823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.862088:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2fda5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28233728->28237823] (req 28233728->28237823) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.862109:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2fda5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28233728->28237823] (req 28233728->28237823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.862118:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c2fda5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28233728->28237823] (req 28233728->28237823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760782.862210:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febea489050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2361:x1876699512745984:12345-192.168.203.56@tcp:4:dd.0 Request processed in 48257us (48450us total) trans 4294974212 rc 0/0 00000100:00100000:1.0:1789760782.862222:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6923 00000100:00100000:1.0:1789760782.862244:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 762342408566 time 762378236507 state 0x2 ref 0 00000100:00100000:2.0:1789760782.866206:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760782.866298:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512746112 00000100:00100000:0.0:1789760782.866309:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512746112 00000100:00100000:0.0:1789760782.866367:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 762342408566 time 762378236507 state 0x0 ref 1 00000100:00100000:0.0:1789760782.866384:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 762378496923 time 762378236507 state 0x1 ref 1 00000100:00100000:0.0:1789760782.866411:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 762378496923 time 762430711149 state 0x0 ref 1 00000100:00100000:0.0:1789760782.866424:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2745500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512746112:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760782.866474:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28237824, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0xd807 00010000:00010000:0.0:1789760782.866483:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2fdac lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.866499:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=762s v=(0 0) 00010000:00010000:0.0:1789760782.866503:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760782.866509:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2fdac lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28237824->28241919] (req 28237824->28241919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.866522:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2fdac lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28237824->28241919] (req 28237824->28241919) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760782.907687:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28237824 -> 28241920 00010000:00010000:0.0:1789760782.907735:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2fdac lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28237824->28241919] (req 28237824->28241919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.907755:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2fdac lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28237824->28241919] (req 28237824->28241919) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.907765:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2fdac lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28237824->28241919] (req 28237824->28241919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.907774:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760782.907779:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2fdac lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28237824->28241919] (req 28237824->28241919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.907788:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2fdac lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28237824->28241919] (req 28237824->28241919) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.907807:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2fdac lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28237824->28241919] (req 28237824->28241919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.907814:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2fdac lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28237824->28241919] (req 28237824->28241919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760782.907889:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2745500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2361:x1876699512746112:12345-192.168.203.56@tcp:4:dd.0 Request processed in 41464us (41689us total) trans 4294974213 rc 0/0 00000100:00100000:0.0:1789760782.907899:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6924 00000100:00100000:0.0:1789760782.907909:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 762378496923 time 762430711149 state 0x2 ref 0 00000001:00080000:1.0:1789760782.908363:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974213 is committed 00000100:00100000:2.0:1789760782.911657:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760782.911969:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512746240 00000100:00100000:0.0:1789760782.911979:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512746240 00000100:00100000:0.0:1789760782.912040:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 762378496923 time 762430711149 state 0x0 ref 1 00000100:00100000:0.0:1789760782.912119:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 762430971565 time 762430711149 state 0x1 ref 1 00000100:00100000:0.0:1789760782.912183:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 762430971565 time 762476481655 state 0x0 ref 1 00000100:00100000:0.0:1789760782.912204:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2744e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512746240:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760782.912257:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28241920, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0xd807 00010000:00010000:0.0:1789760782.912265:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2fdb3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.912282:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=762s v=(0 0) 00010000:00010000:0.0:1789760782.912286:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760782.912293:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2fdb3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28241920->28246015] (req 28241920->28246015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.912306:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2fdb3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28241920->28246015] (req 28241920->28246015) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760782.960227:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974214 is committed 00002000:00010000:1.0:1789760782.960321:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28241920 -> 28246016 00010000:00010000:1.0:1789760782.960336:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2fdb3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28241920->28246015] (req 28241920->28246015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.960354:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2fdb3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28241920->28246015] (req 28241920->28246015) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.960362:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2fdb3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28241920->28246015] (req 28241920->28246015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.960369:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760782.960373:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2fdb3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28241920->28246015] (req 28241920->28246015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.960383:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2fdb3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28241920->28246015] (req 28241920->28246015) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.960428:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2fdb3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28241920->28246015] (req 28241920->28246015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.960436:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2fdb3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28241920->28246015] (req 28241920->28246015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760782.960779:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2744e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512746240:12345-192.168.203.56@tcp:4:dd.0 Request processed in 48571us (49125us total) trans 4294974214 rc 0/0 00000100:00100000:1.0:1789760782.960790:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6925 00000100:00100000:1.0:1789760782.960800:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 762430971565 time 762476481655 state 0x2 ref 0 00000100:00100000:3.0:1789760782.966923:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760782.967054:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512746368 00000100:00100000:1.0:1789760782.967064:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512746368 00000100:00100000:1.0:1789760782.967120:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 762430971565 time 762476481655 state 0x0 ref 1 00000100:00100000:1.0:1789760782.967137:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 762476742071 time 762476481655 state 0x1 ref 1 00000100:00100000:1.0:1789760782.967168:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 762476742071 time 762531467900 state 0x0 ref 1 00000100:00100000:1.0:1789760782.967189:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac531dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512746368:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760782.967238:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28246016, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0xd807 00010000:00010000:1.0:1789760782.967247:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2fdba lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.967263:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=762s v=(0 0) 00010000:00010000:1.0:1789760782.967268:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760782.967275:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2fdba lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28246016->28250111] (req 28246016->28250111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.967288:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2fdba lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28246016->28250111] (req 28246016->28250111) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760783.002091:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974215 is committed 00002000:00010000:1.0:1789760783.002562:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28246016 -> 28250112 00010000:00010000:1.0:1789760783.002577:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2fdba lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28246016->28250111] (req 28246016->28250111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.002590:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2fdba lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28246016->28250111] (req 28246016->28250111) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.002598:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2fdba lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28246016->28250111] (req 28246016->28250111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.002605:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760783.002610:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2fdba lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28246016->28250111] (req 28246016->28250111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.002619:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2fdba lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28246016->28250111] (req 28246016->28250111) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.002638:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2fdba lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28246016->28250111] (req 28246016->28250111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.002645:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2fdba lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28246016->28250111] (req 28246016->28250111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760783.002884:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac531dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512746368:12345-192.168.203.56@tcp:4:dd.0 Request processed in 35701us (35968us total) trans 4294974215 rc 0/0 00000100:00100000:1.0:1789760783.002896:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6926 00000100:00100000:1.0:1789760783.002905:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 762476742071 time 762531467900 state 0x2 ref 0 00000100:00100000:2.0:1789760783.005952:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760783.006051:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512746496 00000100:00100000:0.0:1789760783.006060:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512746496 00000100:00100000:0.0:1789760783.006137:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 762476742071 time 762531467900 state 0x0 ref 1 00000100:00100000:0.0:1789760783.006153:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 762531728316 time 762531467900 state 0x1 ref 1 00000100:00100000:0.0:1789760783.006181:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 762531728316 time 762570480700 state 0x0 ref 1 00000100:00100000:0.0:1789760783.006193:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2745f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512746496:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760783.006240:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28250112, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0xd807 00010000:00010000:0.0:1789760783.006249:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2fdc1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.006264:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=762s v=(0 0) 00010000:00010000:0.0:1789760783.006268:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760783.006275:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2fdc1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28250112->28254207] (req 28250112->28254207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.006288:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2fdc1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28250112->28254207] (req 28250112->28254207) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1789760783.007288:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:1.0:1789760783.043300:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974216 is committed 00002000:00010000:1.0:1789760783.043360:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28250112 -> 28254208 00010000:00010000:1.0:1789760783.043374:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2fdc1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28250112->28254207] (req 28250112->28254207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.043386:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2fdc1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28250112->28254207] (req 28250112->28254207) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.043395:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2fdc1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28250112->28254207] (req 28250112->28254207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.043403:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760783.043408:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2fdc1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28250112->28254207] (req 28250112->28254207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.043418:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2fdc1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28250112->28254207] (req 28250112->28254207) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.043438:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2fdc1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28250112->28254207] (req 28250112->28254207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.043451:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2fdc1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28250112->28254207] (req 28250112->28254207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760783.043550:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2745f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512746496:12345-192.168.203.56@tcp:4:dd.0 Request processed in 37355us (37603us total) trans 4294974216 rc 0/0 00000100:00100000:1.0:1789760783.043560:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6927 00000100:00100000:1.0:1789760783.043569:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 762531728316 time 762570480700 state 0x2 ref 0 00000100:00100000:2.0:1789760783.047210:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760783.048264:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512746624 00000100:00100000:1.0:1789760783.048277:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512746624 00000100:00100000:1.0:1789760783.048333:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 762531728316 time 762570480700 state 0x0 ref 1 00000100:00100000:1.0:1789760783.048350:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 762570741116 time 762570480700 state 0x1 ref 1 00000100:00100000:1.0:1789760783.049261:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 762570741116 time 762613556500 state 0x0 ref 1 00000100:00100000:1.0:1789760783.049279:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2746a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512746624:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760783.049331:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28254208, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0xd807 00010000:00010000:1.0:1789760783.049340:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2fdc8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.049356:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=762s v=(0 0) 00010000:00010000:1.0:1789760783.049361:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760783.049368:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2fdc8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28254208->28258303] (req 28254208->28258303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.049387:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2fdc8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28254208->28258303] (req 28254208->28258303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760783.107358:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974217 is committed 00002000:00010000:1.0:1789760783.107605:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28254208 -> 28258304 00010000:00010000:1.0:1789760783.107622:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2fdc8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28254208->28258303] (req 28254208->28258303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.107634:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2fdc8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28254208->28258303] (req 28254208->28258303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.107644:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2fdc8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28254208->28258303] (req 28254208->28258303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.107652:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760783.107657:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2fdc8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28254208->28258303] (req 28254208->28258303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.107667:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2fdc8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28254208->28258303] (req 28254208->28258303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.107686:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2fdc8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28254208->28258303] (req 28254208->28258303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.107729:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2fdc8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28254208->28258303] (req 28254208->28258303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760783.107818:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2746a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512746624:12345-192.168.203.56@tcp:4:dd.0 Request processed in 58540us (60687us total) trans 4294974217 rc 0/0 00000100:00100000:1.0:1789760783.107838:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6928 00000100:00100000:1.0:1789760783.107847:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 762570741116 time 762613556500 state 0x2 ref 0 00000100:00100000:2.0:1789760783.111190:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760783.111945:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512746752 00000100:00100000:1.0:1789760783.111956:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512746752 00000100:00100000:1.0:1789760783.112010:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 762570741116 time 762613556500 state 0x0 ref 1 00000100:00100000:1.0:1789760783.112037:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 762613816916 time 762613556500 state 0x1 ref 1 00000100:00100000:1.0:1789760783.112062:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 762613816916 time 762676361696 state 0x0 ref 1 00000100:00100000:1.0:1789760783.112074:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2745880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512746752:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760783.112124:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28258304, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0xd807 00010000:00010000:1.0:1789760783.112132:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2fdcf lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.112148:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=762s v=(0 0) 00010000:00010000:1.0:1789760783.112152:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760783.112159:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2fdcf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28258304->28262399] (req 28258304->28262399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.112174:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2fdcf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28258304->28262399] (req 28258304->28262399) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760783.195553:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974218 is committed 00002000:00010000:1.0:1789760783.195760:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28258304 -> 28262400 00010000:00010000:1.0:1789760783.195777:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2fdcf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28258304->28262399] (req 28258304->28262399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.195789:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2fdcf lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28258304->28262399] (req 28258304->28262399) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.195797:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2fdcf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28258304->28262399] (req 28258304->28262399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.195804:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760783.195809:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2fdcf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28258304->28262399] (req 28258304->28262399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.195818:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2fdcf lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28258304->28262399] (req 28258304->28262399) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.195837:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2fdcf lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28258304->28262399] (req 28258304->28262399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.195940:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2fdcf lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28258304->28262399] (req 28258304->28262399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760783.196088:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2745880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512746752:12345-192.168.203.56@tcp:4:dd.0 Request processed in 84010us (84968us total) trans 4294974218 rc 0/0 00000100:00100000:1.0:1789760783.196102:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6929 00000100:00100000:1.0:1789760783.196112:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 762613816916 time 762676361696 state 0x2 ref 0 00000100:00100000:0.0:1789760783.207985:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760783.208079:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512746880 00000100:00100000:1.0:1789760783.208090:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512746880 00000100:00100000:1.0:1789760783.208214:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 762613816916 time 762676361696 state 0x0 ref 1 00000100:00100000:1.0:1789760783.208232:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 762676622112 time 762676361696 state 0x1 ref 1 00000100:00100000:1.0:1789760783.208266:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 762676622112 time 762772565654 state 0x0 ref 1 00000100:00100000:1.0:1789760783.208280:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac28f0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512746880:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760783.208333:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28262400, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0xd807 00010000:00010000:1.0:1789760783.208343:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2fdd6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.208358:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=762s v=(0 0) 00010000:00010000:1.0:1789760783.208362:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760783.208368:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2fdd6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28262400->28266495] (req 28262400->28266495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.208382:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2fdd6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28262400->28266495] (req 28262400->28266495) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760783.281435:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974219 is committed 00002000:00010000:1.0:1789760783.281627:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28262400 -> 28266496 00010000:00010000:1.0:1789760783.281643:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2fdd6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28262400->28266495] (req 28262400->28266495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.281656:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2fdd6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28262400->28266495] (req 28262400->28266495) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.281666:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2fdd6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28262400->28266495] (req 28262400->28266495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.281673:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760783.281678:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2fdd6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28262400->28266495] (req 28262400->28266495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.281687:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2fdd6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28262400->28266495] (req 28262400->28266495) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.281760:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2fdd6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28262400->28266495] (req 28262400->28266495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.281768:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2fdd6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28262400->28266495] (req 28262400->28266495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760783.281862:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac28f0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512746880:12345-192.168.203.56@tcp:4:dd.0 Request processed in 73580us (73882us total) trans 4294974219 rc 0/0 00000100:00100000:1.0:1789760783.281873:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6930 00000100:00100000:1.0:1789760783.281883:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 762676622112 time 762772565654 state 0x2 ref 0 00000100:00100000:2.0:1789760783.287674:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760783.288112:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512747008 00000100:00100000:1.0:1789760783.288122:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512747008 00000100:00100000:1.0:1789760783.288245:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 762676622112 time 762772565654 state 0x0 ref 1 00000100:00100000:1.0:1789760783.288260:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 762772826070 time 762772565654 state 0x1 ref 1 00000100:00100000:1.0:1789760783.288319:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 762772826070 time 762852618325 state 0x0 ref 1 00000100:00100000:1.0:1789760783.288332:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2745180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512747008:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760783.288378:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28266496, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0xd807 00010000:00010000:1.0:1789760783.288387:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2fddd lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.288403:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=762s v=(0 0) 00010000:00010000:1.0:1789760783.288407:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760783.288413:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2fddd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28266496->28270591] (req 28266496->28270591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.288426:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2fddd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28266496->28270591] (req 28266496->28270591) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760783.341974:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974220 is committed 00002000:00010000:1.0:1789760783.342170:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28266496 -> 28270592 00010000:00010000:1.0:1789760783.342186:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2fddd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28266496->28270591] (req 28266496->28270591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.342199:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2fddd lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28266496->28270591] (req 28266496->28270591) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.342213:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2fddd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28266496->28270591] (req 28266496->28270591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.342219:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760783.342224:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2fddd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28266496->28270591] (req 28266496->28270591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.342232:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2fddd lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28266496->28270591] (req 28266496->28270591) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.342249:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2fddd lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28266496->28270591] (req 28266496->28270591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.342255:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2fddd lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28266496->28270591] (req 28266496->28270591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760783.342523:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2745180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512747008:12345-192.168.203.56@tcp:4:dd.0 Request processed in 54187us (54852us total) trans 4294974220 rc 0/0 00000100:00100000:1.0:1789760783.342534:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6931 00000100:00100000:1.0:1789760783.342543:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 762772826070 time 762852618325 state 0x2 ref 0 00000100:00100000:0.0:1789760783.350815:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760783.350902:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512747136 00000100:00100000:1.0:1789760783.350913:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512747136 00000100:00100000:1.0:1789760783.350964:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 762772826070 time 762852618325 state 0x0 ref 1 00000100:00100000:1.0:1789760783.350979:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 762852878741 time 762852618325 state 0x1 ref 1 00000100:00100000:1.0:1789760783.351013:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 762852878741 time 762915312951 state 0x0 ref 1 00000100:00100000:1.0:1789760783.351040:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac28f2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512747136:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760783.351093:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28270592, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0xd807 00010000:00010000:1.0:1789760783.351106:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2fde4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.351233:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=762s v=(0 0) 00010000:00010000:1.0:1789760783.351239:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760783.351246:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2fde4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28270592->28274687] (req 28270592->28274687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.351261:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2fde4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28270592->28274687] (req 28270592->28274687) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760783.410182:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974221 is committed 00002000:00010000:1.0:1789760783.410339:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28270592 -> 28274688 00010000:00010000:1.0:1789760783.410363:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2fde4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28270592->28274687] (req 28270592->28274687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.410375:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2fde4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28270592->28274687] (req 28270592->28274687) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.410384:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2fde4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28270592->28274687] (req 28270592->28274687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.410392:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760783.410398:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2fde4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28270592->28274687] (req 28270592->28274687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.410407:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2fde4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28270592->28274687] (req 28270592->28274687) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.410433:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2fde4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28270592->28274687] (req 28270592->28274687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.410440:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2fde4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28270592->28274687] (req 28270592->28274687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760783.410531:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac28f2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512747136:12345-192.168.203.56@tcp:4:dd.0 Request processed in 59502us (59722us total) trans 4294974221 rc 0/0 00000100:00100000:1.0:1789760783.410541:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6932 00000100:00100000:1.0:1789760783.410551:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 762852878741 time 762915312951 state 0x2 ref 0 00000100:00100000:0.0:1789760783.414141:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760783.415040:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512747264 00000100:00100000:1.0:1789760783.415051:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512747264 00000100:00100000:1.0:1789760783.415164:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 762852878741 time 762915312951 state 0x0 ref 1 00000100:00100000:1.0:1789760783.415179:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 762915573367 time 762915312951 state 0x1 ref 1 00000100:00100000:1.0:1789760783.415200:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 762915573367 time 762979499658 state 0x0 ref 1 00000100:00100000:1.0:1789760783.415213:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4360a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512747264:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760783.415262:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28274688, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0xd807 00010000:00010000:1.0:1789760783.415271:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2fdeb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.415287:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=762s v=(0 0) 00010000:00010000:1.0:1789760783.415291:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760783.415298:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2fdeb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28274688->28278783] (req 28274688->28278783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.415312:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2fdeb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28274688->28278783] (req 28274688->28278783) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760783.447061:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28274688 -> 28278784 00010000:00010000:2.0:1789760783.447080:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2fdeb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28274688->28278783] (req 28274688->28278783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.447092:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2fdeb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28274688->28278783] (req 28274688->28278783) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.447101:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2fdeb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28274688->28278783] (req 28274688->28278783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.447107:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760783.447111:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2fdeb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28274688->28278783] (req 28274688->28278783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.447119:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2fdeb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28274688->28278783] (req 28274688->28278783) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.447138:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2fdeb lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28274688->28278783] (req 28274688->28278783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.447146:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2fdeb lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28274688->28278783] (req 28274688->28278783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760783.447235:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4360a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2359:x1876699512747264:12345-192.168.203.56@tcp:4:dd.0 Request processed in 32019us (33099us total) trans 4294974222 rc 0/0 00000100:00100000:2.0:1789760783.447245:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6933 00000100:00100000:2.0:1789760783.447253:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 762915573367 time 762979499658 state 0x2 ref 0 00000001:00080000:2.0:1789760783.447291:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974222 is committed 00000100:00100000:0.0:1789760783.449988:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760783.450257:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512747392 00000100:00100000:1.0:1789760783.450267:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512747392 00000100:00100000:1.0:1789760783.450319:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 762915573367 time 762979499658 state 0x0 ref 1 00000100:00100000:1.0:1789760783.450335:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 762979760074 time 762979499658 state 0x1 ref 1 00000100:00100000:1.0:1789760783.450413:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 762979760074 time 763014711260 state 0x0 ref 1 00000100:00100000:1.0:1789760783.450426:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4362a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512747392:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760783.450476:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28278784, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0xd807 00010000:00010000:1.0:1789760783.450485:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2fdf2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.450501:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=763s v=(0 0) 00010000:00010000:1.0:1789760783.450505:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760783.450511:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2fdf2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28278784->28282879] (req 28278784->28282879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.450524:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2fdf2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28278784->28282879] (req 28278784->28282879) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760783.490134:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974223 is committed 00002000:00010000:3.0:1789760783.490162:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28278784 -> 28282880 00010000:00010000:3.0:1789760783.490180:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2fdf2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28278784->28282879] (req 28278784->28282879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.490192:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2fdf2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28278784->28282879] (req 28278784->28282879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.490206:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2fdf2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28278784->28282879] (req 28278784->28282879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.490278:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760783.490283:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2fdf2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28278784->28282879] (req 28278784->28282879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.490292:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2fdf2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28278784->28282879] (req 28278784->28282879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.490310:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2fdf2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28278784->28282879] (req 28278784->28282879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.490317:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c2fdf2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28278784->28282879] (req 28278784->28282879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760783.490410:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4362a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512747392:12345-192.168.203.56@tcp:4:dd.0 Request processed in 39980us (40425us total) trans 4294974223 rc 0/0 00000100:00100000:3.0:1789760783.490420:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6934 00000100:00100000:3.0:1789760783.490431:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 762979760074 time 763014711260 state 0x2 ref 0 00000100:00100000:0.0:1789760783.497196:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760783.497296:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512747520 00000100:00100000:1.0:1789760783.497307:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512747520 00000100:00100000:1.0:1789760783.497363:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 762979760074 time 763014711260 state 0x0 ref 1 00000100:00100000:1.0:1789760783.497379:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 763014971676 time 763014711260 state 0x1 ref 1 00000100:00100000:1.0:1789760783.497450:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 763014971676 time 763061748804 state 0x0 ref 1 00000100:00100000:1.0:1789760783.497463:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4363100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512747520:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760783.497515:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28282880, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0xd807 00010000:00010000:1.0:1789760783.497525:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2fdf9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.497540:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=763s v=(0 0) 00010000:00010000:1.0:1789760783.497545:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760783.497550:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2fdf9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28282880->28286975] (req 28282880->28286975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.497565:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2fdf9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28282880->28286975] (req 28282880->28286975) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760783.540749:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28282880 -> 28286976 00002000:00010000:1.0:1789760783.540758:0:6710:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760782 -> 1789760783 00002000:00010000:1.0:1789760783.540761:0:6710:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760782 -> 1789760783 00010000:00010000:1.0:1789760783.540775:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2fdf9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28282880->28286975] (req 28282880->28286975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.540790:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2fdf9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28282880->28286975] (req 28282880->28286975) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.540800:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2fdf9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28282880->28286975] (req 28282880->28286975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.540807:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760783.540812:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2fdf9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28282880->28286975] (req 28282880->28286975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.540821:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2fdf9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28282880->28286975] (req 28282880->28286975) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.540840:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2fdf9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28282880->28286975] (req 28282880->28286975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.540848:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2fdf9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28282880->28286975] (req 28282880->28286975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760783.540946:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4363100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2361:x1876699512747520:12345-192.168.203.56@tcp:4:dd.0 Request processed in 43481us (43756us total) trans 4294974224 rc 0/0 00000100:00100000:1.0:1789760783.540956:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6935 00000100:00100000:1.0:1789760783.540966:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 763014971676 time 763061748804 state 0x2 ref 0 00000001:00080000:3.0:1789760783.541684:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974224 is committed 00000100:00100000:0.0:1789760783.548679:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760783.549066:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512747648 00000100:00100000:2.0:1789760783.549076:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512747648 00000100:00100000:2.0:1789760783.549133:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 763014971676 time 763061748804 state 0x0 ref 1 00000100:00100000:2.0:1789760783.549148:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 763062009220 time 763061748804 state 0x1 ref 1 00000100:00100000:2.0:1789760783.549182:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 763062009220 time 763113481541 state 0x0 ref 1 00000100:00100000:2.0:1789760783.549315:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4362d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512747648:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760783.549364:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28286976, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0xd807 00010000:00010000:2.0:1789760783.549372:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2fe00 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.549387:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=763s v=(0 0) 00010000:00010000:2.0:1789760783.549391:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760783.549396:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2fe00 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28286976->28291071] (req 28286976->28291071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.549409:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2fe00 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28286976->28291071] (req 28286976->28291071) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760783.592939:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974225 is committed 00002000:00010000:3.0:1789760783.593164:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28286976 -> 28291072 00010000:00010000:3.0:1789760783.593179:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2fe00 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28286976->28291071] (req 28286976->28291071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.593194:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2fe00 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28286976->28291071] (req 28286976->28291071) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.593202:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2fe00 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28286976->28291071] (req 28286976->28291071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.593210:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760783.593214:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2fe00 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28286976->28291071] (req 28286976->28291071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.593223:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2fe00 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28286976->28291071] (req 28286976->28291071) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.593242:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2fe00 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28286976->28291071] (req 28286976->28291071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.593249:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2fe00 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28286976->28291071] (req 28286976->28291071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760783.593348:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4362d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512747648:12345-192.168.203.56@tcp:4:dd.0 Request processed in 44025us (44666us total) trans 4294974225 rc 0/0 00000100:00100000:3.0:1789760783.593358:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6936 00000100:00100000:3.0:1789760783.593368:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 763062009220 time 763113481541 state 0x2 ref 0 00000100:00100000:0.0:1789760783.596189:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760783.596273:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512747776 00000100:00100000:3.0:1789760783.596284:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512747776 00000100:00100000:3.0:1789760783.596351:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 763062009220 time 763113481541 state 0x0 ref 1 00000100:00100000:3.0:1789760783.596368:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 763113741957 time 763113481541 state 0x1 ref 1 00000100:00100000:3.0:1789760783.596394:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 763113741957 time 763160694120 state 0x0 ref 1 00000100:00100000:3.0:1789760783.596407:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4076a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512747776:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760783.596455:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28291072, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0xd807 00010000:00010000:3.0:1789760783.596464:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2fe07 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.596479:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=763s v=(0 0) 00010000:00010000:3.0:1789760783.596483:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760783.596489:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2fe07 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28291072->28295167] (req 28291072->28295167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.596503:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2fe07 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28291072->28295167] (req 28291072->28295167) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760783.645896:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28291072 -> 28295168 00010000:00010000:0.0:1789760783.645915:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2fe07 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28291072->28295167] (req 28291072->28295167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.645929:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2fe07 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28291072->28295167] (req 28291072->28295167) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.645938:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2fe07 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28291072->28295167] (req 28291072->28295167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.645946:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760783.645951:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2fe07 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28291072->28295167] (req 28291072->28295167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.645960:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2fe07 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28291072->28295167] (req 28291072->28295167) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.645980:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2fe07 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28291072->28295167] (req 28291072->28295167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.645987:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2fe07 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28291072->28295167] (req 28291072->28295167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760783.646091:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4076a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2359:x1876699512747776:12345-192.168.203.56@tcp:4:dd.0 Request processed in 49680us (49906us total) trans 4294974226 rc 0/0 00000100:00100000:0.0:1789760783.646101:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6937 00000100:00100000:0.0:1789760783.646113:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 763113741957 time 763160694120 state 0x2 ref 0 00000001:00080000:2.0:1789760783.646192:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974226 is committed 00000100:00100000:0.0:1789760783.649393:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760783.649457:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512747904 00000100:00100000:1.0:1789760783.649466:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512747904 00000100:00100000:1.0:1789760783.649520:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 763113741957 time 763160694120 state 0x0 ref 1 00000100:00100000:1.0:1789760783.649536:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 763160954536 time 763160694120 state 0x1 ref 1 00000100:00100000:1.0:1789760783.649567:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 763160954536 time 763213867322 state 0x0 ref 1 00000100:00100000:1.0:1789760783.649579:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4074e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512747904:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760783.649627:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28295168, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0xd807 00010000:00010000:1.0:1789760783.649635:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2fe0e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.649651:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=763s v=(0 0) 00010000:00010000:1.0:1789760783.649655:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760783.649661:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2fe0e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28295168->28299263] (req 28295168->28299263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.649675:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2fe0e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28295168->28299263] (req 28295168->28299263) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760783.679822:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28295168 -> 28299264 00010000:00010000:1.0:1789760783.679842:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2fe0e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28295168->28299263] (req 28295168->28299263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.679855:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2fe0e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28295168->28299263] (req 28295168->28299263) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.679863:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2fe0e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28295168->28299263] (req 28295168->28299263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.679869:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760783.679874:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2fe0e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28295168->28299263] (req 28295168->28299263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.679882:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2fe0e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28295168->28299263] (req 28295168->28299263) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.679901:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2fe0e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28295168->28299263] (req 28295168->28299263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.679909:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2fe0e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28295168->28299263] (req 28295168->28299263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760783.680147:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4074e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2359:x1876699512747904:12345-192.168.203.56@tcp:4:dd.0 Request processed in 30565us (30758us total) trans 4294974227 rc 0/0 00000100:00100000:1.0:1789760783.680161:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6938 00000100:00100000:1.0:1789760783.680171:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 763160954536 time 763213867322 state 0x2 ref 0 00000001:00080000:0.0:1789760783.680291:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974227 is committed 00000100:00100000:1.0:1789760783.689274:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760783.689539:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512748032 00000100:00100000:3.0:1789760783.689549:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512748032 00000100:00100000:3.0:1789760783.689605:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 763160954536 time 763213867322 state 0x0 ref 1 00000100:00100000:3.0:1789760783.689621:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 763214127738 time 763213867322 state 0x1 ref 1 00000100:00100000:3.0:1789760783.689744:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 763214127738 time 763254042801 state 0x0 ref 1 00000100:00100000:3.0:1789760783.689760:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac287b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512748032:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760783.689806:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28299264, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0xd807 00010000:00010000:3.0:1789760783.689815:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2fe15 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.689832:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=763s v=(0 0) 00010000:00010000:3.0:1789760783.689836:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760783.689842:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2fe15 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28299264->28303359] (req 28299264->28303359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.689857:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2fe15 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28299264->28303359] (req 28299264->28303359) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760783.720220:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974228 is committed 00002000:00010000:3.0:1789760783.720451:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28299264 -> 28303360 00010000:00010000:3.0:1789760783.720466:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2fe15 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28299264->28303359] (req 28299264->28303359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.720479:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2fe15 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28299264->28303359] (req 28299264->28303359) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.720487:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2fe15 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28299264->28303359] (req 28299264->28303359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.720494:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760783.720499:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2fe15 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28299264->28303359] (req 28299264->28303359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.720509:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2fe15 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28299264->28303359] (req 28299264->28303359) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.720527:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2fe15 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28299264->28303359] (req 28299264->28303359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.720535:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2fe15 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28299264->28303359] (req 28299264->28303359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760783.720626:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac287b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512748032:12345-192.168.203.56@tcp:4:dd.0 Request processed in 30864us (31358us total) trans 4294974228 rc 0/0 00000100:00100000:3.0:1789760783.720637:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6939 00000100:00100000:3.0:1789760783.720647:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 763214127738 time 763254042801 state 0x2 ref 0 00000100:00100000:2.0:1789760783.726148:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760783.726243:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512748160 00000100:00100000:0.0:1789760783.726253:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512748160 00000100:00100000:0.0:1789760783.726303:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 763214127738 time 763254042801 state 0x0 ref 1 00000100:00100000:0.0:1789760783.726317:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 763254303217 time 763254042801 state 0x1 ref 1 00000100:00100000:0.0:1789760783.726351:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 763254303217 time 763290650530 state 0x0 ref 1 00000100:00100000:0.0:1789760783.726368:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2746300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512748160:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760783.726413:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28303360, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0xd807 00010000:00010000:0.0:1789760783.726422:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2fe1c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.726437:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=763s v=(0 0) 00010000:00010000:0.0:1789760783.726442:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760783.726448:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2fe1c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28303360->28307455] (req 28303360->28307455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.726461:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2fe1c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28303360->28307455] (req 28303360->28307455) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.731047:0:7647:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:2.0:1789760783.756098:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974229 is committed 00002000:00010000:1.0:1789760783.756203:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28303360 -> 28307456 00010000:00010000:1.0:1789760783.756218:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2fe1c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28303360->28307455] (req 28303360->28307455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.756231:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2fe1c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28303360->28307455] (req 28303360->28307455) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.756240:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2fe1c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28303360->28307455] (req 28303360->28307455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.756247:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760783.756251:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2fe1c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28303360->28307455] (req 28303360->28307455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.756259:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2fe1c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28303360->28307455] (req 28303360->28307455) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.756276:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2fe1c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28303360->28307455] (req 28303360->28307455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.756283:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2fe1c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28303360->28307455] (req 28303360->28307455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760783.756376:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2746300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512748160:12345-192.168.203.56@tcp:4:dd.0 Request processed in 30011us (30234us total) trans 4294974229 rc 0/0 00000100:00100000:1.0:1789760783.756387:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6940 00000100:00100000:1.0:1789760783.756396:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 763254303217 time 763290650530 state 0x2 ref 0 00000100:00100000:0.0:1789760783.759854:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760783.759895:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512748288 00000100:00100000:1.0:1789760783.759904:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512748288 00000100:00100000:1.0:1789760783.759951:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 763254303217 time 763290650530 state 0x0 ref 1 00000100:00100000:1.0:1789760783.759966:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 763290910946 time 763290650530 state 0x1 ref 1 00000100:00100000:1.0:1789760783.759992:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 763290910946 time 763324291935 state 0x0 ref 1 00000100:00100000:1.0:1789760783.760003:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4075500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512748288:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760783.760061:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28307456, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0xd807 00010000:00010000:1.0:1789760783.760071:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2fe23 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.760086:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=763s v=(0 0) 00010000:00010000:1.0:1789760783.760091:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760783.760097:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2fe23 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28307456->28311551] (req 28307456->28311551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.760110:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2fe23 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28307456->28311551] (req 28307456->28311551) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760783.789184:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974230 is committed 00002000:00010000:1.0:1789760783.789269:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28307456 -> 28311552 00010000:00010000:1.0:1789760783.789286:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2fe23 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28307456->28311551] (req 28307456->28311551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.789299:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2fe23 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28307456->28311551] (req 28307456->28311551) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.789308:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2fe23 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28307456->28311551] (req 28307456->28311551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.789315:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760783.789320:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2fe23 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28307456->28311551] (req 28307456->28311551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.789330:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2fe23 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28307456->28311551] (req 28307456->28311551) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.789348:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2fe23 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28307456->28311551] (req 28307456->28311551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.789355:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2fe23 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28307456->28311551] (req 28307456->28311551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760783.789472:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4075500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512748288:12345-192.168.203.56@tcp:4:dd.0 Request processed in 29466us (29621us total) trans 4294974230 rc 0/0 00000100:00100000:1.0:1789760783.789487:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6941 00000100:00100000:1.0:1789760783.789496:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 763290910946 time 763324291935 state 0x2 ref 0 00000100:00100000:2.0:1789760783.792400:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760783.792538:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512748416 00000100:00100000:3.0:1789760783.792547:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512748416 00000100:00100000:3.0:1789760783.792669:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 763290910946 time 763324291935 state 0x0 ref 1 00000100:00100000:3.0:1789760783.792685:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 763324552351 time 763324291935 state 0x1 ref 1 00000100:00100000:3.0:1789760783.792902:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 763324552351 time 763357200960 state 0x0 ref 1 00000100:00100000:3.0:1789760783.792915:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2744a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512748416:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760783.792961:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28311552, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0xd807 00010000:00010000:3.0:1789760783.792970:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2fe2a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.792986:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=763s v=(0 0) 00010000:00010000:3.0:1789760783.792991:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760783.792997:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2fe2a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28311552->28315647] (req 28311552->28315647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.793010:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2fe2a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28311552->28315647] (req 28311552->28315647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760783.825149:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974231 is committed 00002000:00010000:3.0:1789760783.825259:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28311552 -> 28315648 00002000:00010000:3.0:1789760783.825264:0:6709:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb blocks from disk: 55303 -> 57351 00010000:00010000:3.0:1789760783.825286:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2fe2a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28311552->28315647] (req 28311552->28315647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.825299:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2fe2a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28311552->28315647] (req 28311552->28315647) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.825309:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2fe2a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28311552->28315647] (req 28311552->28315647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.825316:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760783.825321:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2fe2a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28311552->28315647] (req 28311552->28315647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.825335:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2fe2a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28311552->28315647] (req 28311552->28315647) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.825354:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2fe2a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28311552->28315647] (req 28311552->28315647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.825361:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2fe2a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28311552->28315647] (req 28311552->28315647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760783.825628:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2744a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512748416:12345-192.168.203.56@tcp:4:dd.0 Request processed in 32710us (33233us total) trans 4294974231 rc 0/0 00000100:00100000:3.0:1789760783.825640:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6942 00000100:00100000:3.0:1789760783.825649:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 763324552351 time 763357200960 state 0x2 ref 0 00000100:00100000:2.0:1789760783.829186:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760783.829291:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512748544 00000100:00100000:1.0:1789760783.829300:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512748544 00000100:00100000:1.0:1789760783.829347:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 763324552351 time 763357200960 state 0x0 ref 1 00000100:00100000:1.0:1789760783.829362:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 763357461376 time 763357200960 state 0x1 ref 1 00000100:00100000:1.0:1789760783.829430:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 763357461376 time 763393729160 state 0x0 ref 1 00000100:00100000:1.0:1789760783.829443:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac437fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512748544:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760783.829486:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28315648, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0xe007 00010000:00010000:1.0:1789760783.829495:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2fe31 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.829510:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=763s v=(0 0) 00010000:00010000:1.0:1789760783.829514:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760783.829521:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2fe31 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28315648->28319743] (req 28315648->28319743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.829534:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2fe31 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28315648->28319743] (req 28315648->28319743) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760783.859602:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974232 is committed 00002000:00010000:3.0:1789760783.859882:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28315648 -> 28319744 00010000:00010000:3.0:1789760783.859897:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2fe31 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28315648->28319743] (req 28315648->28319743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.859910:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2fe31 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28315648->28319743] (req 28315648->28319743) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.859919:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2fe31 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28315648->28319743] (req 28315648->28319743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.859926:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760783.859931:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2fe31 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28315648->28319743] (req 28315648->28319743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.859940:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2fe31 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28315648->28319743] (req 28315648->28319743) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.859957:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2fe31 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28315648->28319743] (req 28315648->28319743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.859966:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2fe31 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28315648->28319743] (req 28315648->28319743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760783.860061:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac437fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512748544:12345-192.168.203.56@tcp:4:dd.0 Request processed in 30615us (30878us total) trans 4294974232 rc 0/0 00000100:00100000:3.0:1789760783.860072:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6943 00000100:00100000:3.0:1789760783.860080:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 763357461376 time 763393729160 state 0x2 ref 0 00000100:00100000:3.0:1789760783.862840:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760783.863001:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512748672 00000100:00100000:1.0:1789760783.863010:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512748672 00000100:00100000:1.0:1789760783.863070:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 763357461376 time 763393729160 state 0x0 ref 1 00000100:00100000:1.0:1789760783.863086:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 763393989576 time 763393729160 state 0x1 ref 1 00000100:00100000:1.0:1789760783.863115:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 763393989576 time 763427415024 state 0x0 ref 1 00000100:00100000:1.0:1789760783.863127:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac531d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512748672:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760783.863172:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28319744, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0xe007 00010000:00010000:1.0:1789760783.863181:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2fe38 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.863196:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=763s v=(0 0) 00010000:00010000:1.0:1789760783.863201:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760783.863207:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2fe38 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28319744->28323839] (req 28319744->28323839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.863220:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2fe38 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28319744->28323839] (req 28319744->28323839) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760783.893161:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974233 is committed 00002000:00010000:3.0:1789760783.893258:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28319744 -> 28323840 00010000:00010000:3.0:1789760783.893273:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2fe38 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28319744->28323839] (req 28319744->28323839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.893285:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2fe38 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28319744->28323839] (req 28319744->28323839) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.893294:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2fe38 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28319744->28323839] (req 28319744->28323839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.893305:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760783.893310:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2fe38 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28319744->28323839] (req 28319744->28323839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.893318:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2fe38 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28319744->28323839] (req 28319744->28323839) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.893335:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2fe38 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28319744->28323839] (req 28319744->28323839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.893342:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2fe38 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28319744->28323839] (req 28319744->28323839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760783.893494:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac531d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512748672:12345-192.168.203.56@tcp:4:dd.0 Request processed in 30363us (30657us total) trans 4294974233 rc 0/0 00000100:00100000:3.0:1789760783.893503:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6944 00000100:00100000:3.0:1789760783.893512:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 763393989576 time 763427415024 state 0x2 ref 0 00000100:00100000:3.0:1789760783.896087:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760783.896163:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512748800 00000100:00100000:0.0:1789760783.896172:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512748800 00000100:00100000:0.0:1789760783.896224:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 763393989576 time 763427415024 state 0x0 ref 1 00000100:00100000:0.0:1789760783.896240:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 763427675440 time 763427415024 state 0x1 ref 1 00000100:00100000:0.0:1789760783.896273:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 763427675440 time 763460572653 state 0x0 ref 1 00000100:00100000:0.0:1789760783.896284:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac531ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512748800:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760783.896332:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28323840, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0xe007 00010000:00010000:0.0:1789760783.896340:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2fe3f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.896357:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=763s v=(0 0) 00010000:00010000:0.0:1789760783.896361:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760783.896367:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2fe3f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28323840->28327935] (req 28323840->28327935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.896380:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2fe3f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28323840->28327935] (req 28323840->28327935) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760783.918810:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28323840 -> 28327936 00000001:00080000:3.0:1789760783.918827:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974234 is committed 00010000:00010000:0.0:1789760783.918832:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2fe3f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28323840->28327935] (req 28323840->28327935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.918845:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2fe3f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28323840->28327935] (req 28323840->28327935) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.918861:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2fe3f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28323840->28327935] (req 28323840->28327935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.918869:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760783.918874:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2fe3f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28323840->28327935] (req 28323840->28327935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.918881:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2fe3f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28323840->28327935] (req 28323840->28327935) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.918969:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2fe3f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28323840->28327935] (req 28323840->28327935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.918978:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2fe3f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28323840->28327935] (req 28323840->28327935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760783.919075:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac531ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512748800:12345-192.168.203.56@tcp:4:dd.0 Request processed in 22787us (22991us total) trans 4294974234 rc 0/0 00000100:00100000:0.0:1789760783.919154:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6945 00000100:00100000:0.0:1789760783.919164:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 763427675440 time 763460572653 state 0x2 ref 0 00000100:00100000:0.0:1789760783.921311:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760783.921417:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512748928 00000100:00100000:3.0:1789760783.921426:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512748928 00000100:00100000:3.0:1789760783.921479:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 763427675440 time 763460572653 state 0x0 ref 1 00000100:00100000:3.0:1789760783.921491:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 763460833069 time 763460572653 state 0x1 ref 1 00000100:00100000:3.0:1789760783.921517:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 763460833069 time 763485817434 state 0x0 ref 1 00000100:00100000:3.0:1789760783.921528:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4076300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512748928:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760783.921881:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28327936, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0xe007 00010000:00010000:3.0:1789760783.921888:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2fe46 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.921904:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=763s v=(0 0) 00010000:00010000:3.0:1789760783.921908:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760783.921914:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2fe46 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28327936->28332031] (req 28327936->28332031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.921926:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2fe46 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28327936->28332031] (req 28327936->28332031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760783.943952:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974235 is committed 00002000:00010000:3.0:1789760783.944037:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28327936 -> 28332032 00010000:00010000:3.0:1789760783.944053:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2fe46 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28327936->28332031] (req 28327936->28332031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.944066:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2fe46 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28327936->28332031] (req 28327936->28332031) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.944076:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2fe46 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28327936->28332031] (req 28327936->28332031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.944084:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760783.944089:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2fe46 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28327936->28332031] (req 28327936->28332031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.944098:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2fe46 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28327936->28332031] (req 28327936->28332031) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.944115:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2fe46 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28327936->28332031] (req 28327936->28332031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.944123:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2fe46 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28327936->28332031] (req 28327936->28332031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760783.944248:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4076300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512748928:12345-192.168.203.56@tcp:4:dd.0 Request processed in 22717us (22941us total) trans 4294974235 rc 0/0 00000100:00100000:3.0:1789760783.944258:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6946 00000100:00100000:3.0:1789760783.944268:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 763460833069 time 763485817434 state 0x2 ref 0 00000100:00100000:3.0:1789760783.947937:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760783.948030:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512749056 00000100:00100000:1.0:1789760783.948040:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512749056 00000100:00100000:1.0:1789760783.948090:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 763460833069 time 763485817434 state 0x0 ref 1 00000100:00100000:1.0:1789760783.948105:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 763486077850 time 763485817434 state 0x1 ref 1 00000100:00100000:1.0:1789760783.948135:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 763486077850 time 763512434992 state 0x0 ref 1 00000100:00100000:1.0:1789760783.948146:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac531f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512749056:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760783.948198:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28332032, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0xe007 00010000:00010000:1.0:1789760783.948207:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2fe4d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.948222:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=763s v=(0 0) 00010000:00010000:1.0:1789760783.948227:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760783.948233:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2fe4d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28332032->28336127] (req 28332032->28336127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.948247:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2fe4d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28332032->28336127] (req 28332032->28336127) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760783.980183:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974236 is committed 00002000:00010000:2.0:1789760783.980319:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28332032 -> 28336128 00010000:00010000:2.0:1789760783.980335:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2fe4d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28332032->28336127] (req 28332032->28336127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.980348:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2fe4d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28332032->28336127] (req 28332032->28336127) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.980357:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2fe4d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28332032->28336127] (req 28332032->28336127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.980365:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760783.980370:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2fe4d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28332032->28336127] (req 28332032->28336127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.980378:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2fe4d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28332032->28336127] (req 28332032->28336127) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.980398:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2fe4d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28332032->28336127] (req 28332032->28336127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.980411:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c2fe4d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28332032->28336127] (req 28332032->28336127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760783.980590:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac531f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512749056:12345-192.168.203.56@tcp:4:dd.0 Request processed in 32439us (32655us total) trans 4294974236 rc 0/0 00000100:00100000:2.0:1789760783.980600:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6947 00000100:00100000:2.0:1789760783.980610:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 763486077850 time 763512434992 state 0x2 ref 0 00000100:00100000:1.0:1789760783.985932:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760783.986010:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512749184 00000100:00100000:0.0:1789760783.986029:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512749184 00000100:00100000:0.0:1789760783.986082:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 763486077850 time 763512434992 state 0x0 ref 1 00000100:00100000:0.0:1789760783.986099:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 763512695408 time 763512434992 state 0x1 ref 1 00000100:00100000:0.0:1789760783.986162:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 763512695408 time 763550461282 state 0x0 ref 1 00000100:00100000:0.0:1789760783.986176:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2878000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512749184:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760783.986232:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28336128, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0xe007 00010000:00010000:0.0:1789760783.986241:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2fe54 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.986256:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=763s v=(0 0) 00010000:00010000:0.0:1789760783.986260:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760783.986266:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2fe54 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28336128->28340223] (req 28336128->28340223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.986280:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2fe54 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28336128->28340223] (req 28336128->28340223) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760784.036293:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974237 is committed 00002000:00010000:1.0:1789760784.036383:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28336128 -> 28340224 00010000:00010000:1.0:1789760784.036402:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2fe54 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28336128->28340223] (req 28336128->28340223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.036416:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2fe54 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28336128->28340223] (req 28336128->28340223) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.036426:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2fe54 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28336128->28340223] (req 28336128->28340223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.036434:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760784.036439:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2fe54 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28336128->28340223] (req 28336128->28340223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.036454:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2fe54 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28336128->28340223] (req 28336128->28340223) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.036475:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2fe54 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28336128->28340223] (req 28336128->28340223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.036483:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2fe54 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28336128->28340223] (req 28336128->28340223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760784.036846:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2878000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512749184:12345-192.168.203.56@tcp:4:dd.0 Request processed in 50510us (50759us total) trans 4294974237 rc 0/0 00000100:00100000:1.0:1789760784.036860:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6948 00000100:00100000:1.0:1789760784.036871:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 763512695408 time 763550461282 state 0x2 ref 0 00000100:00100000:1.0:1789760784.039616:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760784.039873:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512749312 00000100:00100000:3.0:1789760784.039884:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512749312 00000100:00100000:3.0:1789760784.039942:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 763512695408 time 763550461282 state 0x0 ref 1 00000100:00100000:3.0:1789760784.039956:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 763550721698 time 763550461282 state 0x1 ref 1 00000100:00100000:3.0:1789760784.039984:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 763550721698 time 763604283677 state 0x0 ref 1 00000100:00100000:3.0:1789760784.039997:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2879180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512749312:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760784.040062:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28340224, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0xe007 00010000:00010000:3.0:1789760784.040072:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2fe5b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.040087:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=763s v=(0 0) 00010000:00010000:3.0:1789760784.040091:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760784.040097:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2fe5b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28340224->28344319] (req 28340224->28344319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.040110:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2fe5b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28340224->28344319] (req 28340224->28344319) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760784.081135:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974238 is committed 00002000:00010000:0.0:1789760784.081361:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28340224 -> 28344320 00010000:00010000:0.0:1789760784.081378:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2fe5b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28340224->28344319] (req 28340224->28344319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.081391:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2fe5b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28340224->28344319] (req 28340224->28344319) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.081401:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2fe5b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28340224->28344319] (req 28340224->28344319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.081410:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760784.081415:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2fe5b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28340224->28344319] (req 28340224->28344319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.081425:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2fe5b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28340224->28344319] (req 28340224->28344319) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.081591:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2fe5b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28340224->28344319] (req 28340224->28344319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.081602:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c2fe5b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28340224->28344319] (req 28340224->28344319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760784.081873:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2879180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512749312:12345-192.168.203.56@tcp:4:dd.0 Request processed in 41873us (42257us total) trans 4294974238 rc 0/0 00000100:00100000:0.0:1789760784.081884:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6949 00000100:00100000:0.0:1789760784.082288:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 763550721698 time 763604283677 state 0x2 ref 0 00000100:00100000:3.0:1789760784.090424:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760784.091128:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512749440 00000100:00100000:2.0:1789760784.091138:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512749440 00000100:00100000:2.0:1789760784.091196:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 763550721698 time 763604283677 state 0x0 ref 1 00000100:00100000:2.0:1789760784.091212:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 763604544093 time 763604283677 state 0x1 ref 1 00000100:00100000:2.0:1789760784.091250:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 763604544093 time 763655549344 state 0x0 ref 1 00000100:00100000:2.0:1789760784.091264:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac531df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512749440:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760784.091314:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28344320, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0xe007 00010000:00010000:2.0:1789760784.091324:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2fe62 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.091341:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=763s v=(0 0) 00010000:00010000:2.0:1789760784.091345:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760784.091352:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2fe62 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28344320->28348415] (req 28344320->28348415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.091366:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2fe62 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28344320->28348415] (req 28344320->28348415) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760784.117921:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28344320 -> 28348416 00010000:00010000:1.0:1789760784.117940:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2fe62 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28344320->28348415] (req 28344320->28348415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.117953:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2fe62 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28344320->28348415] (req 28344320->28348415) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.117962:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2fe62 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28344320->28348415] (req 28344320->28348415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.117974:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40e00 already unused, calling callback (ffffffffc15d8350) 00000001:00080000:3.0:1789760784.117975:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974239 is committed 00010000:00010000:1.0:1789760784.117980:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2fe62 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28344320->28348415] (req 28344320->28348415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.117989:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2fe62 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28344320->28348415] (req 28344320->28348415) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.118007:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2fe62 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28344320->28348415] (req 28344320->28348415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.118027:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2fe62 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28344320->28348415] (req 28344320->28348415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760784.118112:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac531df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512749440:12345-192.168.203.56@tcp:4:dd.0 Request processed in 26842us (27688us total) trans 4294974239 rc 0/0 00000100:00100000:1.0:1789760784.118123:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6950 00000100:00100000:1.0:1789760784.118132:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 763604544093 time 763655549344 state 0x2 ref 0 00000100:00100000:1.0:1789760784.122977:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760784.123064:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512749568 00000100:00100000:0.0:1789760784.123074:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512749568 00000100:00100000:0.0:1789760784.123127:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 763604544093 time 763655549344 state 0x0 ref 1 00000100:00100000:0.0:1789760784.123220:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 763655809760 time 763655549344 state 0x1 ref 1 00000100:00100000:0.0:1789760784.123264:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 763655809760 time 763687561995 state 0x0 ref 1 00000100:00100000:0.0:1789760784.123284:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2878380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512749568:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760784.123348:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28348416, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0xe007 00010000:00010000:0.0:1789760784.123359:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2fe69 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.123376:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=763s v=(0 0) 00010000:00010000:0.0:1789760784.123382:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760784.123391:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2fe69 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28348416->28352511] (req 28348416->28352511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.123408:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2fe69 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28348416->28352511] (req 28348416->28352511) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760784.152072:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974240 is committed 00002000:00010000:1.0:1789760784.152231:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28348416 -> 28352512 00010000:00010000:1.0:1789760784.152250:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2fe69 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28348416->28352511] (req 28348416->28352511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.152264:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2fe69 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28348416->28352511] (req 28348416->28352511) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.152279:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2fe69 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28348416->28352511] (req 28348416->28352511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.152287:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760784.152292:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2fe69 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28348416->28352511] (req 28348416->28352511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.152302:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2fe69 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28348416->28352511] (req 28348416->28352511) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.152322:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2fe69 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28348416->28352511] (req 28348416->28352511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.152330:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c2fe69 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28348416->28352511] (req 28348416->28352511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760784.152425:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2878380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512749568:12345-192.168.203.56@tcp:4:dd.0 Request processed in 29141us (29451us total) trans 4294974240 rc 0/0 00000100:00100000:1.0:1789760784.152436:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6951 00000100:00100000:1.0:1789760784.152445:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 763655809760 time 763687561995 state 0x2 ref 0 00000100:00100000:2.0:1789760784.173673:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760784.174088:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512749696 00000100:00100000:0.0:1789760784.174098:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512749696 00000100:00100000:0.0:1789760784.174148:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 763655809760 time 763687561995 state 0x0 ref 1 00000100:00100000:0.0:1789760784.174162:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 763687822411 time 763687561995 state 0x1 ref 1 00000100:00100000:0.0:1789760784.174230:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 763687822411 time 763738529278 state 0x0 ref 1 00000100:00100000:0.0:1789760784.174243:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac437d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512749696:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760784.174294:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28352512, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0xe007 00010000:00010000:0.0:1789760784.174304:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2fe70 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.174323:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=763s v=(0 0) 00010000:00010000:0.0:1789760784.174328:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760784.174335:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2fe70 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28352512->28356607] (req 28352512->28356607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.174348:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2fe70 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28352512->28356607] (req 28352512->28356607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760784.218616:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974241 is committed 00002000:00010000:1.0:1789760784.218880:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28352512 -> 28356608 00010000:00010000:1.0:1789760784.218896:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2fe70 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28352512->28356607] (req 28352512->28356607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.218913:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2fe70 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28352512->28356607] (req 28352512->28356607) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.218920:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2fe70 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28352512->28356607] (req 28352512->28356607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.218927:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760784.218931:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2fe70 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28352512->28356607] (req 28352512->28356607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.218939:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2fe70 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28352512->28356607] (req 28352512->28356607) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.218958:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2fe70 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28352512->28356607] (req 28352512->28356607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.218965:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c2fe70 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28352512->28356607] (req 28352512->28356607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760784.219066:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac437d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512749696:12345-192.168.203.56@tcp:4:dd.0 Request processed in 44820us (45397us total) trans 4294974241 rc 0/0 00000100:00100000:1.0:1789760784.219076:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6952 00000100:00100000:1.0:1789760784.219086:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 763687822411 time 763738529278 state 0x2 ref 0 00000100:00100000:0.0:1789760784.222568:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760784.222656:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512749824 00000100:00100000:1.0:1789760784.222665:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512749824 00000100:00100000:1.0:1789760784.222755:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 763687822411 time 763738529278 state 0x0 ref 1 00000100:00100000:1.0:1789760784.222771:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 763738789694 time 763738529278 state 0x1 ref 1 00000100:00100000:1.0:1789760784.222794:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 763738789694 time 763787093621 state 0x0 ref 1 00000100:00100000:1.0:1789760784.222808:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4075180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512749824:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760784.222862:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28356608, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0xe007 00010000:00010000:1.0:1789760784.222871:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2fe77 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.222887:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=763s v=(0 0) 00010000:00010000:1.0:1789760784.222892:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760784.222897:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2fe77 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28356608->28360703] (req 28356608->28360703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.222911:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2fe77 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28356608->28360703] (req 28356608->28360703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760784.262498:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974242 is committed 00002000:00010000:1.0:1789760784.262673:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28356608 -> 28360704 00010000:00010000:1.0:1789760784.262690:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2fe77 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28356608->28360703] (req 28356608->28360703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.262729:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2fe77 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28356608->28360703] (req 28356608->28360703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.262743:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2fe77 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28356608->28360703] (req 28356608->28360703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.262750:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760784.262755:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2fe77 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28356608->28360703] (req 28356608->28360703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.262774:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2fe77 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28356608->28360703] (req 28356608->28360703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.262794:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2fe77 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28356608->28360703] (req 28356608->28360703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.262802:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2fe77 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28356608->28360703] (req 28356608->28360703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760784.262899:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4075180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512749824:12345-192.168.203.56@tcp:4:dd.0 Request processed in 40088us (40334us total) trans 4294974242 rc 0/0 00000100:00100000:1.0:1789760784.262908:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6953 00000100:00100000:1.0:1789760784.262917:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 763738789694 time 763787093621 state 0x2 ref 0 00000100:00100000:3.0:1789760784.268974:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760784.269087:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512749952 00000100:00100000:1.0:1789760784.269096:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512749952 00000100:00100000:1.0:1789760784.269154:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 763738789694 time 763787093621 state 0x0 ref 1 00000100:00100000:1.0:1789760784.269170:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 763787354037 time 763787093621 state 0x1 ref 1 00000100:00100000:1.0:1789760784.269204:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 763787354037 time 763833504183 state 0x0 ref 1 00000100:00100000:1.0:1789760784.269216:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac531c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512749952:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760784.269263:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28360704, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0xe007 00010000:00010000:1.0:1789760784.269272:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2fe7e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.269287:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=763s v=(0 0) 00010000:00010000:1.0:1789760784.269291:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760784.269296:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2fe7e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28360704->28364799] (req 28360704->28364799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.269309:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2fe7e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28360704->28364799] (req 28360704->28364799) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760784.313894:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974243 is committed 00002000:00010000:0.0:1789760784.316955:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28360704 -> 28364800 00010000:00010000:0.0:1789760784.316975:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2fe7e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28360704->28364799] (req 28360704->28364799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.316989:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2fe7e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28360704->28364799] (req 28360704->28364799) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.316999:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2fe7e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28360704->28364799] (req 28360704->28364799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.317006:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760784.317011:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2fe7e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28360704->28364799] (req 28360704->28364799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.317030:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2fe7e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28360704->28364799] (req 28360704->28364799) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.317054:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2fe7e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28360704->28364799] (req 28360704->28364799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.317063:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2fe7e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28360704->28364799] (req 28360704->28364799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760784.317171:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac531c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512749952:12345-192.168.203.56@tcp:4:dd.0 Request processed in 47951us (48202us total) trans 4294974243 rc 0/0 00000100:00100000:0.0:1789760784.317181:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6954 00000100:00100000:0.0:1789760784.317190:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 763787354037 time 763833504183 state 0x2 ref 0 00000100:00100000:2.0:1789760784.320490:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760784.323873:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512750080 00000100:00100000:1.0:1789760784.323889:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512750080 00000100:00100000:1.0:1789760784.323955:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 763787354037 time 763833504183 state 0x0 ref 1 00000100:00100000:1.0:1789760784.323972:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 763833764599 time 763833504183 state 0x1 ref 1 00000100:00100000:1.0:1789760784.324000:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 763833764599 time 763888299761 state 0x0 ref 1 00000100:00100000:1.0:1789760784.324024:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac437df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512750080:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760784.324625:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28364800, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0xe007 00010000:00010000:1.0:1789760784.324637:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2fe85 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.324653:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=763s v=(0 0) 00010000:00010000:1.0:1789760784.324658:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760784.324664:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2fe85 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28364800->28368895] (req 28364800->28368895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.324678:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2fe85 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28364800->28368895] (req 28364800->28368895) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760784.364643:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974244 is committed 00002000:00010000:3.0:1789760784.365220:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28364800 -> 28368896 00010000:00010000:3.0:1789760784.365233:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2fe85 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28364800->28368895] (req 28364800->28368895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.365243:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2fe85 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28364800->28368895] (req 28364800->28368895) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.365249:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2fe85 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28364800->28368895] (req 28364800->28368895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.365253:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760784.365257:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2fe85 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28364800->28368895] (req 28364800->28368895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.365264:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2fe85 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28364800->28368895] (req 28364800->28368895) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.365283:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2fe85 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28364800->28368895] (req 28364800->28368895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.365289:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2fe85 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28364800->28368895] (req 28364800->28368895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760784.365390:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac437df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512750080:12345-192.168.203.56@tcp:4:dd.0 Request processed in 41373us (44904us total) trans 4294974244 rc 0/0 00000100:00100000:3.0:1789760784.365410:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6955 00000100:00100000:3.0:1789760784.365421:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 763833764599 time 763888299761 state 0x2 ref 0 00000100:00100000:3.0:1789760784.370595:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760784.370685:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512750208 00000100:00100000:1.0:1789760784.370825:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512750208 00000100:00100000:1.0:1789760784.370880:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 763833764599 time 763888299761 state 0x0 ref 1 00000100:00100000:1.0:1789760784.370895:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 763888560177 time 763888299761 state 0x1 ref 1 00000100:00100000:1.0:1789760784.370927:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 763888560177 time 763935226471 state 0x0 ref 1 00000100:00100000:1.0:1789760784.370939:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac531d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512750208:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760784.370999:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28368896, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0xe007 00010000:00010000:1.0:1789760784.371008:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2fe8c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.371036:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=763s v=(0 0) 00010000:00010000:1.0:1789760784.371039:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760784.371046:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2fe8c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28368896->28372991] (req 28368896->28372991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.371064:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2fe8c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28368896->28372991] (req 28368896->28372991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760784.399608:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974245 is committed 00002000:00010000:0.0:1789760784.399716:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28368896 -> 28372992 00010000:00010000:0.0:1789760784.399731:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2fe8c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28368896->28372991] (req 28368896->28372991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.399743:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2fe8c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28368896->28372991] (req 28368896->28372991) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.399751:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2fe8c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28368896->28372991] (req 28368896->28372991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.399758:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760784.399762:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2fe8c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28368896->28372991] (req 28368896->28372991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.399771:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2fe8c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28368896->28372991] (req 28368896->28372991) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.399788:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2fe8c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28368896->28372991] (req 28368896->28372991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.399795:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c2fe8c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28368896->28372991] (req 28368896->28372991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760784.399881:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac531d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512750208:12345-192.168.203.56@tcp:4:dd.0 Request processed in 28938us (29289us total) trans 4294974245 rc 0/0 00000100:00100000:0.0:1789760784.399891:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6956 00000100:00100000:0.0:1789760784.399901:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 763888560177 time 763935226471 state 0x2 ref 0 00000100:00100000:1.0:1789760784.408522:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760784.408621:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512750336 00000100:00100000:0.0:1789760784.408632:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512750336 00000100:00100000:0.0:1789760784.408689:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 763888560177 time 763935226471 state 0x0 ref 1 00000100:00100000:0.0:1789760784.408740:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 763935486887 time 763935226471 state 0x1 ref 1 00000100:00100000:0.0:1789760784.408763:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 763935486887 time 763973063358 state 0x0 ref 1 00000100:00100000:0.0:1789760784.408777:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2879c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512750336:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760784.408827:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28372992, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0xe007 00010000:00010000:0.0:1789760784.408838:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2fe93 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.408856:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=763s v=(0 0) 00010000:00010000:0.0:1789760784.408860:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760784.408866:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2fe93 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28372992->28377087] (req 28372992->28377087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.408880:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2fe93 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28372992->28377087] (req 28372992->28377087) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760784.451160:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974246 is committed 00002000:00010000:2.0:1789760784.451316:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28372992 -> 28377088 00010000:00010000:2.0:1789760784.451332:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2fe93 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28372992->28377087] (req 28372992->28377087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.451345:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2fe93 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28372992->28377087] (req 28372992->28377087) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.451354:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2fe93 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28372992->28377087] (req 28372992->28377087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.451361:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760784.451366:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2fe93 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28372992->28377087] (req 28372992->28377087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.451375:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2fe93 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28372992->28377087] (req 28372992->28377087) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.451401:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2fe93 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28372992->28377087] (req 28372992->28377087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.451526:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2fe93 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28372992->28377087] (req 28372992->28377087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760784.451622:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2879c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512750336:12345-192.168.203.56@tcp:4:dd.0 Request processed in 42842us (43106us total) trans 4294974246 rc 0/0 00000100:00100000:2.0:1789760784.451632:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6957 00000100:00100000:2.0:1789760784.451641:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 763935486887 time 763973063358 state 0x2 ref 0 00000100:00100000:3.0:1789760784.460439:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760784.460547:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512750464 00000100:00100000:1.0:1789760784.460557:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512750464 00000100:00100000:1.0:1789760784.460609:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 763935486887 time 763973063358 state 0x0 ref 1 00000100:00100000:1.0:1789760784.460625:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 763973323774 time 763973063358 state 0x1 ref 1 00000100:00100000:1.0:1789760784.460692:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 763973323774 time 764024991083 state 0x0 ref 1 00000100:00100000:1.0:1789760784.460724:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac531e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512750464:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760784.460774:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28377088, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0xe007 00010000:00010000:1.0:1789760784.460783:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2fe9a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.460800:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=764s v=(0 0) 00010000:00010000:1.0:1789760784.460804:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760784.460811:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2fe9a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28377088->28381183] (req 28377088->28381183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.460825:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2fe9a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28377088->28381183] (req 28377088->28381183) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760784.501815:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974247 is committed 00002000:00010000:2.0:1789760784.501963:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28377088 -> 28381184 00010000:00010000:2.0:1789760784.501986:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2fe9a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28377088->28381183] (req 28377088->28381183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.501999:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2fe9a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28377088->28381183] (req 28377088->28381183) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.502010:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2fe9a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28377088->28381183] (req 28377088->28381183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.502025:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760784.502029:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2fe9a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28377088->28381183] (req 28377088->28381183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.502043:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2fe9a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28377088->28381183] (req 28377088->28381183) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.502060:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2fe9a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28377088->28381183] (req 28377088->28381183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.502067:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c2fe9a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28377088->28381183] (req 28377088->28381183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760784.502181:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac531e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512750464:12345-192.168.203.56@tcp:4:dd.0 Request processed in 41455us (41747us total) trans 4294974247 rc 0/0 00000100:00100000:2.0:1789760784.502190:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6958 00000100:00100000:2.0:1789760784.502200:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 763973323774 time 764024991083 state 0x2 ref 0 00000100:00100000:0.0:1789760784.519284:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760784.519406:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512750592 00000100:00100000:2.0:1789760784.519415:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512750592 00000100:00100000:2.0:1789760784.519463:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 763973323774 time 764024991083 state 0x0 ref 1 00000100:00100000:2.0:1789760784.519474:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 764025251499 time 764024991083 state 0x1 ref 1 00000100:00100000:2.0:1789760784.519498:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 764025251499 time 764083796750 state 0x0 ref 1 00000100:00100000:2.0:1789760784.519510:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4074a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512750592:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760784.519557:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28381184, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0xe007 00010000:00010000:2.0:1789760784.519564:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2fea1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.519576:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=764s v=(0 0) 00010000:00010000:2.0:1789760784.519579:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760784.519584:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2fea1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28381184->28385279] (req 28381184->28385279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.519595:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2fea1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28381184->28385279] (req 28381184->28385279) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760784.602284:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974248 is committed 00002000:00010000:0.0:1789760784.602491:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28381184 -> 28385280 00002000:00010000:0.0:1789760784.602496:0:6710:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760783 -> 1789760784 00002000:00010000:0.0:1789760784.602498:0:6710:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760783 -> 1789760784 00010000:00010000:0.0:1789760784.602511:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2fea1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28381184->28385279] (req 28381184->28385279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.602524:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2fea1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28381184->28385279] (req 28381184->28385279) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.602533:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2fea1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28381184->28385279] (req 28381184->28385279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.602541:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760784.602546:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2fea1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28381184->28385279] (req 28381184->28385279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.602559:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2fea1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28381184->28385279] (req 28381184->28385279) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.602579:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2fea1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28381184->28385279] (req 28381184->28385279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.602588:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c2fea1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28381184->28385279] (req 28381184->28385279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760784.602680:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4074a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512750592:12345-192.168.203.56@tcp:4:dd.0 Request processed in 83166us (83398us total) trans 4294974248 rc 0/0 00000100:00100000:0.0:1789760784.602750:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6959 00000100:00100000:0.0:1789760784.602760:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 764025251499 time 764083796750 state 0x2 ref 0 00000100:00100000:1.0:1789760784.610208:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760784.610283:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512750720 00000100:00100000:2.0:1789760784.610295:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512750720 00000100:00100000:2.0:1789760784.610356:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 764025251499 time 764083796750 state 0x0 ref 1 00000100:00100000:2.0:1789760784.610385:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 764084057166 time 764083796750 state 0x1 ref 1 00000100:00100000:2.0:1789760784.610418:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 764084057166 time 764174718172 state 0x0 ref 1 00000100:00100000:2.0:1789760784.610430:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac287bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512750720:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760784.610477:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28385280, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0xe007 00010000:00010000:2.0:1789760784.610487:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2fea8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.610502:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=764s v=(0 0) 00010000:00010000:2.0:1789760784.610507:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760784.610512:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2fea8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28385280->28389375] (req 28385280->28389375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.610531:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2fea8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28385280->28389375] (req 28385280->28389375) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760784.723788:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974249 is committed 00002000:00010000:0.0:1789760784.723894:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28385280 -> 28389376 00010000:00010000:0.0:1789760784.723910:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2fea8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28385280->28389375] (req 28385280->28389375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.723923:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2fea8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28385280->28389375] (req 28385280->28389375) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.723941:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2fea8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28385280->28389375] (req 28385280->28389375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.723949:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760784.723955:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2fea8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28385280->28389375] (req 28385280->28389375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.723963:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2fea8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28385280->28389375] (req 28385280->28389375) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.723981:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2fea8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28385280->28389375] (req 28385280->28389375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.723989:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c2fea8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28385280->28389375] (req 28385280->28389375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760784.724086:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac287bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512750720:12345-192.168.203.56@tcp:4:dd.0 Request processed in 113653us (113884us total) trans 4294974249 rc 0/0 00000100:00100000:0.0:1789760784.724095:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6960 00000100:00100000:0.0:1789760784.724104:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 764084057166 time 764174718172 state 0x2 ref 0 00000100:00100000:3.0:1789760784.731198:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760784.731306:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512750848 00000100:00100000:2.0:1789760784.731315:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512750848 00000100:00100000:2.0:1789760784.731367:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 764084057166 time 764174718172 state 0x0 ref 1 00000100:00100000:2.0:1789760784.731383:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 764174978588 time 764174718172 state 0x1 ref 1 00000100:00100000:2.0:1789760784.731414:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 764174978588 time 764295714430 state 0x0 ref 1 00000100:00100000:2.0:1789760784.731425:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac531ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512750848:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760784.731470:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28389376, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0xe007 00010000:00010000:2.0:1789760784.731479:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2feaf lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.731504:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=764s v=(0 0) 00010000:00010000:2.0:1789760784.731509:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760784.731517:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2feaf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28389376->28393471] (req 28389376->28393471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.731531:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2feaf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28389376->28393471] (req 28389376->28393471) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.756485:0:5836:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:3.0:1789760784.765671:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28389376 -> 28393472 00010000:00010000:3.0:1789760784.765890:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2feaf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28389376->28393471] (req 28389376->28393471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.765905:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2feaf lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28389376->28393471] (req 28389376->28393471) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.765915:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2feaf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28389376->28393471] (req 28389376->28393471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.765924:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760784.765930:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2feaf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28389376->28393471] (req 28389376->28393471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.765939:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2feaf lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28389376->28393471] (req 28389376->28393471) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.765959:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2feaf lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28389376->28393471] (req 28389376->28393471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.765966:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c2feaf lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28389376->28393471] (req 28389376->28393471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760784.766124:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974250 is committed 00000100:00100000:3.0:1789760784.766420:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac531ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512750848:12345-192.168.203.56@tcp:4:dd.0 Request processed in 34990us (35224us total) trans 4294974250 rc 0/0 00000100:00100000:3.0:1789760784.766431:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6961 00000100:00100000:3.0:1789760784.766441:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 764174978588 time 764295714430 state 0x2 ref 0 00000100:00100000:3.0:1789760784.773384:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760784.773478:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512750976 00000100:00100000:0.0:1789760784.773488:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512750976 00000100:00100000:0.0:1789760784.773542:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 764174978588 time 764295714430 state 0x0 ref 1 00000100:00100000:0.0:1789760784.773558:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 764295974846 time 764295714430 state 0x1 ref 1 00000100:00100000:0.0:1789760784.773592:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 764295974846 time 764337891516 state 0x0 ref 1 00000100:00100000:0.0:1789760784.773604:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febe7905f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512750976:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760784.773652:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28393472, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0xe007 00010000:00010000:0.0:1789760784.773662:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2feb6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.773677:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=764s v=(0 0) 00010000:00010000:0.0:1789760784.773682:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760784.773851:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2feb6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28393472->28397567] (req 28393472->28397567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.773867:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2feb6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28393472->28397567] (req 28393472->28397567) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760784.808722:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28393472 -> 28397568 00010000:00010000:0.0:1789760784.808742:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2feb6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28393472->28397567] (req 28393472->28397567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.808755:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2feb6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28393472->28397567] (req 28393472->28397567) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.808764:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2feb6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28393472->28397567] (req 28393472->28397567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.808770:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760784.808774:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2feb6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28393472->28397567] (req 28393472->28397567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.808783:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2feb6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28393472->28397567] (req 28393472->28397567) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.808805:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2feb6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28393472->28397567] (req 28393472->28397567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.808812:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2feb6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28393472->28397567] (req 28393472->28397567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760784.808904:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febe7905f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2359:x1876699512750976:12345-192.168.203.56@tcp:4:dd.0 Request processed in 35298us (35524us total) trans 4294974251 rc 0/0 00000100:00100000:0.0:1789760784.808914:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6962 00000100:00100000:0.0:1789760784.808924:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 764295974846 time 764337891516 state 0x2 ref 0 00000001:00080000:3.0:1789760784.809313:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974251 is committed 00000100:00100000:1.0:1789760784.814087:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760784.814262:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512751104 00000100:00100000:0.0:1789760784.814273:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512751104 00000100:00100000:0.0:1789760784.814400:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 764295974846 time 764337891516 state 0x0 ref 1 00000100:00100000:0.0:1789760784.814416:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 764338151932 time 764337891516 state 0x1 ref 1 00000100:00100000:0.0:1789760784.814490:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 764338151932 time 764378789050 state 0x0 ref 1 00000100:00100000:0.0:1789760784.814503:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac287b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512751104:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760784.814555:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28397568, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0xe007 00010000:00010000:0.0:1789760784.814563:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2febd lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.814578:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=764s v=(0 0) 00010000:00010000:0.0:1789760784.814583:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760784.814590:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2febd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28397568->28401663] (req 28397568->28401663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.814604:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2febd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28397568->28401663] (req 28397568->28401663) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760784.852266:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28397568 -> 28401664 00010000:00010000:0.0:1789760784.852286:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2febd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28397568->28401663] (req 28397568->28401663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.852299:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2febd lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28397568->28401663] (req 28397568->28401663) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.852308:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2febd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28397568->28401663] (req 28397568->28401663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.852316:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760784.852321:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2febd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28397568->28401663] (req 28397568->28401663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.852331:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2febd lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28397568->28401663] (req 28397568->28401663) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.852350:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2febd lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28397568->28401663] (req 28397568->28401663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.852358:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2febd lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28397568->28401663] (req 28397568->28401663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760784.852454:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac287b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2359:x1876699512751104:12345-192.168.203.56@tcp:4:dd.0 Request processed in 37937us (38360us total) trans 4294974252 rc 0/0 00000100:00100000:0.0:1789760784.852465:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6963 00000100:00100000:0.0:1789760784.852487:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 764338151932 time 764378789050 state 0x2 ref 0 00000001:00080000:0.0:1789760784.852866:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974252 is committed 00000100:00100000:2.0:1789760784.858524:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760784.858632:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512751232 00000100:00100000:3.0:1789760784.858643:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512751232 00000100:00100000:3.0:1789760784.859140:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 764338151932 time 764378789050 state 0x0 ref 1 00000100:00100000:3.0:1789760784.859162:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 764379049466 time 764378789050 state 0x1 ref 1 00000100:00100000:3.0:1789760784.859240:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 764379049466 time 764423538622 state 0x0 ref 1 00000100:00100000:3.0:1789760784.859257:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac437e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512751232:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760784.859304:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28401664, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0xe007 00010000:00010000:3.0:1789760784.859313:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2fec4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.859330:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=764s v=(0 0) 00010000:00010000:3.0:1789760784.859335:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760784.859341:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2fec4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28401664->28405759] (req 28401664->28405759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.859355:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2fec4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28401664->28405759] (req 28401664->28405759) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760784.908784:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974253 is committed 00002000:00010000:2.0:1789760784.908814:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28401664 -> 28405760 00010000:00010000:2.0:1789760784.908831:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2fec4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28401664->28405759] (req 28401664->28405759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.908843:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2fec4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28401664->28405759] (req 28401664->28405759) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.908852:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2fec4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28401664->28405759] (req 28401664->28405759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.908859:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760784.908864:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2fec4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28401664->28405759] (req 28401664->28405759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.908872:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2fec4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28401664->28405759] (req 28401664->28405759) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.908892:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2fec4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28401664->28405759] (req 28401664->28405759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.908900:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c2fec4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28401664->28405759] (req 28401664->28405759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760784.909000:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac437e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512751232:12345-192.168.203.56@tcp:4:dd.0 Request processed in 49743us (50479us total) trans 4294974253 rc 0/0 00000100:00100000:2.0:1789760784.909830:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6964 00000100:00100000:2.0:1789760784.909840:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 764379049466 time 764423538622 state 0x2 ref 0 00000100:00100000:0.0:1789760784.916317:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760784.916460:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512751360 00000100:00100000:2.0:1789760784.916470:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512751360 00000100:00100000:2.0:1789760784.916526:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 764379049466 time 764423538622 state 0x0 ref 1 00000100:00100000:2.0:1789760784.916540:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 764423799038 time 764423538622 state 0x1 ref 1 00000100:00100000:2.0:1789760784.916581:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 764423799038 time 764480881137 state 0x0 ref 1 00000100:00100000:2.0:1789760784.916595:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4074700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512751360:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760784.916647:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28405760, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0xe007 00010000:00010000:2.0:1789760784.916655:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2fecb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.916671:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=764s v=(0 0) 00010000:00010000:2.0:1789760784.916676:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760784.916682:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2fecb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28405760->28409855] (req 28405760->28409855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.916741:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2fecb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28405760->28409855] (req 28405760->28409855) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760784.955992:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974254 is committed 00002000:00010000:2.0:1789760784.956166:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28405760 -> 28409856 00010000:00010000:2.0:1789760784.956182:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2fecb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28405760->28409855] (req 28405760->28409855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.956194:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2fecb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28405760->28409855] (req 28405760->28409855) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.956203:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2fecb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28405760->28409855] (req 28405760->28409855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.956211:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760784.956215:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2fecb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28405760->28409855] (req 28405760->28409855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.956225:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2fecb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28405760->28409855] (req 28405760->28409855) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.956244:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2fecb lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28405760->28409855] (req 28405760->28409855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.956252:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c2fecb lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28405760->28409855] (req 28405760->28409855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760784.956343:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4074700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512751360:12345-192.168.203.56@tcp:4:dd.0 Request processed in 39746us (40032us total) trans 4294974254 rc 0/0 00000100:00100000:2.0:1789760784.956353:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6965 00000100:00100000:2.0:1789760784.956361:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 764423799038 time 764480881137 state 0x2 ref 0 00000100:00100000:1.0:1789760784.959431:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760784.959521:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512751488 00000100:00100000:0.0:1789760784.959530:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512751488 00000100:00100000:0.0:1789760784.959583:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 764423799038 time 764480881137 state 0x0 ref 1 00000100:00100000:0.0:1789760784.959599:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 764481141553 time 764480881137 state 0x1 ref 1 00000100:00100000:0.0:1789760784.959632:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 764481141553 time 764523931575 state 0x0 ref 1 00000100:00100000:0.0:1789760784.959644:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4130000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512751488:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760784.959729:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28409856, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0xe007 00010000:00010000:0.0:1789760784.959738:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2fed2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.959754:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=764s v=(0 0) 00010000:00010000:0.0:1789760784.959758:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760784.959764:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2fed2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28409856->28413951] (req 28409856->28413951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.959783:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2fed2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28409856->28413951] (req 28409856->28413951) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760784.997257:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28409856 -> 28413952 00010000:00010000:0.0:1789760784.997277:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2fed2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28409856->28413951] (req 28409856->28413951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.997289:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2fed2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28409856->28413951] (req 28409856->28413951) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.997297:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2fed2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28409856->28413951] (req 28409856->28413951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.997304:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760784.997308:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2fed2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28409856->28413951] (req 28409856->28413951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.997316:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2fed2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28409856->28413951] (req 28409856->28413951) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.997334:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2fed2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28409856->28413951] (req 28409856->28413951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.997341:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2fed2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28409856->28413951] (req 28409856->28413951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760784.997397:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974255 is committed 00000100:00100000:0.0:1789760784.997472:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4130000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512751488:12345-192.168.203.56@tcp:4:dd.0 Request processed in 37824us (38044us total) trans 4294974255 rc 0/0 00000100:00100000:0.0:1789760784.997482:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6966 00000100:00100000:0.0:1789760784.997492:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 764481141553 time 764523931575 state 0x2 ref 0 00000100:00100000:2.0:1789760785.001338:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760785.001496:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512751616 00000100:00100000:3.0:1789760785.001507:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512751616 00000100:00100000:3.0:1789760785.001618:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 764481141553 time 764523931575 state 0x0 ref 1 00000100:00100000:3.0:1789760785.001633:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 764524191991 time 764523931575 state 0x1 ref 1 00000100:00100000:3.0:1789760785.001659:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 764524191991 time 764565957913 state 0x0 ref 1 00000100:00100000:3.0:1789760785.001675:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac437f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512751616:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760785.001758:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28413952, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0xe007 00010000:00010000:3.0:1789760785.001767:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2fed9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.001781:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=764s v=(0 0) 00010000:00010000:3.0:1789760785.001786:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760785.001791:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2fed9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28413952->28418047] (req 28413952->28418047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.001805:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2fed9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28413952->28418047] (req 28413952->28418047) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1789760785.004794:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00010000:3.0:1789760785.041812:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28413952 -> 28418048 00010000:00010000:3.0:1789760785.041832:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2fed9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28413952->28418047] (req 28413952->28418047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.041846:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2fed9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28413952->28418047] (req 28413952->28418047) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.041856:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2fed9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28413952->28418047] (req 28413952->28418047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.041862:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760785.041867:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2fed9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28413952->28418047] (req 28413952->28418047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.041877:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2fed9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28413952->28418047] (req 28413952->28418047) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.041897:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2fed9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28413952->28418047] (req 28413952->28418047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.041904:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2fed9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28413952->28418047] (req 28413952->28418047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760785.041994:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac437f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2360:x1876699512751616:12345-192.168.203.56@tcp:4:dd.0 Request processed in 40320us (40662us total) trans 4294974256 rc 0/0 00000100:00100000:3.0:1789760785.042004:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6967 00000100:00100000:3.0:1789760785.042024:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 764524191991 time 764565957913 state 0x2 ref 0 00000001:00080000:3.0:1789760785.042062:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974256 is committed 00000100:00100000:0.0:1789760785.045130:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760785.045210:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512751744 00000100:00100000:3.0:1789760785.045221:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512751744 00000100:00100000:3.0:1789760785.045277:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 764524191991 time 764565957913 state 0x0 ref 1 00000100:00100000:3.0:1789760785.045295:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 764566218329 time 764565957913 state 0x1 ref 1 00000100:00100000:3.0:1789760785.045326:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 764566218329 time 764609625399 state 0x0 ref 1 00000100:00100000:3.0:1789760785.045338:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febfc977100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512751744:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760785.045387:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28418048, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0xe007 00010000:00010000:3.0:1789760785.045396:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2fee0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.045429:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=764s v=(0 0) 00010000:00010000:3.0:1789760785.045433:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760785.045440:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2fee0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28418048->28422143] (req 28418048->28422143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.045453:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2fee0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28418048->28422143] (req 28418048->28422143) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760785.092326:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974257 is committed 00002000:00010000:0.0:1789760785.092518:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28418048 -> 28422144 00010000:00010000:0.0:1789760785.092535:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2fee0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28418048->28422143] (req 28418048->28422143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.092547:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2fee0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28418048->28422143] (req 28418048->28422143) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.092556:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2fee0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28418048->28422143] (req 28418048->28422143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.092877:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760785.092885:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2fee0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28418048->28422143] (req 28418048->28422143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.092898:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2fee0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28418048->28422143] (req 28418048->28422143) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.092918:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2fee0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28418048->28422143] (req 28418048->28422143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.092933:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c2fee0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28418048->28422143] (req 28418048->28422143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760785.093046:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febfc977100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512751744:12345-192.168.203.56@tcp:4:dd.0 Request processed in 47705us (47922us total) trans 4294974257 rc 0/0 00000100:00100000:0.0:1789760785.093056:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6968 00000100:00100000:0.0:1789760785.093065:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 764566218329 time 764609625399 state 0x2 ref 0 00000100:00100000:1.0:1789760785.097085:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760785.097186:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512751872 00000100:00100000:0.0:1789760785.097200:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512751872 00000100:00100000:0.0:1789760785.097276:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 764566218329 time 764609625399 state 0x0 ref 1 00000100:00100000:0.0:1789760785.097294:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 764609885815 time 764609625399 state 0x1 ref 1 00000100:00100000:0.0:1789760785.097365:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 764609885815 time 764661663052 state 0x0 ref 1 00000100:00100000:0.0:1789760785.097379:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4130700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512751872:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760785.097457:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28422144, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0xe007 00010000:00010000:0.0:1789760785.097467:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2fee7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.097488:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=764s v=(0 0) 00010000:00010000:0.0:1789760785.097494:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760785.097501:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2fee7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28422144->28426239] (req 28422144->28426239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.097530:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2fee7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28422144->28426239] (req 28422144->28426239) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760785.136885:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974258 is committed 00002000:00010000:2.0:1789760785.139248:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28422144 -> 28426240 00010000:00010000:2.0:1789760785.139268:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2fee7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28422144->28426239] (req 28422144->28426239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.139282:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2fee7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28422144->28426239] (req 28422144->28426239) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.139292:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2fee7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28422144->28426239] (req 28422144->28426239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.139299:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760785.139304:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2fee7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28422144->28426239] (req 28422144->28426239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.139313:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2fee7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28422144->28426239] (req 28422144->28426239) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.139331:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2fee7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28422144->28426239] (req 28422144->28426239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.139339:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c2fee7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28422144->28426239] (req 28422144->28426239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760785.139433:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4130700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512751872:12345-192.168.203.56@tcp:4:dd.0 Request processed in 42052us (42356us total) trans 4294974258 rc 0/0 00000100:00100000:2.0:1789760785.139447:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6969 00000100:00100000:2.0:1789760785.139457:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 764609885815 time 764661663052 state 0x2 ref 0 00000100:00100000:2.0:1789760785.145413:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760785.146242:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512752000 00000100:00100000:3.0:1789760785.146255:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512752000 00000100:00100000:3.0:1789760785.146311:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 764609885815 time 764661663052 state 0x0 ref 1 00000100:00100000:3.0:1789760785.146327:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 764661923468 time 764661663052 state 0x1 ref 1 00000100:00100000:3.0:1789760785.146416:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 764661923468 time 764710714371 state 0x0 ref 1 00000100:00100000:3.0:1789760785.146430:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac437d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512752000:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760785.146481:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28426240, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0xe007 00010000:00010000:3.0:1789760785.146490:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2feee lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.146505:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=764s v=(0 0) 00010000:00010000:3.0:1789760785.146510:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760785.146516:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2feee lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28426240->28430335] (req 28426240->28430335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.146530:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2feee lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28426240->28430335] (req 28426240->28430335) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760785.182577:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974259 is committed 00002000:00010000:0.0:1789760785.183108:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28426240 -> 28430336 00010000:00010000:0.0:1789760785.183124:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2feee lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28426240->28430335] (req 28426240->28430335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.183136:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2feee lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28426240->28430335] (req 28426240->28430335) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.183146:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2feee lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28426240->28430335] (req 28426240->28430335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.183153:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760785.183158:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2feee lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28426240->28430335] (req 28426240->28430335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.183166:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2feee lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28426240->28430335] (req 28426240->28430335) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.183184:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2feee lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28426240->28430335] (req 28426240->28430335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.183192:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c2feee lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28426240->28430335] (req 28426240->28430335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760785.183278:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac437d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512752000:12345-192.168.203.56@tcp:4:dd.0 Request processed in 36847us (37883us total) trans 4294974259 rc 0/0 00000100:00100000:0.0:1789760785.183288:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6970 00000100:00100000:0.0:1789760785.183297:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 764661923468 time 764710714371 state 0x2 ref 0 00000100:00100000:0.0:1789760785.199432:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760785.199556:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512752128 00000100:00100000:3.0:1789760785.199573:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512752128 00000100:00100000:3.0:1789760785.199660:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 764661923468 time 764710714371 state 0x0 ref 1 00000100:00100000:3.0:1789760785.199940:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 764710974787 time 764710714371 state 0x1 ref 1 00000100:00100000:2.0:1789760785.200103:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 764710974787 time 764764394968 state 0x0 ref 1 00000100:00100000:2.0:1789760785.200126:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac28a2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512752128:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760785.200196:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28430336, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0xe007 00010000:00010000:2.0:1789760785.200206:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2fef5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.200224:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=764s v=(0 0) 00010000:00010000:2.0:1789760785.200228:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760785.200235:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2fef5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28430336->28434431] (req 28430336->28434431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.200248:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2fef5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28430336->28434431] (req 28430336->28434431) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760785.246953:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974260 is committed 00002000:00010000:0.0:1789760785.247164:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28430336 -> 28434432 00010000:00010000:0.0:1789760785.247179:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2fef5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28430336->28434431] (req 28430336->28434431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.247191:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2fef5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28430336->28434431] (req 28430336->28434431) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.247200:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2fef5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28430336->28434431] (req 28430336->28434431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.247207:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760785.247212:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2fef5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28430336->28434431] (req 28430336->28434431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.247221:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2fef5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28430336->28434431] (req 28430336->28434431) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.247242:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2fef5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28430336->28434431] (req 28430336->28434431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.247254:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000078fcc5d1/0xe6aa143190c2fef5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28430336->28434431] (req 28430336->28434431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760785.247380:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac28a2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512752128:12345-192.168.203.56@tcp:4:dd.0 Request processed in 47257us (47954us total) trans 4294974260 rc 0/0 00000100:00100000:0.0:1789760785.247392:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6971 00000100:00100000:0.0:1789760785.247402:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 764710974787 time 764764394968 state 0x2 ref 0 00000100:00100000:1.0:1789760785.251334:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760785.251396:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512752256 00000100:00100000:2.0:1789760785.251406:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512752256 00000100:00100000:2.0:1789760785.251464:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 764710974787 time 764764394968 state 0x0 ref 1 00000100:00100000:2.0:1789760785.251479:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 764764655384 time 764764394968 state 0x1 ref 1 00000100:00100000:2.0:1789760785.251546:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 764764655384 time 764815844917 state 0x0 ref 1 00000100:00100000:2.0:1789760785.251558:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4131f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512752256:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760785.251605:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28434432, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0xe007 00010000:00010000:2.0:1789760785.251613:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2fefc lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.251628:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=764s v=(0 0) 00010000:00010000:2.0:1789760785.251632:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760785.251638:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2fefc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28434432->28438527] (req 28434432->28438527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.251652:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2fefc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28434432->28438527] (req 28434432->28438527) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760785.291877:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28434432 -> 28438528 00010000:00010000:2.0:1789760785.291899:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2fefc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28434432->28438527] (req 28434432->28438527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.291912:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2fefc lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28434432->28438527] (req 28434432->28438527) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.291922:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2fefc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28434432->28438527] (req 28434432->28438527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.291930:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760785.291936:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2fefc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28434432->28438527] (req 28434432->28438527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.291953:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2fefc lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28434432->28438527] (req 28434432->28438527) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.291973:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2fefc lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28434432->28438527] (req 28434432->28438527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.291981:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2fefc lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28434432->28438527] (req 28434432->28438527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760785.292084:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4131f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2358:x1876699512752256:12345-192.168.203.56@tcp:4:dd.0 Request processed in 40523us (40755us total) trans 4294974261 rc 0/0 00000100:00100000:2.0:1789760785.292095:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6972 00000100:00100000:2.0:1789760785.292105:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 764764655384 time 764815844917 state 0x2 ref 0 00000001:00080000:3.0:1789760785.293911:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974261 is committed 00000100:00100000:2.0:1789760785.296375:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760785.296464:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512752384 00000100:00100000:0.0:1789760785.296475:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512752384 00000100:00100000:0.0:1789760785.296526:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 764764655384 time 764815844917 state 0x0 ref 1 00000100:00100000:0.0:1789760785.296540:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 764816105333 time 764815844917 state 0x1 ref 1 00000100:00100000:0.0:1789760785.296572:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 764816105333 time 764860871994 state 0x0 ref 1 00000100:00100000:0.0:1789760785.296585:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac437c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512752384:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760785.296633:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28438528, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0xe007 00010000:00010000:0.0:1789760785.296641:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2ff03 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.296655:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=764s v=(0 0) 00010000:00010000:0.0:1789760785.296659:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760785.296665:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2ff03 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28438528->28442623] (req 28438528->28442623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.296724:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2ff03 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28438528->28442623] (req 28438528->28442623) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760785.335087:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974262 is committed 00002000:00010000:0.0:1789760785.335259:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28438528 -> 28442624 00010000:00010000:0.0:1789760785.335274:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2ff03 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28438528->28442623] (req 28438528->28442623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.335287:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2ff03 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28438528->28442623] (req 28438528->28442623) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.335296:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2ff03 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28438528->28442623] (req 28438528->28442623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.335304:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760785.335309:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2ff03 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28438528->28442623] (req 28438528->28442623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.335318:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2ff03 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28438528->28442623] (req 28438528->28442623) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.335337:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2ff03 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28438528->28442623] (req 28438528->28442623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.335345:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2ff03 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28438528->28442623] (req 28438528->28442623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760785.335444:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac437c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512752384:12345-192.168.203.56@tcp:4:dd.0 Request processed in 38857us (39075us total) trans 4294974262 rc 0/0 00000100:00100000:0.0:1789760785.335455:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6973 00000100:00100000:0.0:1789760785.335464:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 764816105333 time 764860871994 state 0x2 ref 0 00000100:00100000:0.0:1789760785.341622:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760785.341842:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512752512 00000100:00100000:2.0:1789760785.341852:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512752512 00000100:00100000:2.0:1789760785.341905:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 764816105333 time 764860871994 state 0x0 ref 1 00000100:00100000:2.0:1789760785.341922:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 764861132410 time 764860871994 state 0x1 ref 1 00000100:00100000:2.0:1789760785.342056:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 764861132410 time 764906354310 state 0x0 ref 1 00000100:00100000:2.0:1789760785.342071:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac28a0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512752512:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760785.342126:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28442624, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0xe007 00010000:00010000:2.0:1789760785.342135:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2ff0a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.342151:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=764s v=(0 0) 00010000:00010000:2.0:1789760785.342155:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760785.342160:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2ff0a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28442624->28446719] (req 28442624->28446719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.342173:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2ff0a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28442624->28446719] (req 28442624->28446719) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760785.403966:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28442624 -> 28446720 00010000:00010000:1.0:1789760785.403984:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2ff0a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28442624->28446719] (req 28442624->28446719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.403996:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2ff0a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28442624->28446719] (req 28442624->28446719) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.404004:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2ff0a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28442624->28446719] (req 28442624->28446719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.404024:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760785.404029:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2ff0a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28442624->28446719] (req 28442624->28446719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.404038:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2ff0a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28442624->28446719] (req 28442624->28446719) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.404132:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2ff0a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28442624->28446719] (req 28442624->28446719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.404142:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c2ff0a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28442624->28446719] (req 28442624->28446719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760785.404497:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac28a0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2359:x1876699512752512:12345-192.168.203.56@tcp:4:dd.0 Request processed in 62424us (62877us total) trans 4294974263 rc 0/0 00000100:00100000:1.0:1789760785.404507:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6974 00000100:00100000:1.0:1789760785.404942:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 764861132410 time 764906354310 state 0x2 ref 0 00000001:00080000:1.0:1789760785.405002:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974263 is committed 00000100:00100000:1.0:1789760785.408046:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760785.408144:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512752640 00000100:00100000:0.0:1789760785.408154:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512752640 00000100:00100000:0.0:1789760785.408208:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 764861132410 time 764906354310 state 0x0 ref 1 00000100:00100000:0.0:1789760785.408222:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 764906614726 time 764906354310 state 0x1 ref 1 00000100:00100000:0.0:1789760785.408251:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 764906614726 time 764972550567 state 0x0 ref 1 00000100:00100000:0.0:1789760785.408263:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4130380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512752640:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760785.408313:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28446720, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0xe007 00010000:00010000:0.0:1789760785.408321:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2ff11 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.408336:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=764s v=(0 0) 00010000:00010000:0.0:1789760785.408340:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760785.408347:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2ff11 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28446720->28450815] (req 28446720->28450815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.408360:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2ff11 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28446720->28450815] (req 28446720->28450815) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760785.442968:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974264 is committed 00002000:00010000:0.0:1789760785.443046:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28446720 -> 28450816 00010000:00010000:0.0:1789760785.443061:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2ff11 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28446720->28450815] (req 28446720->28450815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.443074:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2ff11 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28446720->28450815] (req 28446720->28450815) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.443084:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2ff11 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28446720->28450815] (req 28446720->28450815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.443091:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760785.443095:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2ff11 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28446720->28450815] (req 28446720->28450815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.443104:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2ff11 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28446720->28450815] (req 28446720->28450815) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.443127:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2ff11 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28446720->28450815] (req 28446720->28450815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.443134:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2ff11 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28446720->28450815] (req 28446720->28450815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760785.443215:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4130380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512752640:12345-192.168.203.56@tcp:4:dd.0 Request processed in 34949us (35172us total) trans 4294974264 rc 0/0 00000100:00100000:0.0:1789760785.443403:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6975 00000100:00100000:0.0:1789760785.443413:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 764906614726 time 764972550567 state 0x2 ref 0 00000100:00100000:3.0:1789760785.447616:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760785.447677:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512752768 00000100:00100000:0.0:1789760785.447686:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512752768 00000100:00100000:0.0:1789760785.447774:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 764906614726 time 764972550567 state 0x0 ref 1 00000100:00100000:0.0:1789760785.447788:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 764972810983 time 764972550567 state 0x1 ref 1 00000100:00100000:0.0:1789760785.447816:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 764972810983 time 765012116371 state 0x0 ref 1 00000100:00100000:0.0:1789760785.447826:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febe7905880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512752768:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760785.447868:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28450816, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0xe007 00010000:00010000:0.0:1789760785.447877:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2ff18 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.447892:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=765s v=(0 0) 00010000:00010000:0.0:1789760785.447896:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760785.447902:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2ff18 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28450816->28454911] (req 28450816->28454911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.447916:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2ff18 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28450816->28454911] (req 28450816->28454911) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760785.495690:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28450816 -> 28454912 00010000:00010000:3.0:1789760785.495735:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2ff18 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28450816->28454911] (req 28450816->28454911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.495748:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2ff18 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28450816->28454911] (req 28450816->28454911) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.495758:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2ff18 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28450816->28454911] (req 28450816->28454911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.495765:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760785.495770:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2ff18 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28450816->28454911] (req 28450816->28454911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.495779:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2ff18 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28450816->28454911] (req 28450816->28454911) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.495913:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2ff18 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28450816->28454911] (req 28450816->28454911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.495923:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c2ff18 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28450816->28454911] (req 28450816->28454911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760785.496033:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974265 is committed 00000100:00100000:3.0:1789760785.496290:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febe7905880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512752768:12345-192.168.203.56@tcp:4:dd.0 Request processed in 48459us (48679us total) trans 4294974265 rc 0/0 00000100:00100000:3.0:1789760785.496300:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6976 00000100:00100000:3.0:1789760785.496309:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 764972810983 time 765012116371 state 0x2 ref 0 00000100:00100000:0.0:1789760785.501372:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760785.501427:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512752896 00000100:00100000:1.0:1789760785.501436:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512752896 00000100:00100000:1.0:1789760785.501488:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 764972810983 time 765012116371 state 0x0 ref 1 00000100:00100000:1.0:1789760785.501503:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 765012376787 time 765012116371 state 0x1 ref 1 00000100:00100000:1.0:1789760785.501535:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 765012376787 time 765065836057 state 0x0 ref 1 00000100:00100000:1.0:1789760785.501547:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac28a3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512752896:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760785.501592:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28454912, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0xe007 00010000:00010000:1.0:1789760785.501601:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2ff1f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.501621:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=765s v=(0 0) 00010000:00010000:1.0:1789760785.501626:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760785.501631:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2ff1f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28454912->28459007] (req 28454912->28459007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.501645:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2ff1f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28454912->28459007] (req 28454912->28459007) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760785.540602:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974266 is committed 00002000:00010000:0.0:1789760785.540730:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28454912 -> 28459008 00002000:00010000:0.0:1789760785.540735:0:6710:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760784 -> 1789760785 00002000:00010000:0.0:1789760785.540737:0:6710:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760784 -> 1789760785 00010000:00010000:0.0:1789760785.540751:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2ff1f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28454912->28459007] (req 28454912->28459007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.540765:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2ff1f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28454912->28459007] (req 28454912->28459007) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.540776:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2ff1f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28454912->28459007] (req 28454912->28459007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.540784:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760785.540789:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2ff1f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28454912->28459007] (req 28454912->28459007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.540929:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2ff1f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28454912->28459007] (req 28454912->28459007) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.540950:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2ff1f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28454912->28459007] (req 28454912->28459007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.540959:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c2ff1f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28454912->28459007] (req 28454912->28459007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760785.541071:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac28a3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512752896:12345-192.168.203.56@tcp:4:dd.0 Request processed in 39521us (39712us total) trans 4294974266 rc 0/0 00000100:00100000:0.0:1789760785.541082:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6977 00000100:00100000:0.0:1789760785.541093:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 765012376787 time 765065836057 state 0x2 ref 0 00000100:00100000:1.0:1789760785.544562:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760785.544650:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512753024 00000100:00100000:0.0:1789760785.544659:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512753024 00000100:00100000:0.0:1789760785.544778:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 765012376787 time 765065836057 state 0x0 ref 1 00000100:00100000:0.0:1789760785.544860:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 765066096473 time 765065836057 state 0x1 ref 1 00000100:00100000:0.0:1789760785.544895:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 765066096473 time 765109194005 state 0x0 ref 1 00000100:00100000:0.0:1789760785.544908:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4133b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512753024:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760785.544957:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28459008, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0xe007 00010000:00010000:0.0:1789760785.544975:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2ff26 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.544992:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=765s v=(0 0) 00010000:00010000:0.0:1789760785.544996:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760785.545167:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2ff26 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28459008->28463103] (req 28459008->28463103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.545192:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2ff26 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28459008->28463103] (req 28459008->28463103) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760785.572811:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974267 is committed 00002000:00010000:0.0:1789760785.572909:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28459008 -> 28463104 00010000:00010000:0.0:1789760785.572924:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2ff26 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28459008->28463103] (req 28459008->28463103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.572941:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2ff26 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28459008->28463103] (req 28459008->28463103) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.572949:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2ff26 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28459008->28463103] (req 28459008->28463103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.572956:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760785.572961:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2ff26 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28459008->28463103] (req 28459008->28463103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.572970:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2ff26 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28459008->28463103] (req 28459008->28463103) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.572989:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2ff26 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28459008->28463103] (req 28459008->28463103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.572998:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2ff26 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28459008->28463103] (req 28459008->28463103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760785.573098:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4133b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512753024:12345-192.168.203.56@tcp:4:dd.0 Request processed in 28185us (28539us total) trans 4294974267 rc 0/0 00000100:00100000:0.0:1789760785.573108:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6978 00000100:00100000:0.0:1789760785.573118:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 765066096473 time 765109194005 state 0x2 ref 0 00000100:00100000:1.0:1789760785.578306:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760785.578512:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512753152 00000100:00100000:3.0:1789760785.578522:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512753152 00000100:00100000:2.0:1789760785.578719:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 765066096473 time 765109194005 state 0x0 ref 1 00000100:00100000:2.0:1789760785.578735:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 765109454421 time 765109194005 state 0x1 ref 1 00000100:00100000:2.0:1789760785.578802:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 765109454421 time 765143100126 state 0x0 ref 1 00000100:00100000:2.0:1789760785.578815:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4133800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512753152:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760785.578863:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28463104, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0xe007 00010000:00010000:2.0:1789760785.578872:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2ff2d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.578887:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=765s v=(0 0) 00010000:00010000:2.0:1789760785.578892:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760785.578899:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2ff2d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28463104->28467199] (req 28463104->28467199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.578913:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2ff2d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28463104->28467199] (req 28463104->28467199) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760785.602436:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974268 is committed 00002000:00010000:0.0:1789760785.602508:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28463104 -> 28467200 00010000:00010000:0.0:1789760785.602525:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2ff2d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28463104->28467199] (req 28463104->28467199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.602539:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2ff2d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28463104->28467199] (req 28463104->28467199) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.602548:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2ff2d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28463104->28467199] (req 28463104->28467199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.602555:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760785.602560:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2ff2d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28463104->28467199] (req 28463104->28467199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.602569:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2ff2d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28463104->28467199] (req 28463104->28467199) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.602589:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2ff2d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28463104->28467199] (req 28463104->28467199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.602596:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c2ff2d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28463104->28467199] (req 28463104->28467199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760785.602725:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4133800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512753152:12345-192.168.203.56@tcp:4:dd.0 Request processed in 23880us (24396us total) trans 4294974268 rc 0/0 00000100:00100000:0.0:1789760785.602735:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6979 00000100:00100000:0.0:1789760785.602745:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 765109454421 time 765143100126 state 0x2 ref 0 00000100:00100000:2.0:1789760785.606410:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760785.606610:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512753280 00000100:00100000:0.0:1789760785.606621:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512753280 00000100:00100000:0.0:1789760785.606679:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 765109454421 time 765143100126 state 0x0 ref 1 00000100:00100000:0.0:1789760785.606791:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 765143360542 time 765143100126 state 0x1 ref 1 00000100:00100000:3.0:1789760785.606950:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 765143360542 time 765171246895 state 0x0 ref 1 00000100:00100000:3.0:1789760785.606966:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febea488050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512753280:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760785.607023:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28467200, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0xe007 00010000:00010000:3.0:1789760785.607032:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2ff34 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.607047:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=765s v=(0 0) 00010000:00010000:3.0:1789760785.607051:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760785.607058:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2ff34 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28467200->28471295] (req 28467200->28471295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.607077:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2ff34 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28467200->28471295] (req 28467200->28471295) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760785.639110:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974269 is committed 00002000:00010000:3.0:1789760785.639208:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28467200 -> 28471296 00010000:00010000:3.0:1789760785.639225:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2ff34 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28467200->28471295] (req 28467200->28471295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.639239:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2ff34 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28467200->28471295] (req 28467200->28471295) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.639249:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2ff34 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28467200->28471295] (req 28467200->28471295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.639256:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760785.639262:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2ff34 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28467200->28471295] (req 28467200->28471295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.639271:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2ff34 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28467200->28471295] (req 28467200->28471295) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.639290:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2ff34 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28467200->28471295] (req 28467200->28471295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.639298:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2ff34 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28467200->28471295] (req 28467200->28471295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760785.639396:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febea488050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512753280:12345-192.168.203.56@tcp:4:dd.0 Request processed in 32429us (32990us total) trans 4294974269 rc 0/0 00000100:00100000:3.0:1789760785.639406:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6980 00000100:00100000:3.0:1789760785.639415:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 765143360542 time 765171246895 state 0x2 ref 0 00000100:00100000:1.0:1789760785.645187:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760785.645475:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512753408 00000100:00100000:0.0:1789760785.645485:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512753408 00000100:00100000:0.0:1789760785.645537:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 765143360542 time 765171246895 state 0x0 ref 1 00000100:00100000:0.0:1789760785.645552:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 765171507311 time 765171246895 state 0x1 ref 1 00000100:00100000:0.0:1789760785.645620:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 765171507311 time 765209918269 state 0x0 ref 1 00000100:00100000:0.0:1789760785.645633:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4132680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512753408:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760785.645686:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28471296, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0xe007 00010000:00010000:0.0:1789760785.645737:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2ff3b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.645755:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=765s v=(0 0) 00010000:00010000:0.0:1789760785.645759:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760785.645766:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2ff3b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28471296->28475391] (req 28471296->28475391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.645778:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2ff3b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28471296->28475391] (req 28471296->28475391) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760785.672002:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760785.672062:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760785.672105:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760785.672139:0:8229:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512753664 00000100:00100000:2.0:1789760785.672148:0:8229:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512753664 00000100:00100000:0.0:1789760785.672249:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512753536 00000100:00100000:0.0:1789760785.672254:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512753536 00000100:00100000:0.0:1789760785.672301:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 414 00000100:00100000:0.0:1789760785.672310:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febe00c7050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:e6cf7267-9b19-462a-a263-14115df15dab+13:2360:x1876699512753536:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:0.0:1789760785.672374:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febe00c7050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:e6cf7267-9b19-462a-a263-14115df15dab+13:2360:x1876699512753536:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 65us (310us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760785.672382:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 414 00000100:00100000:3.0:1789760785.672550:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512753792 00000100:00100000:3.0:1789760785.672555:0:5847:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512753792 00000100:00100000:2.0:1789760785.672624:0:8229:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 514 00000100:00100000:2.0:1789760785.672633:0:8229:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac437fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+8:2360:x1876699512753664:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:2.0:1789760785.672672:0:8229:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac437fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+8:2360:x1876699512753664:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 41us (680us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760785.672679:0:8229:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 514 00000001:00080000:0.0:1789760785.673115:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974270 is committed 00002000:00010000:0.0:1789760785.673177:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28471296 -> 28475392 00010000:00010000:0.0:1789760785.673190:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2ff3b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28471296->28475391] (req 28471296->28475391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.673201:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2ff3b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28471296->28475391] (req 28471296->28475391) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.673210:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2ff3b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28471296->28475391] (req 28471296->28475391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.673218:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760785.673223:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2ff3b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28471296->28475391] (req 28471296->28475391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.673233:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2ff3b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28471296->28475391] (req 28471296->28475391) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.673258:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2ff3b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28471296->28475391] (req 28471296->28475391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.673266:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2ff3b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28471296->28475391] (req 28471296->28475391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760785.673309:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4132680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512753408:12345-192.168.203.56@tcp:4:dd.0 Request processed in 27678us (28132us total) trans 4294974270 rc 0/0 00000100:00100000:0.0:1789760785.673318:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6981 00000100:00100000:0.0:1789760785.673327:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 765171507311 time 765209918269 state 0x2 ref 0 00000100:00100000:3.0:1789760785.673646:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 515 00000100:00100000:3.0:1789760785.673656:0:5847:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac437d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:d2c40150-e97b-40da-952b-cb9f18bf7e96+5:2360:x1876699512753792:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:3.0:1789760785.673689:0:5847:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac437d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:d2c40150-e97b-40da-952b-cb9f18bf7e96+5:2360:x1876699512753792:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 35us (1583us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760785.673727:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 515 00000100:00080000:2.0:1789760785.673985:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789760785.677339:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760785.678869:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512753920 00000100:00100000:0.0:1789760785.678883:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512753920 00000100:00100000:0.0:1789760785.678964:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 765171507311 time 765209918269 state 0x0 ref 1 00000100:00100000:0.0:1789760785.678979:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 765210178685 time 765209918269 state 0x1 ref 1 00000100:00100000:0.0:1789760785.679028:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 765210178685 time 765243327681 state 0x0 ref 1 00000100:00100000:0.0:1789760785.679041:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac437df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512753920:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760785.679091:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28475392, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0xe007 00010000:00010000:0.0:1789760785.679100:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2ff42 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.679116:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=765s v=(0 0) 00010000:00010000:0.0:1789760785.679121:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760785.679128:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2ff42 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28475392->28479487] (req 28475392->28479487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.679140:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2ff42 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28475392->28479487] (req 28475392->28479487) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760785.729895:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28475392 -> 28479488 00010000:00010000:0.0:1789760785.729919:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2ff42 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28475392->28479487] (req 28475392->28479487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.729933:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2ff42 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28475392->28479487] (req 28475392->28479487) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.729943:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2ff42 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28475392->28479487] (req 28475392->28479487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.729951:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760785.729956:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2ff42 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28475392->28479487] (req 28475392->28479487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.729966:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2ff42 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28475392->28479487] (req 28475392->28479487) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.729985:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2ff42 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28475392->28479487] (req 28475392->28479487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.729998:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c2ff42 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28475392->28479487] (req 28475392->28479487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760785.730098:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac437df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2361:x1876699512753920:12345-192.168.203.56@tcp:4:dd.0 Request processed in 51055us (52764us total) trans 4294974271 rc 0/0 00000100:00100000:0.0:1789760785.730123:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6982 00000100:00100000:0.0:1789760785.730132:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 765210178685 time 765243327681 state 0x2 ref 0 00000001:00080000:3.0:1789760785.730609:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974271 is committed 00000100:00100000:2.0:1789760785.737930:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760785.738092:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512754048 00000100:00100000:3.0:1789760785.738112:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512754048 00000100:00100000:3.0:1789760785.738172:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 765210178685 time 765243327681 state 0x0 ref 1 00000100:00100000:3.0:1789760785.738187:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 765243588097 time 765243327681 state 0x1 ref 1 00000100:00100000:3.0:1789760785.738271:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 765243588097 time 765302569425 state 0x0 ref 1 00000100:00100000:3.0:1789760785.738293:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac437e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512754048:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760785.738341:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28479488, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0xe007 00010000:00010000:3.0:1789760785.738351:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2ff49 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.738367:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=765s v=(0 0) 00010000:00010000:3.0:1789760785.738371:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760785.738377:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2ff49 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28479488->28483583] (req 28479488->28483583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.738390:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2ff49 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28479488->28483583] (req 28479488->28483583) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.779370:0:5835:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:0.0:1789760785.790542:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974272 is committed 00002000:00010000:3.0:1789760785.790750:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28479488 -> 28483584 00010000:00010000:3.0:1789760785.790764:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2ff49 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28479488->28483583] (req 28479488->28483583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.790776:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2ff49 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28479488->28483583] (req 28479488->28483583) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.790784:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2ff49 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28479488->28483583] (req 28479488->28483583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.790791:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760785.790796:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2ff49 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28479488->28483583] (req 28479488->28483583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.790805:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2ff49 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28479488->28483583] (req 28479488->28483583) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.790821:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2ff49 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28479488->28483583] (req 28479488->28483583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.790828:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2ff49 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28479488->28483583] (req 28479488->28483583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760785.790908:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac437e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512754048:12345-192.168.203.56@tcp:4:dd.0 Request processed in 52622us (52984us total) trans 4294974272 rc 0/0 00000100:00100000:3.0:1789760785.790917:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6983 00000100:00100000:3.0:1789760785.790925:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 765243588097 time 765302569425 state 0x2 ref 0 00000100:00100000:2.0:1789760785.797897:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760785.798123:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512754176 00000100:00100000:0.0:1789760785.798133:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512754176 00000100:00100000:0.0:1789760785.798184:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 765243588097 time 765302569425 state 0x0 ref 1 00000100:00100000:0.0:1789760785.798199:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 765302829841 time 765302569425 state 0x1 ref 1 00000100:00100000:0.0:1789760785.798230:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 765302829841 time 765362529807 state 0x0 ref 1 00000100:00100000:0.0:1789760785.798241:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac437f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512754176:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760785.798286:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28483584, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0xe007 00010000:00010000:0.0:1789760785.798295:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2ff50 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.798311:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=765s v=(0 0) 00010000:00010000:0.0:1789760785.798315:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760785.798321:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2ff50 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28483584->28487679] (req 28483584->28487679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.798334:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2ff50 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28483584->28487679] (req 28483584->28487679) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760785.847263:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974273 is committed 00002000:00010000:1.0:1789760785.847358:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28483584 -> 28487680 00010000:00010000:1.0:1789760785.847375:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2ff50 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28483584->28487679] (req 28483584->28487679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.847388:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2ff50 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28483584->28487679] (req 28483584->28487679) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.847396:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2ff50 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28483584->28487679] (req 28483584->28487679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.847403:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760785.847407:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2ff50 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28483584->28487679] (req 28483584->28487679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.847420:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2ff50 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28483584->28487679] (req 28483584->28487679) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.847439:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2ff50 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28483584->28487679] (req 28483584->28487679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.847446:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c2ff50 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28483584->28487679] (req 28483584->28487679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760785.847538:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac437f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512754176:12345-192.168.203.56@tcp:4:dd.0 Request processed in 49293us (49645us total) trans 4294974273 rc 0/0 00000100:00100000:1.0:1789760785.847547:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6984 00000100:00100000:1.0:1789760785.847557:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 765302829841 time 765362529807 state 0x2 ref 0 00000100:00100000:0.0:1789760785.850137:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760785.850313:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512754304 00000100:00100000:2.0:1789760785.850325:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512754304 00000100:00100000:2.0:1789760785.850383:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 765302829841 time 765362529807 state 0x0 ref 1 00000100:00100000:2.0:1789760785.850399:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 765362790223 time 765362529807 state 0x1 ref 1 00000100:00100000:2.0:1789760785.850433:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 765362790223 time 765414732607 state 0x0 ref 1 00000100:00100000:2.0:1789760785.850445:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac28a2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512754304:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760785.850493:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28487680, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0xe007 00010000:00010000:2.0:1789760785.850503:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2ff57 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.850518:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=765s v=(0 0) 00010000:00010000:2.0:1789760785.850523:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760785.850529:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2ff57 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28487680->28491775] (req 28487680->28491775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.850541:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2ff57 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28487680->28491775] (req 28487680->28491775) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760785.892382:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974274 is committed 00002000:00010000:0.0:1789760785.892479:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28487680 -> 28491776 00010000:00010000:0.0:1789760785.892495:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2ff57 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28487680->28491775] (req 28487680->28491775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.892507:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2ff57 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28487680->28491775] (req 28487680->28491775) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.892516:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2ff57 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28487680->28491775] (req 28487680->28491775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.892523:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760785.892528:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2ff57 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28487680->28491775] (req 28487680->28491775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.892536:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2ff57 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28487680->28491775] (req 28487680->28491775) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.892555:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2ff57 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28487680->28491775] (req 28487680->28491775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.892563:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c2ff57 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28487680->28491775] (req 28487680->28491775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760785.892895:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac28a2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512754304:12345-192.168.203.56@tcp:4:dd.0 Request processed in 42447us (42762us total) trans 4294974274 rc 0/0 00000100:00100000:0.0:1789760785.892906:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6985 00000100:00100000:0.0:1789760785.892916:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 765362790223 time 765414732607 state 0x2 ref 0 00000100:00100000:3.0:1789760785.897893:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760785.898129:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512754432 00000100:00100000:1.0:1789760785.898139:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512754432 00000100:00100000:1.0:1789760785.898203:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 765362790223 time 765414732607 state 0x0 ref 1 00000100:00100000:1.0:1789760785.898221:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 765414993023 time 765414732607 state 0x1 ref 1 00000100:00100000:1.0:1789760785.898254:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 765414993023 time 765462553898 state 0x0 ref 1 00000100:00100000:1.0:1789760785.898279:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febe7905f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512754432:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760785.898327:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28491776, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0xe007 00010000:00010000:1.0:1789760785.898336:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2ff5e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.898354:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=765s v=(0 0) 00010000:00010000:1.0:1789760785.898359:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760785.898366:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2ff5e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28491776->28495871] (req 28491776->28495871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.898379:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2ff5e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28491776->28495871] (req 28491776->28495871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760785.936510:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974275 is committed 00002000:00010000:1.0:1789760785.936964:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28491776 -> 28495872 00010000:00010000:1.0:1789760785.936980:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2ff5e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28491776->28495871] (req 28491776->28495871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.936993:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2ff5e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28491776->28495871] (req 28491776->28495871) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.937011:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2ff5e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28491776->28495871] (req 28491776->28495871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.937019:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760785.937023:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2ff5e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28491776->28495871] (req 28491776->28495871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.937033:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2ff5e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28491776->28495871] (req 28491776->28495871) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.937052:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2ff5e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28491776->28495871] (req 28491776->28495871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.937060:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2ff5e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28491776->28495871] (req 28491776->28495871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760785.937146:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febe7905f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512754432:12345-192.168.203.56@tcp:4:dd.0 Request processed in 38864us (39258us total) trans 4294974275 rc 0/0 00000100:00100000:1.0:1789760785.937155:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6986 00000100:00100000:1.0:1789760785.937165:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 765414993023 time 765462553898 state 0x2 ref 0 00000100:00100000:2.0:1789760785.939590:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760785.939675:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512754560 00000100:00100000:1.0:1789760785.939686:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512754560 00000100:00100000:1.0:1789760785.939778:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 765414993023 time 765462553898 state 0x0 ref 1 00000100:00100000:1.0:1789760785.939793:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 765462814314 time 765462553898 state 0x1 ref 1 00000100:00100000:1.0:1789760785.939815:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 765462814314 time 765504115564 state 0x0 ref 1 00000100:00100000:1.0:1789760785.939828:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac437d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512754560:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760785.939880:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28495872, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0xe007 00010000:00010000:1.0:1789760785.939890:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2ff65 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.939910:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=765s v=(0 0) 00010000:00010000:1.0:1789760785.939915:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760785.939922:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2ff65 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28495872->28499967] (req 28495872->28499967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.939942:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2ff65 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28495872->28499967] (req 28495872->28499967) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760785.986012:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28495872 -> 28499968 00010000:00010000:1.0:1789760785.986033:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2ff65 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28495872->28499967] (req 28495872->28499967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.986045:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2ff65 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28495872->28499967] (req 28495872->28499967) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.986054:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2ff65 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28495872->28499967] (req 28495872->28499967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.986062:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760785.986066:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2ff65 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28495872->28499967] (req 28495872->28499967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.986075:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2ff65 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28495872->28499967] (req 28495872->28499967) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.986094:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2ff65 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28495872->28499967] (req 28495872->28499967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.986103:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2ff65 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28495872->28499967] (req 28495872->28499967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760785.986208:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac437d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2361:x1876699512754560:12345-192.168.203.56@tcp:4:dd.0 Request processed in 46369us (46616us total) trans 4294974276 rc 0/0 00000100:00100000:1.0:1789760785.986219:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6987 00000100:00100000:1.0:1789760785.986229:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 765462814314 time 765504115564 state 0x2 ref 0 00000001:00080000:2.0:1789760785.986371:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974276 is committed 00000100:00100000:0.0:1789760785.988989:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760785.989094:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512754688 00000100:00100000:2.0:1789760785.989104:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512754688 00000100:00100000:2.0:1789760785.989158:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 765462814314 time 765504115564 state 0x0 ref 1 00000100:00100000:2.0:1789760785.989174:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 765504375980 time 765504115564 state 0x1 ref 1 00000100:00100000:2.0:1789760785.989298:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 765504375980 time 765553596905 state 0x0 ref 1 00000100:00100000:2.0:1789760785.989316:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac28a0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512754688:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760785.989366:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28499968, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0xe007 00010000:00010000:2.0:1789760785.989376:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2ff6c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.989392:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=765s v=(0 0) 00010000:00010000:2.0:1789760785.989396:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760785.989402:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2ff6c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28499968->28504063] (req 28499968->28504063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.989414:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2ff6c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28499968->28504063] (req 28499968->28504063) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760786.038566:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974277 is committed 00002000:00010000:2.0:1789760786.038646:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28499968 -> 28504064 00010000:00010000:2.0:1789760786.038660:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2ff6c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28499968->28504063] (req 28499968->28504063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.038753:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2ff6c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28499968->28504063] (req 28499968->28504063) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.038765:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2ff6c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28499968->28504063] (req 28499968->28504063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.038774:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760786.038778:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2ff6c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28499968->28504063] (req 28499968->28504063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.038789:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2ff6c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28499968->28504063] (req 28499968->28504063) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.038809:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2ff6c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28499968->28504063] (req 28499968->28504063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.038818:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2ff6c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28499968->28504063] (req 28499968->28504063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760786.038945:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac28a0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512754688:12345-192.168.203.56@tcp:4:dd.0 Request processed in 49627us (49959us total) trans 4294974277 rc 0/0 00000100:00100000:2.0:1789760786.038956:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6988 00000100:00100000:2.0:1789760786.038965:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 765504375980 time 765553596905 state 0x2 ref 0 00000100:00100000:3.0:1789760786.042074:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760786.042149:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512754816 00000100:00100000:1.0:1789760786.042159:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512754816 00000100:00100000:1.0:1789760786.042214:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 765504375980 time 765553596905 state 0x0 ref 1 00000100:00100000:1.0:1789760786.042229:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 765553857321 time 765553596905 state 0x1 ref 1 00000100:00100000:1.0:1789760786.042260:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 765553857321 time 765606559807 state 0x0 ref 1 00000100:00100000:1.0:1789760786.042273:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febe7905880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512754816:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760786.042328:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28504064, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0xe007 00010000:00010000:1.0:1789760786.042337:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2ff73 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.042352:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=765s v=(0 0) 00010000:00010000:1.0:1789760786.042357:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760786.042363:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2ff73 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28504064->28508159] (req 28504064->28508159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.042377:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2ff73 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28504064->28508159] (req 28504064->28508159) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760786.072540:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974278 is committed 00002000:00010000:1.0:1789760786.072640:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28504064 -> 28508160 00010000:00010000:1.0:1789760786.072656:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2ff73 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28504064->28508159] (req 28504064->28508159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.072668:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2ff73 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28504064->28508159] (req 28504064->28508159) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.072688:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2ff73 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28504064->28508159] (req 28504064->28508159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.072694:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760786.072726:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2ff73 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28504064->28508159] (req 28504064->28508159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.072736:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2ff73 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28504064->28508159] (req 28504064->28508159) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.072756:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2ff73 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28504064->28508159] (req 28504064->28508159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.072764:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c2ff73 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28504064->28508159] (req 28504064->28508159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760786.072840:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febe7905880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512754816:12345-192.168.203.56@tcp:4:dd.0 Request processed in 30564us (30770us total) trans 4294974278 rc 0/0 00000100:00100000:1.0:1789760786.072853:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6989 00000100:00100000:1.0:1789760786.072863:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 765553857321 time 765606559807 state 0x2 ref 0 00000100:00100000:2.0:1789760786.076039:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760786.076115:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512754944 00000100:00100000:1.0:1789760786.076125:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512754944 00000100:00100000:1.0:1789760786.076177:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 765553857321 time 765606559807 state 0x0 ref 1 00000100:00100000:1.0:1789760786.076191:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 765606820223 time 765606559807 state 0x1 ref 1 00000100:00100000:1.0:1789760786.076224:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 765606820223 time 765640523755 state 0x0 ref 1 00000100:00100000:1.0:1789760786.076235:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac437c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512754944:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760786.076337:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28508160, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0xe007 00010000:00010000:1.0:1789760786.076346:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2ff7a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.076362:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=765s v=(0 0) 00010000:00010000:1.0:1789760786.076367:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760786.076374:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2ff7a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28508160->28512255] (req 28508160->28512255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.076387:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2ff7a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28508160->28512255] (req 28508160->28512255) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760786.120255:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28508160 -> 28512256 00010000:00010000:1.0:1789760786.120274:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2ff7a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28508160->28512255] (req 28508160->28512255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.120287:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2ff7a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28508160->28512255] (req 28508160->28512255) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.120296:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2ff7a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28508160->28512255] (req 28508160->28512255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.120303:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760786.120308:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2ff7a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28508160->28512255] (req 28508160->28512255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.120316:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2ff7a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28508160->28512255] (req 28508160->28512255) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.120336:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2ff7a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28508160->28512255] (req 28508160->28512255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.120344:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2ff7a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28508160->28512255] (req 28508160->28512255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760786.120428:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974279 is committed 00000100:00100000:1.0:1789760786.120449:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac437c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512754944:12345-192.168.203.56@tcp:4:dd.0 Request processed in 44211us (44414us total) trans 4294974279 rc 0/0 00000100:00100000:1.0:1789760786.120459:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6990 00000100:00100000:1.0:1789760786.120469:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 765606820223 time 765640523755 state 0x2 ref 0 00000100:00100000:0.0:1789760786.126101:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760786.126190:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512755072 00000100:00100000:2.0:1789760786.126622:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512755072 00000100:00100000:2.0:1789760786.126672:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 765606820223 time 765640523755 state 0x0 ref 1 00000100:00100000:2.0:1789760786.126688:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 765640784171 time 765640523755 state 0x1 ref 1 00000100:00100000:2.0:1789760786.126750:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 765640784171 time 765691048757 state 0x0 ref 1 00000100:00100000:2.0:1789760786.126763:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac28a3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512755072:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760786.126809:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28512256, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0xe007 00010000:00010000:2.0:1789760786.126819:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2ff81 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.126833:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=765s v=(0 0) 00010000:00010000:2.0:1789760786.126838:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760786.126844:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2ff81 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28512256->28516351] (req 28512256->28516351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.126856:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2ff81 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28512256->28516351] (req 28512256->28516351) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760786.165073:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974280 is committed 00002000:00010000:1.0:1789760786.165207:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28512256 -> 28516352 00010000:00010000:1.0:1789760786.165223:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2ff81 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28512256->28516351] (req 28512256->28516351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.165236:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2ff81 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28512256->28516351] (req 28512256->28516351) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.165245:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2ff81 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28512256->28516351] (req 28512256->28516351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.165253:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760786.165258:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2ff81 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28512256->28516351] (req 28512256->28516351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.165269:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2ff81 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28512256->28516351] (req 28512256->28516351) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.165287:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2ff81 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28512256->28516351] (req 28512256->28516351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.165294:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c2ff81 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28512256->28516351] (req 28512256->28516351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760786.165541:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac28a3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512755072:12345-192.168.203.56@tcp:4:dd.0 Request processed in 38774us (39442us total) trans 4294974280 rc 0/0 00000100:00100000:1.0:1789760786.165551:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6991 00000100:00100000:1.0:1789760786.165560:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 765640784171 time 765691048757 state 0x2 ref 0 00000100:00100000:3.0:1789760786.171856:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760786.171964:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512755200 00000100:00100000:1.0:1789760786.171975:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512755200 00000100:00100000:1.0:1789760786.172035:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 765640784171 time 765691048757 state 0x0 ref 1 00000100:00100000:1.0:1789760786.172061:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 765691309173 time 765691048757 state 0x1 ref 1 00000100:00100000:1.0:1789760786.172081:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 765691309173 time 765736381143 state 0x0 ref 1 00000100:00100000:1.0:1789760786.172094:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febe7907100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512755200:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760786.172140:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28516352, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0xe007 00010000:00010000:1.0:1789760786.172150:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2ff88 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.172166:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=765s v=(0 0) 00010000:00010000:1.0:1789760786.172171:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760786.172178:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2ff88 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28516352->28520447] (req 28516352->28520447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.172192:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2ff88 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28516352->28520447] (req 28516352->28520447) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760786.215009:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974281 is committed 00002000:00010000:0.0:1789760786.215335:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28516352 -> 28520448 00010000:00010000:0.0:1789760786.215350:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2ff88 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28516352->28520447] (req 28516352->28520447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.215361:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2ff88 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28516352->28520447] (req 28516352->28520447) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.215369:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2ff88 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28516352->28520447] (req 28516352->28520447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.215376:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760786.215382:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2ff88 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28516352->28520447] (req 28516352->28520447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.215390:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2ff88 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28516352->28520447] (req 28516352->28520447) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.215485:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2ff88 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28516352->28520447] (req 28516352->28520447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.215492:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c2ff88 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28516352->28520447] (req 28516352->28520447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760786.215584:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febe7907100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512755200:12345-192.168.203.56@tcp:4:dd.0 Request processed in 43488us (43735us total) trans 4294974281 rc 0/0 00000100:00100000:0.0:1789760786.215594:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6992 00000100:00100000:0.0:1789760786.215603:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 765691309173 time 765736381143 state 0x2 ref 0 00000100:00100000:3.0:1789760786.218860:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760786.219199:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512755328 00000100:00100000:1.0:1789760786.219209:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512755328 00000100:00100000:1.0:1789760786.219276:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 765691309173 time 765736381143 state 0x0 ref 1 00000100:00100000:1.0:1789760786.219292:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 765736641559 time 765736381143 state 0x1 ref 1 00000100:00100000:1.0:1789760786.219325:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 765736641559 time 765783625112 state 0x0 ref 1 00000100:00100000:1.0:1789760786.219338:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febe7906300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512755328:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760786.219385:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28520448, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0xe007 00010000:00010000:1.0:1789760786.219402:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2ff8f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.219417:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=765s v=(0 0) 00010000:00010000:1.0:1789760786.219422:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760786.219428:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2ff8f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28520448->28524543] (req 28520448->28524543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.219442:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2ff8f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28520448->28524543] (req 28520448->28524543) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760786.309760:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974282 is committed 00002000:00010000:0.0:1789760786.309873:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28520448 -> 28524544 00010000:00010000:0.0:1789760786.309889:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2ff8f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28520448->28524543] (req 28520448->28524543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.309901:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2ff8f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28520448->28524543] (req 28520448->28524543) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.309909:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2ff8f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28520448->28524543] (req 28520448->28524543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.309914:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760786.309918:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2ff8f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28520448->28524543] (req 28520448->28524543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.309930:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2ff8f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28520448->28524543] (req 28520448->28524543) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.309948:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2ff8f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28520448->28524543] (req 28520448->28524543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.309955:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c2ff8f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28520448->28524543] (req 28520448->28524543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760786.310052:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febe7906300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512755328:12345-192.168.203.56@tcp:4:dd.0 Request processed in 90711us (91196us total) trans 4294974282 rc 0/0 00000100:00100000:0.0:1789760786.310062:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6993 00000100:00100000:0.0:1789760786.310071:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 765736641559 time 765783625112 state 0x2 ref 0 00000100:00100000:1.0:1789760786.316135:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760786.316267:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512755456 00000100:00100000:2.0:1789760786.316278:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512755456 00000100:00100000:2.0:1789760786.316328:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 765736641559 time 765783625112 state 0x0 ref 1 00000100:00100000:2.0:1789760786.316342:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 765783885528 time 765783625112 state 0x1 ref 1 00000100:00100000:2.0:1789760786.316369:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 765783885528 time 765880668638 state 0x0 ref 1 00000100:00100000:2.0:1789760786.316381:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4130000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512755456:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760786.316428:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28524544, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0xe007 00010000:00010000:2.0:1789760786.316437:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2ff96 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.316452:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=765s v=(0 0) 00010000:00010000:2.0:1789760786.317224:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760786.317245:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2ff96 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28524544->28528639] (req 28524544->28528639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.317263:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2ff96 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28524544->28528639] (req 28524544->28528639) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760786.400727:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974283 is committed 00002000:00010000:1.0:1789760786.400951:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28524544 -> 28528640 00010000:00010000:1.0:1789760786.400966:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2ff96 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28524544->28528639] (req 28524544->28528639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.400978:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2ff96 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28524544->28528639] (req 28524544->28528639) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.400985:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2ff96 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28524544->28528639] (req 28524544->28528639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.400997:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760786.401012:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2ff96 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28524544->28528639] (req 28524544->28528639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.401020:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2ff96 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28524544->28528639] (req 28524544->28528639) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.401038:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2ff96 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28524544->28528639] (req 28524544->28528639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.401044:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c2ff96 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28524544->28528639] (req 28524544->28528639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760786.401442:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4130000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512755456:12345-192.168.203.56@tcp:4:dd.0 Request processed in 85056us (85310us total) trans 4294974283 rc 0/0 00000100:00100000:1.0:1789760786.401454:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6994 00000100:00100000:1.0:1789760786.401463:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 765783885528 time 765880668638 state 0x2 ref 0 00000100:00100000:1.0:1789760786.405398:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760786.405477:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512755584 00000100:00100000:0.0:1789760786.405486:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512755584 00000100:00100000:0.0:1789760786.405543:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 765783885528 time 765880668638 state 0x0 ref 1 00000100:00100000:0.0:1789760786.405558:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 765880929054 time 765880668638 state 0x1 ref 1 00000100:00100000:0.0:1789760786.405587:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 765880929054 time 765969886860 state 0x0 ref 1 00000100:00100000:0.0:1789760786.405598:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4130700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512755584:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760786.405644:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28528640, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0xe007 00010000:00010000:0.0:1789760786.405654:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2ff9d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.405669:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=765s v=(0 0) 00010000:00010000:0.0:1789760786.405673:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760786.405679:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2ff9d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28528640->28532735] (req 28528640->28532735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.405693:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2ff9d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28528640->28532735] (req 28528640->28532735) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760786.470353:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974284 is committed 00002000:00010000:0.0:1789760786.470564:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28528640 -> 28532736 00010000:00010000:0.0:1789760786.470580:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2ff9d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28528640->28532735] (req 28528640->28532735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.470598:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2ff9d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28528640->28532735] (req 28528640->28532735) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.470608:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2ff9d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28528640->28532735] (req 28528640->28532735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.470616:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760786.470621:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2ff9d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28528640->28532735] (req 28528640->28532735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.470631:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2ff9d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28528640->28532735] (req 28528640->28532735) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.470659:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2ff9d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28528640->28532735] (req 28528640->28532735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.470667:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2ff9d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28528640->28532735] (req 28528640->28532735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760786.470816:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4130700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512755584:12345-192.168.203.56@tcp:4:dd.0 Request processed in 65214us (65422us total) trans 4294974284 rc 0/0 00000100:00100000:0.0:1789760786.470827:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6995 00000100:00100000:0.0:1789760786.470837:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 765880929054 time 765969886860 state 0x2 ref 0 00000100:00100000:3.0:1789760786.483747:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760786.483840:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512755712 00000100:00100000:0.0:1789760786.483850:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512755712 00000100:00100000:0.0:1789760786.483906:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 765880929054 time 765969886860 state 0x0 ref 1 00000100:00100000:0.0:1789760786.483922:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 765970147276 time 765969886860 state 0x1 ref 1 00000100:00100000:0.0:1789760786.484026:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 765970147276 time 766048325300 state 0x0 ref 1 00000100:00100000:0.0:1789760786.484040:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febe7904e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512755712:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760786.484091:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28532736, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0xe007 00010000:00010000:0.0:1789760786.484100:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2ffa4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.484117:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=766s v=(0 0) 00010000:00010000:0.0:1789760786.484122:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760786.484128:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2ffa4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28532736->28536831] (req 28532736->28536831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.484142:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2ffa4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28532736->28536831] (req 28532736->28536831) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760786.546204:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974285 is committed 00002000:00010000:0.0:1789760786.546367:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28532736 -> 28536832 00010000:00010000:0.0:1789760786.546383:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2ffa4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28532736->28536831] (req 28532736->28536831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.546395:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2ffa4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28532736->28536831] (req 28532736->28536831) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.546405:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2ffa4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28532736->28536831] (req 28532736->28536831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.546413:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760786.546418:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2ffa4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28532736->28536831] (req 28532736->28536831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.546427:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2ffa4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28532736->28536831] (req 28532736->28536831) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.546446:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2ffa4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28532736->28536831] (req 28532736->28536831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.546455:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2ffa4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28532736->28536831] (req 28532736->28536831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760786.546557:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febe7904e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512755712:12345-192.168.203.56@tcp:4:dd.0 Request processed in 62514us (62854us total) trans 4294974285 rc 0/0 00000100:00100000:0.0:1789760786.546567:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6996 00000100:00100000:0.0:1789760786.546577:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 765970147276 time 766048325300 state 0x2 ref 0 00000100:00100000:1.0:1789760786.552334:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760786.552433:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512755840 00000100:00100000:2.0:1789760786.552444:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512755840 00000100:00100000:2.0:1789760786.552498:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 765970147276 time 766048325300 state 0x0 ref 1 00000100:00100000:2.0:1789760786.552512:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 766048585716 time 766048325300 state 0x1 ref 1 00000100:00100000:2.0:1789760786.552543:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 766048585716 time 766116843137 state 0x0 ref 1 00000100:00100000:2.0:1789760786.552555:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4131f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512755840:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760786.552601:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28536832, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0xe007 00010000:00010000:2.0:1789760786.552610:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2ffab lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.552626:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=766s v=(0 0) 00010000:00010000:2.0:1789760786.552631:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760786.552637:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2ffab lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28536832->28540927] (req 28536832->28540927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.552651:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2ffab lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28536832->28540927] (req 28536832->28540927) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760786.586050:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28536832 -> 28540928 00002000:00010000:2.0:1789760786.586059:0:13062:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760785 -> 1789760786 00002000:00010000:2.0:1789760786.586061:0:13062:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760785 -> 1789760786 00010000:00010000:2.0:1789760786.586074:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2ffab lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28536832->28540927] (req 28536832->28540927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.586087:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2ffab lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28536832->28540927] (req 28536832->28540927) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.586097:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2ffab lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28536832->28540927] (req 28536832->28540927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.586105:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760786.586110:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2ffab lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28536832->28540927] (req 28536832->28540927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.586118:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2ffab lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28536832->28540927] (req 28536832->28540927) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.586136:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2ffab lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28536832->28540927] (req 28536832->28540927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.586144:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2ffab lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28536832->28540927] (req 28536832->28540927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760786.586236:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4131f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2361:x1876699512755840:12345-192.168.203.56@tcp:4:dd.0 Request processed in 33677us (33905us total) trans 4294974286 rc 0/0 00000100:00100000:2.0:1789760786.586246:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6997 00000100:00100000:2.0:1789760786.586256:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 766048585716 time 766116843137 state 0x2 ref 0 00000001:00080000:2.0:1789760786.586293:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974286 is committed 00000100:00100000:1.0:1789760786.595340:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760786.595438:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512755968 00000100:00100000:0.0:1789760786.595448:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512755968 00000100:00100000:0.0:1789760786.595501:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 766048585716 time 766116843137 state 0x0 ref 1 00000100:00100000:0.0:1789760786.595515:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 766117103553 time 766116843137 state 0x1 ref 1 00000100:00100000:0.0:1789760786.595546:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 766117103553 time 766159846152 state 0x0 ref 1 00000100:00100000:0.0:1789760786.595559:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4130380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512755968:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760786.595608:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28540928, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0xe007 00010000:00010000:0.0:1789760786.595616:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2ffb2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.595632:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=766s v=(0 0) 00010000:00010000:0.0:1789760786.595647:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760786.595653:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2ffb2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28540928->28545023] (req 28540928->28545023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.595676:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2ffb2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28540928->28545023] (req 28540928->28545023) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760786.645523:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28540928 -> 28545024 00010000:00010000:3.0:1789760786.645545:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2ffb2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28540928->28545023] (req 28540928->28545023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.645559:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2ffb2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28540928->28545023] (req 28540928->28545023) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.645575:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2ffb2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28540928->28545023] (req 28540928->28545023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.645583:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760786.645587:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2ffb2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28540928->28545023] (req 28540928->28545023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.645597:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2ffb2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28540928->28545023] (req 28540928->28545023) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.645616:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2ffb2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28540928->28545023] (req 28540928->28545023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.645625:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c2ffb2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28540928->28545023] (req 28540928->28545023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760786.645770:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4130380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2359:x1876699512755968:12345-192.168.203.56@tcp:4:dd.0 Request processed in 50209us (50436us total) trans 4294974287 rc 0/0 00000100:00100000:3.0:1789760786.645782:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6998 00000100:00100000:3.0:1789760786.645791:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 766117103553 time 766159846152 state 0x2 ref 0 00000001:00080000:2.0:1789760786.646428:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974287 is committed 00000100:00100000:3.0:1789760786.651094:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760786.651189:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512756096 00000100:00100000:0.0:1789760786.651199:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512756096 00000100:00100000:0.0:1789760786.651255:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 766117103553 time 766159846152 state 0x0 ref 1 00000100:00100000:0.0:1789760786.651281:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 766160106568 time 766159846152 state 0x1 ref 1 00000100:00100000:0.0:1789760786.651313:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 766160106568 time 766215612370 state 0x0 ref 1 00000100:00100000:0.0:1789760786.651326:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac28f0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512756096:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760786.651383:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28545024, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0xe007 00010000:00010000:0.0:1789760786.651394:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2ffb9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.651411:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=766s v=(0 0) 00010000:00010000:0.0:1789760786.651415:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760786.651421:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2ffb9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28545024->28549119] (req 28545024->28549119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.651435:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2ffb9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28545024->28549119] (req 28545024->28549119) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760786.683623:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28545024 -> 28549120 00010000:00010000:0.0:1789760786.683642:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2ffb9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28545024->28549119] (req 28545024->28549119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.683661:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2ffb9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28545024->28549119] (req 28545024->28549119) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.683680:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2ffb9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28545024->28549119] (req 28545024->28549119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.683689:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760786.683719:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2ffb9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28545024->28549119] (req 28545024->28549119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.683731:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2ffb9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28545024->28549119] (req 28545024->28549119) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.683752:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2ffb9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28545024->28549119] (req 28545024->28549119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.683760:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c2ffb9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28545024->28549119] (req 28545024->28549119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760786.683843:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac28f0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2358:x1876699512756096:12345-192.168.203.56@tcp:4:dd.0 Request processed in 32515us (32754us total) trans 4294974288 rc 0/0 00000100:00100000:0.0:1789760786.683854:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 6999 00000100:00100000:0.0:1789760786.683864:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 766160106568 time 766215612370 state 0x2 ref 0 00000001:00080000:3.0:1789760786.685102:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974288 is committed 00000100:00100000:1.0:1789760786.696273:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760786.696351:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512756224 00000100:00100000:2.0:1789760786.696362:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512756224 00000100:00100000:2.0:1789760786.696417:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 766160106568 time 766215612370 state 0x0 ref 1 00000100:00100000:2.0:1789760786.696434:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 766215872786 time 766215612370 state 0x1 ref 1 00000100:00100000:2.0:1789760786.696465:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 766215872786 time 766260765651 state 0x0 ref 1 00000100:00100000:2.0:1789760786.696477:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4133b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512756224:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760786.696538:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28549120, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0xe007 00010000:00010000:2.0:1789760786.696547:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2ffc0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.696564:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=766s v=(0 0) 00010000:00010000:2.0:1789760786.696568:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760786.696573:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2ffc0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28549120->28553215] (req 28549120->28553215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.696591:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2ffc0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28549120->28553215] (req 28549120->28553215) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760786.731827:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974289 is committed 00002000:00010000:2.0:1789760786.733691:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28549120 -> 28553216 00010000:00010000:2.0:1789760786.733816:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2ffc0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28549120->28553215] (req 28549120->28553215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.733830:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2ffc0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28549120->28553215] (req 28549120->28553215) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.733840:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2ffc0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28549120->28553215] (req 28549120->28553215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.733848:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760786.733853:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2ffc0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28549120->28553215] (req 28549120->28553215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.733864:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2ffc0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28549120->28553215] (req 28549120->28553215) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.733883:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2ffc0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28549120->28553215] (req 28549120->28553215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.733892:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c2ffc0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28549120->28553215] (req 28549120->28553215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760786.736860:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4133b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512756224:12345-192.168.203.56@tcp:4:dd.0 Request processed in 40372us (40583us total) trans 4294974289 rc 0/0 00000100:00100000:3.0:1789760786.736880:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7000 00000100:00100000:3.0:1789760786.736891:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 766215872786 time 766260765651 state 0x2 ref 0 00000100:00100000:1.0:1789760786.744134:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760786.744225:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512756352 00000100:00100000:3.0:1789760786.744235:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512756352 00000100:00100000:3.0:1789760786.744287:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 766215872786 time 766260765651 state 0x0 ref 1 00000100:00100000:3.0:1789760786.744301:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 766261026067 time 766260765651 state 0x1 ref 1 00000100:00100000:3.0:1789760786.744328:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 766261026067 time 766308628594 state 0x0 ref 1 00000100:00100000:3.0:1789760786.744339:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4133800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512756352:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760786.744390:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28553216, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0xe007 00010000:00010000:3.0:1789760786.744399:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2ffc7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.744413:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=766s v=(0 0) 00010000:00010000:3.0:1789760786.744418:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760786.744424:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2ffc7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28553216->28557311] (req 28553216->28557311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.744437:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2ffc7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28553216->28557311] (req 28553216->28557311) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760786.789201:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974290 is committed 00002000:00010000:3.0:1789760786.789313:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28553216 -> 28557312 00010000:00010000:3.0:1789760786.789330:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2ffc7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28553216->28557311] (req 28553216->28557311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.789344:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2ffc7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28553216->28557311] (req 28553216->28557311) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.789353:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2ffc7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28553216->28557311] (req 28553216->28557311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.789361:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760786.789366:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2ffc7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28553216->28557311] (req 28553216->28557311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.789375:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2ffc7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28553216->28557311] (req 28553216->28557311) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.789402:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2ffc7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28553216->28557311] (req 28553216->28557311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.789412:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2ffc7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28553216->28557311] (req 28553216->28557311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760786.789506:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4133800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512756352:12345-192.168.203.56@tcp:4:dd.0 Request processed in 45164us (45376us total) trans 4294974290 rc 0/0 00000100:00100000:3.0:1789760786.789516:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7001 00000100:00100000:3.0:1789760786.789525:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 766261026067 time 766308628594 state 0x2 ref 0 00000100:00100000:0.0:1789760786.791822:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760786.791907:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512756480 00000100:00100000:1.0:1789760786.791916:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512756480 00000100:00100000:1.0:1789760786.791965:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 766261026067 time 766308628594 state 0x0 ref 1 00000100:00100000:1.0:1789760786.791979:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 766308889010 time 766308628594 state 0x1 ref 1 00000100:00100000:1.0:1789760786.792055:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 766308889010 time 766356354543 state 0x0 ref 1 00000100:00100000:1.0:1789760786.792068:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2875c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512756480:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760786.792113:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28557312, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0xe007 00010000:00010000:1.0:1789760786.792122:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2ffce lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.792136:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=766s v=(0 0) 00010000:00010000:1.0:1789760786.792140:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760786.792343:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2ffce lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28557312->28561407] (req 28557312->28561407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.792361:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2ffce lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28557312->28561407] (req 28557312->28561407) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.803167:0:7647:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:2.0:1789760786.829039:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974291 is committed 00002000:00010000:1.0:1789760786.829211:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28557312 -> 28561408 00010000:00010000:1.0:1789760786.829227:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2ffce lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28557312->28561407] (req 28557312->28561407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.829239:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2ffce lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28557312->28561407] (req 28557312->28561407) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.829248:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2ffce lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28557312->28561407] (req 28557312->28561407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.829255:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760786.829260:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2ffce lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28557312->28561407] (req 28557312->28561407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.829269:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2ffce lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28557312->28561407] (req 28557312->28561407) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.829287:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2ffce lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28557312->28561407] (req 28557312->28561407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.829295:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c2ffce lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28557312->28561407] (req 28557312->28561407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760786.829392:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2875c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512756480:12345-192.168.203.56@tcp:4:dd.0 Request processed in 37322us (37725us total) trans 4294974291 rc 0/0 00000100:00100000:1.0:1789760786.829403:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7002 00000100:00100000:1.0:1789760786.829412:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 766308889010 time 766356354543 state 0x2 ref 0 00000100:00100000:1.0:1789760786.832505:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760786.832640:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512756608 00000100:00100000:3.0:1789760786.832651:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512756608 00000100:00100000:3.0:1789760786.832745:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 766308889010 time 766356354543 state 0x0 ref 1 00000100:00100000:3.0:1789760786.832761:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 766356614959 time 766356354543 state 0x1 ref 1 00000100:00100000:3.0:1789760786.832790:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 766356614959 time 766397089388 state 0x0 ref 1 00000100:00100000:3.0:1789760786.832803:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febd02e0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512756608:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760786.832850:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28561408, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0xe007 00010000:00010000:3.0:1789760786.832859:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2ffd5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.832875:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=766s v=(0 0) 00010000:00010000:3.0:1789760786.832879:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760786.832886:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2ffd5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28561408->28565503] (req 28561408->28565503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.832899:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2ffd5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28561408->28565503] (req 28561408->28565503) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760786.865616:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28561408 -> 28565504 00010000:00010000:3.0:1789760786.865637:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2ffd5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28561408->28565503] (req 28561408->28565503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.865650:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2ffd5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28561408->28565503] (req 28561408->28565503) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.865664:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2ffd5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28561408->28565503] (req 28561408->28565503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.865684:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760786.865689:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2ffd5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28561408->28565503] (req 28561408->28565503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.865725:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2ffd5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28561408->28565503] (req 28561408->28565503) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.865746:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2ffd5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28561408->28565503] (req 28561408->28565503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.865754:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c2ffd5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28561408->28565503] (req 28561408->28565503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760786.865842:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febd02e0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2361:x1876699512756608:12345-192.168.203.56@tcp:4:dd.0 Request processed in 33037us (33341us total) trans 4294974292 rc 0/0 00000100:00100000:3.0:1789760786.865852:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7003 00000100:00100000:3.0:1789760786.865862:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 766356614959 time 766397089388 state 0x2 ref 0 00000001:00080000:1.0:1789760786.866524:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974292 is committed 00000100:00100000:0.0:1789760786.872269:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760786.872342:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512756736 00000100:00100000:3.0:1789760786.872352:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512756736 00000100:00100000:3.0:1789760786.872514:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 766356614959 time 766397089388 state 0x0 ref 1 00000100:00100000:3.0:1789760786.872529:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 766397349804 time 766397089388 state 0x1 ref 1 00000100:00100000:3.0:1789760786.872562:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 766397349804 time 766436861418 state 0x0 ref 1 00000100:00100000:3.0:1789760786.872575:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2877100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512756736:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760786.872624:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28565504, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0xe007 00010000:00010000:3.0:1789760786.872634:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2ffdc lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.872648:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=766s v=(0 0) 00010000:00010000:3.0:1789760786.872653:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760786.872658:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2ffdc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28565504->28569599] (req 28565504->28569599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.872682:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2ffdc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28565504->28569599] (req 28565504->28569599) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760786.914409:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974293 is committed 00002000:00010000:3.0:1789760786.914624:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28565504 -> 28569600 00010000:00010000:3.0:1789760786.914639:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2ffdc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28565504->28569599] (req 28565504->28569599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.914650:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2ffdc lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28565504->28569599] (req 28565504->28569599) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.914658:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2ffdc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28565504->28569599] (req 28565504->28569599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.914676:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760786.914682:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2ffdc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28565504->28569599] (req 28565504->28569599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.914690:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2ffdc lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28565504->28569599] (req 28565504->28569599) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.914732:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2ffdc lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28565504->28569599] (req 28565504->28569599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.914740:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2ffdc lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28565504->28569599] (req 28565504->28569599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760786.914829:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2877100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512756736:12345-192.168.203.56@tcp:4:dd.0 Request processed in 42251us (42563us total) trans 4294974293 rc 0/0 00000100:00100000:3.0:1789760786.914838:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7004 00000100:00100000:3.0:1789760786.914848:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 766397349804 time 766436861418 state 0x2 ref 0 00000100:00100000:2.0:1789760786.918093:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760786.918192:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512756864 00000100:00100000:1.0:1789760786.918202:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512756864 00000100:00100000:1.0:1789760786.918254:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 766397349804 time 766436861418 state 0x0 ref 1 00000100:00100000:1.0:1789760786.918269:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 766437121834 time 766436861418 state 0x1 ref 1 00000100:00100000:1.0:1789760786.918308:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 766437121834 time 766482608241 state 0x0 ref 1 00000100:00100000:1.0:1789760786.918320:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febfb8d4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512756864:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760786.918365:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28569600, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0xe007 00010000:00010000:1.0:1789760786.918375:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2ffe3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.918400:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=766s v=(0 0) 00010000:00010000:1.0:1789760786.918404:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760786.918411:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2ffe3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28569600->28573695] (req 28569600->28573695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.918426:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2ffe3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28569600->28573695] (req 28569600->28573695) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760786.952589:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974294 is committed 00002000:00010000:1.0:1789760786.952647:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28569600 -> 28573696 00010000:00010000:1.0:1789760786.952662:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2ffe3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28569600->28573695] (req 28569600->28573695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.952687:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2ffe3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28569600->28573695] (req 28569600->28573695) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.952716:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2ffe3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28569600->28573695] (req 28569600->28573695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.952729:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760786.952735:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2ffe3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28569600->28573695] (req 28569600->28573695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.952744:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2ffe3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28569600->28573695] (req 28569600->28573695) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.953011:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2ffe3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28569600->28573695] (req 28569600->28573695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.953021:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c2ffe3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28569600->28573695] (req 28569600->28573695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760786.953112:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febfb8d4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512756864:12345-192.168.203.56@tcp:4:dd.0 Request processed in 34790us (35024us total) trans 4294974294 rc 0/0 00000100:00100000:1.0:1789760786.953122:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7005 00000100:00100000:1.0:1789760786.953132:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 766437121834 time 766482608241 state 0x2 ref 0 00000100:00100000:1.0:1789760786.961635:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760786.961881:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512756992 00000100:00100000:3.0:1789760786.961892:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512756992 00000100:00100000:3.0:1789760786.961943:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 766437121834 time 766482608241 state 0x0 ref 1 00000100:00100000:3.0:1789760786.961958:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 766482868657 time 766482608241 state 0x1 ref 1 00000100:00100000:3.0:1789760786.962049:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 766482868657 time 766526348447 state 0x0 ref 1 00000100:00100000:3.0:1789760786.962063:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febd02e3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512756992:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760786.962116:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28573696, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0xe007 00010000:00010000:3.0:1789760786.962125:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2ffea lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.962141:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=766s v=(0 0) 00010000:00010000:3.0:1789760786.962145:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760786.962151:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2ffea lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28573696->28577791] (req 28573696->28577791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.962165:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2ffea lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28573696->28577791] (req 28573696->28577791) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760787.002719:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974295 is committed 00002000:00010000:3.0:1789760787.002861:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28573696 -> 28577792 00010000:00010000:3.0:1789760787.002878:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2ffea lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28573696->28577791] (req 28573696->28577791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.002890:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2ffea lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28573696->28577791] (req 28573696->28577791) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.002899:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2ffea lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28573696->28577791] (req 28573696->28577791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.002906:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760787.002911:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2ffea lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28573696->28577791] (req 28573696->28577791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.002919:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2ffea lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28573696->28577791] (req 28573696->28577791) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.002943:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2ffea lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28573696->28577791] (req 28573696->28577791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.002951:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2ffea lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28573696->28577791] (req 28573696->28577791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760787.003048:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febd02e3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512756992:12345-192.168.203.56@tcp:4:dd.0 Request processed in 40983us (41418us total) trans 4294974295 rc 0/0 00000100:00100000:3.0:1789760787.003059:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7006 00000100:00100000:3.0:1789760787.003068:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 766482868657 time 766526348447 state 0x2 ref 0 00000100:00100000:3.0:1789760787.006304:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760787.006410:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512757120 00000100:00100000:2.0:1789760787.006421:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512757120 00000100:00100000:2.0:1789760787.006505:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 766482868657 time 766526348447 state 0x0 ref 1 00000100:00100000:2.0:1789760787.006520:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 766526608863 time 766526348447 state 0x1 ref 1 00000100:00100000:2.0:1789760787.006552:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 766526608863 time 766570851223 state 0x0 ref 1 00000100:00100000:2.0:1789760787.006565:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac28f2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512757120:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760787.006623:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28577792, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0xe007 00010000:00010000:2.0:1789760787.006632:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2fff1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.006648:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=766s v=(0 0) 00010000:00010000:2.0:1789760787.006652:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760787.006658:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2fff1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28577792->28581887] (req 28577792->28581887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.006680:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2fff1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28577792->28581887] (req 28577792->28581887) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1789760787.007914:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:3.0:1789760787.034791:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974296 is committed 00002000:00010000:2.0:1789760787.035355:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28577792 -> 28581888 00010000:00010000:2.0:1789760787.035372:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2fff1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28577792->28581887] (req 28577792->28581887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.035385:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2fff1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28577792->28581887] (req 28577792->28581887) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.035394:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2fff1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28577792->28581887] (req 28577792->28581887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.035402:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760787.035407:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2fff1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28577792->28581887] (req 28577792->28581887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.035416:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2fff1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28577792->28581887] (req 28577792->28581887) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.035436:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2fff1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28577792->28581887] (req 28577792->28581887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.035444:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c2fff1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28577792->28581887] (req 28577792->28581887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760787.035635:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac28f2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512757120:12345-192.168.203.56@tcp:4:dd.0 Request processed in 29068us (29336us total) trans 4294974296 rc 0/0 00000100:00100000:2.0:1789760787.035646:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7007 00000100:00100000:2.0:1789760787.035656:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 766526608863 time 766570851223 state 0x2 ref 0 00000100:00100000:2.0:1789760787.046926:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760787.048347:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512757248 00000100:00100000:1.0:1789760787.048359:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512757248 00000100:00100000:1.0:1789760787.048421:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 766526608863 time 766570851223 state 0x0 ref 1 00000100:00100000:1.0:1789760787.048437:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 766571111639 time 766570851223 state 0x1 ref 1 00000100:00100000:1.0:1789760787.048472:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 766571111639 time 766612771861 state 0x0 ref 1 00000100:00100000:1.0:1789760787.048486:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febc7b32680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512757248:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760787.048535:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28581888, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0xe007 00010000:00010000:1.0:1789760787.048545:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2fff8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.048561:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=766s v=(0 0) 00010000:00010000:1.0:1789760787.048565:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760787.048571:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2fff8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28581888->28585983] (req 28581888->28585983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.048585:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2fff8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28581888->28585983] (req 28581888->28585983) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760787.081261:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28581888 -> 28585984 00010000:00010000:1.0:1789760787.081280:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2fff8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28581888->28585983] (req 28581888->28585983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.081292:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2fff8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28581888->28585983] (req 28581888->28585983) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.081302:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2fff8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28581888->28585983] (req 28581888->28585983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.081309:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760787.081325:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2fff8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28581888->28585983] (req 28581888->28585983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.081334:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2fff8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28581888->28585983] (req 28581888->28585983) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.081354:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2fff8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28581888->28585983] (req 28581888->28585983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.081362:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c2fff8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28581888->28585983] (req 28581888->28585983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760787.081459:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febc7b32680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2359:x1876699512757248:12345-192.168.203.56@tcp:4:dd.0 Request processed in 32971us (34537us total) trans 4294974297 rc 0/0 00000100:00100000:1.0:1789760787.081470:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7008 00000100:00100000:1.0:1789760787.081479:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 766571111639 time 766612771861 state 0x2 ref 0 00000001:00080000:3.0:1789760787.082530:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974297 is committed 00000100:00100000:1.0:1789760787.084481:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760787.084598:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512757376 00000100:00100000:3.0:1789760787.084607:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512757376 00000100:00100000:3.0:1789760787.084660:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 766571111639 time 766612771861 state 0x0 ref 1 00000100:00100000:3.0:1789760787.084685:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 766613032277 time 766612771861 state 0x1 ref 1 00000100:00100000:3.0:1789760787.084747:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 766613032277 time 766649046488 state 0x0 ref 1 00000100:00100000:3.0:1789760787.084760:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febd02e3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512757376:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760787.084804:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28585984, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0xe007 00010000:00010000:3.0:1789760787.084812:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2ffff lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.084826:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=766s v=(0 0) 00010000:00010000:3.0:1789760787.084831:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760787.084837:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2ffff lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28585984->28590079] (req 28585984->28590079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.084851:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2ffff lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28585984->28590079] (req 28585984->28590079) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760787.119047:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974298 is committed 00002000:00010000:1.0:1789760787.119161:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28585984 -> 28590080 00010000:00010000:1.0:1789760787.119177:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2ffff lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28585984->28590079] (req 28585984->28590079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.119190:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2ffff lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28585984->28590079] (req 28585984->28590079) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.119199:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2ffff lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28585984->28590079] (req 28585984->28590079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.119211:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760787.119216:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2ffff lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28585984->28590079] (req 28585984->28590079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.119224:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2ffff lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28585984->28590079] (req 28585984->28590079) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.119244:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2ffff lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28585984->28590079] (req 28585984->28590079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.119253:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c2ffff lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28585984->28590079] (req 28585984->28590079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760787.119380:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febd02e3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512757376:12345-192.168.203.56@tcp:4:dd.0 Request processed in 34618us (34902us total) trans 4294974298 rc 0/0 00000100:00100000:1.0:1789760787.119392:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7009 00000100:00100000:1.0:1789760787.119401:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 766613032277 time 766649046488 state 0x2 ref 0 00000100:00100000:0.0:1789760787.121616:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760787.121965:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512757504 00000100:00100000:2.0:1789760787.121974:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512757504 00000100:00100000:2.0:1789760787.122043:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 766613032277 time 766649046488 state 0x0 ref 1 00000100:00100000:2.0:1789760787.122058:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 766649306904 time 766649046488 state 0x1 ref 1 00000100:00100000:2.0:1789760787.122085:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 766649306904 time 766686385332 state 0x0 ref 1 00000100:00100000:2.0:1789760787.122097:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2fba680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512757504:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760787.122142:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28590080, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0xe007 00010000:00010000:2.0:1789760787.122151:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c30006 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.122165:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=766s v=(0 0) 00010000:00010000:2.0:1789760787.122169:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760787.122175:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c30006 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28590080->28594175] (req 28590080->28594175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.122188:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c30006 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28590080->28594175] (req 28590080->28594175) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760787.165090:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974299 is committed 00002000:00010000:1.0:1789760787.165227:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28590080 -> 28594176 00010000:00010000:1.0:1789760787.165243:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c30006 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28590080->28594175] (req 28590080->28594175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.165255:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c30006 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28590080->28594175] (req 28590080->28594175) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.165269:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c30006 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28590080->28594175] (req 28590080->28594175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.165277:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760787.165282:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c30006 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28590080->28594175] (req 28590080->28594175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.165290:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c30006 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28590080->28594175] (req 28590080->28594175) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.165309:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c30006 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28590080->28594175] (req 28590080->28594175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.165317:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c30006 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28590080->28594175] (req 28590080->28594175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760787.165408:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2fba680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512757504:12345-192.168.203.56@tcp:4:dd.0 Request processed in 43307us (43794us total) trans 4294974299 rc 0/0 00000100:00100000:1.0:1789760787.165417:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7010 00000100:00100000:1.0:1789760787.165426:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 766649306904 time 766686385332 state 0x2 ref 0 00000100:00100000:2.0:1789760787.169186:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760787.169322:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512757632 00000100:00100000:1.0:1789760787.169332:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512757632 00000100:00100000:1.0:1789760787.169389:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 766649306904 time 766686385332 state 0x0 ref 1 00000100:00100000:1.0:1789760787.169404:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 766686645748 time 766686385332 state 0x1 ref 1 00000100:00100000:1.0:1789760787.169442:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 766686645748 time 766733741374 state 0x0 ref 1 00000100:00100000:1.0:1789760787.169470:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febc7b33100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512757632:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760787.169518:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28594176, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0xe007 00010000:00010000:1.0:1789760787.169527:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c3000d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.169541:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=766s v=(0 0) 00010000:00010000:1.0:1789760787.169545:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760787.169552:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c3000d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28594176->28598271] (req 28594176->28598271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.169564:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c3000d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28594176->28598271] (req 28594176->28598271) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760787.216009:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974300 is committed 00002000:00010000:1.0:1789760787.216136:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28594176 -> 28598272 00010000:00010000:1.0:1789760787.216153:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c3000d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28594176->28598271] (req 28594176->28598271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.216166:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c3000d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28594176->28598271] (req 28594176->28598271) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.216176:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c3000d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28594176->28598271] (req 28594176->28598271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.216183:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760787.216189:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c3000d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28594176->28598271] (req 28594176->28598271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.216199:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c3000d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28594176->28598271] (req 28594176->28598271) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.216217:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c3000d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28594176->28598271] (req 28594176->28598271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.216225:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c3000d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28594176->28598271] (req 28594176->28598271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760787.216349:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febc7b33100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512757632:12345-192.168.203.56@tcp:4:dd.0 Request processed in 46893us (47168us total) trans 4294974300 rc 0/0 00000100:00100000:1.0:1789760787.216360:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7011 00000100:00100000:1.0:1789760787.216380:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 766686645748 time 766733741374 state 0x2 ref 0 00000100:00100000:2.0:1789760787.219961:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760787.220036:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512757760 00000100:00100000:1.0:1789760787.220056:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512757760 00000100:00100000:1.0:1789760787.220111:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 766686645748 time 766733741374 state 0x0 ref 1 00000100:00100000:1.0:1789760787.220126:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 766734001790 time 766733741374 state 0x1 ref 1 00000100:00100000:1.0:1789760787.220157:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 766734001790 time 766784457035 state 0x0 ref 1 00000100:00100000:1.0:1789760787.220170:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febc7b33480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512757760:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760787.220218:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28598272, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0xe007 00010000:00010000:1.0:1789760787.220228:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c30014 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.220245:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=766s v=(0 0) 00010000:00010000:1.0:1789760787.220249:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760787.220256:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c30014 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28598272->28602367] (req 28598272->28602367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.220271:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c30014 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28598272->28602367] (req 28598272->28602367) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760787.280939:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28598272 -> 28602368 00010000:00010000:1.0:1789760787.280958:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c30014 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28598272->28602367] (req 28598272->28602367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.280972:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c30014 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28598272->28602367] (req 28598272->28602367) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.280982:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c30014 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28598272->28602367] (req 28598272->28602367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.280989:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760787.281006:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c30014 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28598272->28602367] (req 28598272->28602367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.281015:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c30014 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28598272->28602367] (req 28598272->28602367) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.281049:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c30014 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28598272->28602367] (req 28598272->28602367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.281057:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c30014 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28598272->28602367] (req 28598272->28602367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760787.281454:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febc7b33480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2358:x1876699512757760:12345-192.168.203.56@tcp:4:dd.0 Request processed in 61280us (61496us total) trans 4294974301 rc 0/0 00000100:00100000:1.0:1789760787.281464:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7012 00000100:00100000:1.0:1789760787.281473:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 766734001790 time 766784457035 state 0x2 ref 0 00000001:00080000:1.0:1789760787.281511:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974301 is committed 00000100:00100000:2.0:1789760787.285095:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760787.285414:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512757888 00000100:00100000:1.0:1789760787.285423:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512757888 00000100:00100000:1.0:1789760787.285472:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 766734001790 time 766784457035 state 0x0 ref 1 00000100:00100000:1.0:1789760787.285486:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 766784717451 time 766784457035 state 0x1 ref 1 00000100:00100000:1.0:1789760787.285506:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 766784717451 time 766849806577 state 0x0 ref 1 00000100:00100000:1.0:1789760787.285517:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febfbfe5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512757888:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760787.285570:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28602368, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0xe007 00010000:00010000:1.0:1789760787.285579:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c3001b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.285601:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=766s v=(0 0) 00010000:00010000:1.0:1789760787.285605:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760787.285611:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c3001b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28602368->28606463] (req 28602368->28606463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.285624:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c3001b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28602368->28606463] (req 28602368->28606463) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760787.320033:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28602368 -> 28606464 00010000:00010000:1.0:1789760787.320051:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c3001b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28602368->28606463] (req 28602368->28606463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.320065:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c3001b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28602368->28606463] (req 28602368->28606463) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.320149:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c3001b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28602368->28606463] (req 28602368->28606463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.320159:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760787.320164:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c3001b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28602368->28606463] (req 28602368->28606463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.320173:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c3001b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28602368->28606463] (req 28602368->28606463) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.320192:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c3001b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28602368->28606463] (req 28602368->28606463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.320201:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c3001b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28602368->28606463] (req 28602368->28606463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760787.320291:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febfbfe5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2358:x1876699512757888:12345-192.168.203.56@tcp:4:dd.0 Request processed in 34770us (35200us total) trans 4294974302 rc 0/0 00000100:00100000:1.0:1789760787.320301:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7013 00000100:00100000:1.0:1789760787.320311:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 766784717451 time 766849806577 state 0x2 ref 0 00000001:00080000:1.0:1789760787.320349:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974302 is committed 00000100:00100000:0.0:1789760787.324516:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760787.324608:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512758016 00000100:00100000:1.0:1789760787.324617:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512758016 00000100:00100000:1.0:1789760787.324873:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 766784717451 time 766849806577 state 0x0 ref 1 00000100:00100000:1.0:1789760787.324891:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 766850066993 time 766849806577 state 0x1 ref 1 00000100:00100000:1.0:1789760787.324961:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 766850066993 time 766889260289 state 0x0 ref 1 00000100:00100000:1.0:1789760787.324975:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2fb8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512758016:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760787.325030:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28606464, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0xe007 00010000:00010000:1.0:1789760787.325038:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c30022 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.325057:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=766s v=(0 0) 00010000:00010000:1.0:1789760787.325061:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760787.325067:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c30022 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28606464->28610559] (req 28606464->28610559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.325188:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c30022 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28606464->28610559] (req 28606464->28610559) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760787.350240:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974303 is committed 00002000:00010000:1.0:1789760787.350318:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28606464 -> 28610560 00010000:00010000:1.0:1789760787.350333:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c30022 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28606464->28610559] (req 28606464->28610559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.350346:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c30022 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28606464->28610559] (req 28606464->28610559) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.350355:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c30022 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28606464->28610559] (req 28606464->28610559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.350362:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760787.350367:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c30022 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28606464->28610559] (req 28606464->28610559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.350377:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c30022 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28606464->28610559] (req 28606464->28610559) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.350396:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c30022 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28606464->28610559] (req 28606464->28610559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.350403:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c30022 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28606464->28610559] (req 28606464->28610559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760787.350541:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2fb8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512758016:12345-192.168.203.56@tcp:4:dd.0 Request processed in 25563us (26029us total) trans 4294974303 rc 0/0 00000100:00100000:1.0:1789760787.350551:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7014 00000100:00100000:1.0:1789760787.350561:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 766850066993 time 766889260289 state 0x2 ref 0 00000100:00100000:3.0:1789760787.353441:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760787.353519:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512758144 00000100:00100000:1.0:1789760787.353529:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512758144 00000100:00100000:1.0:1789760787.353582:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 766850066993 time 766889260289 state 0x0 ref 1 00000100:00100000:1.0:1789760787.353597:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 766889520705 time 766889260289 state 0x1 ref 1 00000100:00100000:1.0:1789760787.353629:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 766889520705 time 766917929178 state 0x0 ref 1 00000100:00100000:1.0:1789760787.353640:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4360a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512758144:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760787.353688:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28610560, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0xe007 00010000:00010000:1.0:1789760787.353731:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c30029 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.353749:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=766s v=(0 0) 00010000:00010000:1.0:1789760787.353753:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760787.353760:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c30029 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28610560->28614655] (req 28610560->28614655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.353774:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c30029 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28610560->28614655] (req 28610560->28614655) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760787.379441:0:3310:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8feac4363100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3310:1876699540232320:0@lo:400:kworker.0 00000100:00100000:3.0:1789760787.379517:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760787.379528:0:3309:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8feac4362a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:6df98d54-eb52-4c5c-8fc9-ee9d74f90f98:3309:1876699540232192:0@lo:400:kworker.0 00000100:00100000:1.0:1789760787.379559:0:3309:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760787.379563:0:3310:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8feac2745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3310:1876699540232832:0@lo:400:kworker.0 00000100:00100000:1.0:1789760787.379582:0:3309:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8feac2744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3309:1876699540232704:0@lo:400:kworker.0 00000100:00100000:3.0:1789760787.379586:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760787.379601:0:3309:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760787.379618:0:3311:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8feac4362d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3311:1876699540232448:0@lo:400:kworker.0 00000100:00100000:1.0:1789760787.379625:0:3312:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8feac2745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3312:1876699540232576:0@lo:400:kworker.0 00000100:00100000:3.0:1789760787.379641:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760787.379672:0:8229:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540232320 00000100:00100000:3.0:1789760787.379680:0:8229:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540232320 00000100:00100000:3.0:1789760787.379734:0:8229:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540232704 00000100:00100000:3.0:1789760787.379738:0:8229:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540232704 00000100:00100000:1.0:1789760787.379805:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760787.379862:0:5842:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540232192 00000100:00100000:1.0:1789760787.379866:0:5842:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540232192 00000100:00100000:3.0:1789760787.379890:0:8229:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540232448 00000100:00100000:1.0:1789760787.379894:0:5842:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 415 00000100:00100000:3.0:1789760787.379894:0:8229:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540232448 00000100:00100000:1.0:1789760787.379902:0:5842:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febe00c6850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:6df98d54-eb52-4c5c-8fc9-ee9d74f90f98+19:3309:x1876699540232192:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760787.379914:0:8229:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 516 00000100:00100000:3.0:1789760787.379919:0:8229:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2746a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3310:x1876699540232320:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760787.379952:0:5842:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febe00c6850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:6df98d54-eb52-4c5c-8fc9-ee9d74f90f98+19:3309:x1876699540232192:12345-0@lo:400:kworker.0 Request processed in 49us (391us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760787.379952:0:8229:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2746a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3310:x1876699540232320:12345-0@lo:400:kworker.0 Request processed in 31us (433us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760787.379960:0:5842:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 415 00000100:00100000:3.0:1789760787.379960:0:8229:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 516 00000100:00100000:3.0:1789760787.379975:0:8229:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 517 00000100:00100000:3.0:1789760787.379980:0:8229:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febffb9c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3309:x1876699540232704:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760787.379989:0:13064:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540232832 00000100:00100000:3.0:1789760787.380019:0:8229:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febffb9c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3309:x1876699540232704:12345-0@lo:400:kworker.0 Request processed in 39us (417us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760787.380025:0:8229:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 517 00000100:00100000:3.0:1789760787.380056:0:8229:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 518 00000100:00080000:3.0:1789760787.380118:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789760787.380150:0:3310:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8feac4363100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3310:1876699540232320:0@lo:400:kworker.0 00000100:00100000:3.0:1789760787.380173:0:8229:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2745180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3311:x1876699540232448:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760787.380213:0:8229:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2745180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3311:x1876699540232448:12345-0@lo:400:kworker.0 Request processed in 150us (570us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760787.380220:0:8229:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 518 00000100:00100000:1.0:1789760787.380384:0:13064:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540232832 00000100:00100000:1.0:1789760787.380429:0:13064:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540232576 00000100:00100000:1.0:1789760787.380433:0:13064:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540232576 00000100:00100000:1.0:1789760787.380457:0:13064:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 194 00000100:00100000:1.0:1789760787.380463:0:13064:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2745880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3310:x1876699540232832:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760787.380499:0:13064:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2745880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3310:x1876699540232832:12345-0@lo:400:kworker.0 Request processed in 37us (912us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760787.380505:0:13064:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 194 00000100:00100000:1.0:1789760787.380517:0:13064:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 195 00000100:00100000:1.0:1789760787.380522:0:13064:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febffb9c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3312:x1876699540232576:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760787.380553:0:13064:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febffb9c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3312:x1876699540232576:12345-0@lo:400:kworker.0 Request processed in 30us (748us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760787.380559:0:13064:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 195 00000100:00080000:1.0:1789760787.380590:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:1.0:1789760787.380593:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1789760787.380608:0:3309:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8feac4362a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:6df98d54-eb52-4c5c-8fc9-ee9d74f90f98:3309:1876699540232192:0@lo:400:kworker.0 00000100:00100000:1.0:1789760787.380620:0:3309:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8feac2744e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3309:1876699540232704:0@lo:400:kworker.0 00000100:00100000:1.0:1789760787.380653:0:3311:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8febffb9e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3311:1876699540232960:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1789760787.380682:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760787.380720:0:3311:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8feac4362d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3311:1876699540232448:0@lo:400:kworker.0 00000100:00100000:1.0:1789760787.380738:0:6707:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540232960 00000100:00100000:1.0:1789760787.380741:0:6707:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540232960 00000100:00100000:1.0:1789760787.380774:0:6707:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 214 00000100:00100000:1.0:1789760787.380782:0:6707:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febffb9f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3311:x1876699540232960:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1789760787.380798:0:6707:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 528547840 pending 1703936 free 3829399552 avail 3807379456 00000100:00100000:1.0:1789760787.380832:0:6707:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febffb9f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3311:x1876699540232960:12345-0@lo:13:osp-pre-0-0.0 Request processed in 51us (148us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760787.380838:0:6707:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 214 00000100:00100000:1.0:1789760787.380863:0:3312:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8feac2745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3312:1876699540232576:0@lo:400:kworker.0 00000100:00100000:1.0:1789760787.380885:0:3311:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8febffb9e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3311:1876699540232960:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1789760787.380904:0:3310:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8feac2745f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3310:1876699540232832:0@lo:400:kworker.0 00002000:00010000:2.0:1789760787.383525:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28610560 -> 28614656 00010000:00010000:2.0:1789760787.383545:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c30029 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28610560->28614655] (req 28610560->28614655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.383561:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c30029 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28610560->28614655] (req 28610560->28614655) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.383569:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c30029 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28610560->28614655] (req 28610560->28614655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.383576:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760787.383580:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c30029 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28610560->28614655] (req 28610560->28614655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.383588:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c30029 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28610560->28614655] (req 28610560->28614655) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.383607:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c30029 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28610560->28614655] (req 28610560->28614655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.383614:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c30029 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28610560->28614655] (req 28610560->28614655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760787.383740:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4360a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2358:x1876699512758144:12345-192.168.203.56@tcp:4:dd.0 Request processed in 30059us (30267us total) trans 4294974304 rc 0/0 00000100:00100000:2.0:1789760787.383750:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7015 00000001:00080000:0.0:1789760787.383837:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974304 is committed 00000100:00100000:2.0:1789760787.383877:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 766889520705 time 766917929178 state 0x2 ref 0 00000100:00100000:2.0:1789760787.390816:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760787.390907:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512758272 00000100:00100000:1.0:1789760787.390920:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512758272 00000100:00100000:1.0:1789760787.390980:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 766889520705 time 766917929178 state 0x0 ref 1 00000100:00100000:1.0:1789760787.391005:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 766918189594 time 766917929178 state 0x1 ref 1 00000100:00100000:1.0:1789760787.391037:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 766918189594 time 766955336413 state 0x0 ref 1 00000100:00100000:1.0:1789760787.391049:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febfbfe7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512758272:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760787.391374:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28614656, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0xe007 00010000:00010000:1.0:1789760787.391384:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c30030 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.391400:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=766s v=(0 0) 00010000:00010000:1.0:1789760787.391405:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760787.391411:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c30030 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28614656->28618751] (req 28614656->28618751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.391424:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c30030 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28614656->28618751] (req 28614656->28618751) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760787.450406:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974305 is committed 00002000:00010000:1.0:1789760787.450633:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28614656 -> 28618752 00010000:00010000:1.0:1789760787.450648:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c30030 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28614656->28618751] (req 28614656->28618751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.450801:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c30030 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28614656->28618751] (req 28614656->28618751) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.450814:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c30030 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28614656->28618751] (req 28614656->28618751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.450823:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760787.450828:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c30030 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28614656->28618751] (req 28614656->28618751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.450838:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c30030 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28614656->28618751] (req 28614656->28618751) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.450862:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c30030 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28614656->28618751] (req 28614656->28618751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.450871:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c30030 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28614656->28618751] (req 28614656->28618751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760787.451032:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febfbfe7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512758272:12345-192.168.203.56@tcp:4:dd.0 Request processed in 59980us (60239us total) trans 4294974305 rc 0/0 00000100:00100000:1.0:1789760787.451044:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7016 00000100:00100000:1.0:1789760787.451053:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 766918189594 time 766955336413 state 0x2 ref 0 00000100:00100000:2.0:1789760787.457762:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760787.461443:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512758400 00000100:00100000:3.0:1789760787.461460:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512758400 00000100:00100000:0.0:1789760787.464876:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 766918189594 time 766955336413 state 0x0 ref 1 00000100:00100000:0.0:1789760787.464897:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 766955596829 time 766955336413 state 0x1 ref 1 00000100:00100000:0.0:1789760787.464972:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 766955596829 time 767029271254 state 0x0 ref 1 00000100:00100000:0.0:1789760787.464987:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febea772680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512758400:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760787.465046:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28618752, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0xe007 00010000:00010000:0.0:1789760787.465055:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c30037 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.465072:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=767s v=(0 0) 00010000:00010000:0.0:1789760787.465077:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760787.465083:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c30037 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28618752->28622847] (req 28618752->28622847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.465097:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c30037 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28618752->28622847] (req 28618752->28622847) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760787.519215:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974306 is committed 00002000:00010000:0.0:1789760787.519300:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28618752 -> 28622848 00010000:00010000:0.0:1789760787.519316:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c30037 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28618752->28622847] (req 28618752->28622847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.519328:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c30037 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28618752->28622847] (req 28618752->28622847) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.519337:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c30037 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28618752->28622847] (req 28618752->28622847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.519344:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760787.519349:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c30037 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28618752->28622847] (req 28618752->28622847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.519357:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c30037 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28618752->28622847] (req 28618752->28622847) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.519553:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c30037 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28618752->28622847] (req 28618752->28622847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.519563:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c30037 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28618752->28622847] (req 28618752->28622847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760787.519785:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febea772680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512758400:12345-192.168.203.56@tcp:4:dd.0 Request processed in 54797us (62029us total) trans 4294974306 rc 0/0 00000100:00100000:0.0:1789760787.519796:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7017 00000100:00100000:0.0:1789760787.519807:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 766955596829 time 767029271254 state 0x2 ref 0 00000100:00100000:1.0:1789760787.530952:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760787.531048:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512758528 00000100:00100000:2.0:1789760787.531058:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512758528 00000100:00100000:2.0:1789760787.531182:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 766955596829 time 767029271254 state 0x0 ref 1 00000100:00100000:2.0:1789760787.531198:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 767029531670 time 767029271254 state 0x1 ref 1 00000100:00100000:2.0:1789760787.531229:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 767029531670 time 767095528471 state 0x0 ref 1 00000100:00100000:2.0:1789760787.531242:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febffb9e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512758528:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760787.531559:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28622848, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0xe007 00010000:00010000:2.0:1789760787.531569:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c3003e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.531585:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=767s v=(0 0) 00010000:00010000:2.0:1789760787.531589:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760787.531595:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c3003e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28622848->28626943] (req 28622848->28626943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.531609:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c3003e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28622848->28626943] (req 28622848->28626943) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760787.574151:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974307 is committed 00002000:00010000:0.0:1789760787.574306:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28622848 -> 28626944 00002000:00010000:0.0:1789760787.574311:0:13062:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760786 -> 1789760787 00002000:00010000:0.0:1789760787.574314:0:13062:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760786 -> 1789760787 00010000:00010000:0.0:1789760787.574327:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c3003e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28622848->28626943] (req 28622848->28626943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.574340:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c3003e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28622848->28626943] (req 28622848->28626943) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.574349:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c3003e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28622848->28626943] (req 28622848->28626943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.574357:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760787.574362:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c3003e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28622848->28626943] (req 28622848->28626943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.574371:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c3003e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28622848->28626943] (req 28622848->28626943) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.574389:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c3003e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28622848->28626943] (req 28622848->28626943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.574397:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c3003e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28622848->28626943] (req 28622848->28626943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760787.574485:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febffb9e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512758528:12345-192.168.203.56@tcp:4:dd.0 Request processed in 43241us (43538us total) trans 4294974307 rc 0/0 00000100:00100000:0.0:1789760787.574495:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7018 00000100:00100000:0.0:1789760787.574504:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 767029531670 time 767095528471 state 0x2 ref 0 00000100:00100000:2.0:1789760787.578348:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760787.578437:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512758656 00000100:00100000:1.0:1789760787.578446:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512758656 00000100:00100000:1.0:1789760787.578499:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 767029531670 time 767095528471 state 0x0 ref 1 00000100:00100000:1.0:1789760787.578515:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 767095788887 time 767095528471 state 0x1 ref 1 00000100:00100000:1.0:1789760787.578552:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 767095788887 time 767142851128 state 0x0 ref 1 00000100:00100000:1.0:1789760787.578564:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4058380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512758656:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760787.578611:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28626944, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0xe007 00010000:00010000:1.0:1789760787.578621:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c30045 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.578636:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=767s v=(0 0) 00010000:00010000:1.0:1789760787.578641:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760787.578647:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c30045 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28626944->28631039] (req 28626944->28631039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.578661:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c30045 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28626944->28631039] (req 28626944->28631039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760787.611583:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974308 is committed 00002000:00010000:1.0:1789760787.612336:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28626944 -> 28631040 00010000:00010000:1.0:1789760787.612352:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c30045 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28626944->28631039] (req 28626944->28631039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.612364:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c30045 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28626944->28631039] (req 28626944->28631039) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.612373:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c30045 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28626944->28631039] (req 28626944->28631039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.612380:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760787.612386:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c30045 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28626944->28631039] (req 28626944->28631039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.612395:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c30045 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28626944->28631039] (req 28626944->28631039) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.612413:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c30045 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28626944->28631039] (req 28626944->28631039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.612420:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c30045 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28626944->28631039] (req 28626944->28631039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760787.612553:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4058380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512758656:12345-192.168.203.56@tcp:4:dd.0 Request processed in 33987us (34211us total) trans 4294974308 rc 0/0 00000100:00100000:1.0:1789760787.612564:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7019 00000100:00100000:1.0:1789760787.612574:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 767095788887 time 767142851128 state 0x2 ref 0 00000100:00100000:2.0:1789760787.629652:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760787.629801:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512758784 00000100:00100000:0.0:1789760787.629811:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512758784 00000100:00100000:0.0:1789760787.629863:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 767095788887 time 767142851128 state 0x0 ref 1 00000100:00100000:0.0:1789760787.629878:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 767143111544 time 767142851128 state 0x1 ref 1 00000100:00100000:0.0:1789760787.629910:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 767143111544 time 767194209093 state 0x0 ref 1 00000100:00100000:0.0:1789760787.629922:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4058a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512758784:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760787.629967:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28631040, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0xe007 00010000:00010000:0.0:1789760787.629975:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c3004c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.629999:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=767s v=(0 0) 00010000:00010000:0.0:1789760787.630003:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760787.630010:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c3004c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28631040->28635135] (req 28631040->28635135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.630025:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c3004c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28631040->28635135] (req 28631040->28635135) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760787.693605:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974309 is committed 00002000:00010000:1.0:1789760787.693732:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28631040 -> 28635136 00010000:00010000:1.0:1789760787.693750:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c3004c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28631040->28635135] (req 28631040->28635135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.693762:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c3004c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28631040->28635135] (req 28631040->28635135) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.693771:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c3004c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28631040->28635135] (req 28631040->28635135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.693783:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760787.693787:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c3004c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28631040->28635135] (req 28631040->28635135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.693796:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c3004c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28631040->28635135] (req 28631040->28635135) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.693814:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c3004c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28631040->28635135] (req 28631040->28635135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.693821:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c3004c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28631040->28635135] (req 28631040->28635135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760787.694043:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4058a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512758784:12345-192.168.203.56@tcp:4:dd.0 Request processed in 64117us (64396us total) trans 4294974309 rc 0/0 00000100:00100000:1.0:1789760787.694052:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7020 00000100:00100000:1.0:1789760787.694061:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 767143111544 time 767194209093 state 0x2 ref 0 00000100:00100000:1.0:1789760787.701199:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760787.701362:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512758912 00000100:00100000:0.0:1789760787.701371:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512758912 00000100:00100000:0.0:1789760787.701425:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 767143111544 time 767194209093 state 0x0 ref 1 00000100:00100000:0.0:1789760787.701440:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 767194469509 time 767194209093 state 0x1 ref 1 00000100:00100000:0.0:1789760787.701471:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 767194469509 time 767265770562 state 0x0 ref 1 00000100:00100000:0.0:1789760787.701483:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febffb9ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512758912:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760787.701532:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28635136, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0xe007 00010000:00010000:0.0:1789760787.701541:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c30053 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.701557:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=767s v=(0 0) 00010000:00010000:0.0:1789760787.701561:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760787.701568:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c30053 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28635136->28639231] (req 28635136->28639231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.701581:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c30053 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28635136->28639231] (req 28635136->28639231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760787.742317:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28635136 -> 28639232 00010000:00010000:1.0:1789760787.742338:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c30053 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28635136->28639231] (req 28635136->28639231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.742351:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c30053 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28635136->28639231] (req 28635136->28639231) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.742368:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c30053 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28635136->28639231] (req 28635136->28639231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.742375:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760787.742391:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c30053 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28635136->28639231] (req 28635136->28639231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.742400:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c30053 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28635136->28639231] (req 28635136->28639231) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.742419:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c30053 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28635136->28639231] (req 28635136->28639231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.742427:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c30053 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28635136->28639231] (req 28635136->28639231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760787.742513:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febffb9ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2358:x1876699512758912:12345-192.168.203.56@tcp:4:dd.0 Request processed in 41027us (41319us total) trans 4294974310 rc 0/0 00000100:00100000:1.0:1789760787.742522:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7021 00000100:00100000:1.0:1789760787.742531:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 767194469509 time 767265770562 state 0x2 ref 0 00000001:00080000:3.0:1789760787.743832:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974310 is committed 00000100:00100000:2.0:1789760787.745437:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760787.745499:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512759040 00000100:00100000:1.0:1789760787.745507:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512759040 00000100:00100000:1.0:1789760787.745555:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 767194469509 time 767265770562 state 0x0 ref 1 00000100:00100000:1.0:1789760787.745568:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 767266030978 time 767265770562 state 0x1 ref 1 00000100:00100000:1.0:1789760787.745596:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 767266030978 time 767309895861 state 0x0 ref 1 00000100:00100000:1.0:1789760787.745606:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4059500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512759040:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760787.745647:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28639232, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0xe007 00010000:00010000:1.0:1789760787.745655:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c3005a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.745679:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=767s v=(0 0) 00010000:00010000:1.0:1789760787.745683:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760787.745689:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c3005a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28639232->28643327] (req 28639232->28643327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.745727:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c3005a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28639232->28643327] (req 28639232->28643327) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760787.780986:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974311 is committed 00002000:00010000:3.0:1789760787.781090:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28639232 -> 28643328 00010000:00010000:3.0:1789760787.781106:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c3005a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28639232->28643327] (req 28639232->28643327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.781119:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c3005a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28639232->28643327] (req 28639232->28643327) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.781127:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c3005a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28639232->28643327] (req 28639232->28643327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.781139:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760787.781144:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c3005a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28639232->28643327] (req 28639232->28643327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.781154:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c3005a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28639232->28643327] (req 28639232->28643327) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.781174:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c3005a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28639232->28643327] (req 28639232->28643327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.781182:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c3005a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28639232->28643327] (req 28639232->28643327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760787.781283:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4059500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512759040:12345-192.168.203.56@tcp:4:dd.0 Request processed in 35673us (35849us total) trans 4294974311 rc 0/0 00000100:00100000:3.0:1789760787.781294:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7022 00000100:00100000:3.0:1789760787.781304:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 767266030978 time 767309895861 state 0x2 ref 0 00000100:00100000:0.0:1789760787.788332:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760787.788438:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512759168 00000100:00100000:1.0:1789760787.788448:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512759168 00000100:00100000:1.0:1789760787.788502:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 767266030978 time 767309895861 state 0x0 ref 1 00000100:00100000:1.0:1789760787.788517:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 767310156277 time 767309895861 state 0x1 ref 1 00000100:00100000:1.0:1789760787.788548:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 767310156277 time 767352848294 state 0x0 ref 1 00000100:00100000:1.0:1789760787.788560:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2fb8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512759168:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760787.788609:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28643328, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0xe007 00010000:00010000:1.0:1789760787.788617:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c30061 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.788632:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=767s v=(0 0) 00010000:00010000:1.0:1789760787.788636:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760787.788642:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c30061 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28643328->28647423] (req 28643328->28647423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.788656:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c30061 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28643328->28647423] (req 28643328->28647423) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760787.817428:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974312 is committed 00002000:00010000:3.0:1789760787.817574:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28643328 -> 28647424 00010000:00010000:3.0:1789760787.817590:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c30061 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28643328->28647423] (req 28643328->28647423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.817603:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c30061 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28643328->28647423] (req 28643328->28647423) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.817618:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c30061 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28643328->28647423] (req 28643328->28647423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.817625:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760787.817630:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c30061 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28643328->28647423] (req 28643328->28647423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.817639:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c30061 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28643328->28647423] (req 28643328->28647423) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.817668:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c30061 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28643328->28647423] (req 28643328->28647423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.817677:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c30061 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28643328->28647423] (req 28643328->28647423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760787.817801:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2fb8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512759168:12345-192.168.203.56@tcp:4:dd.0 Request processed in 29239us (29473us total) trans 4294974312 rc 0/0 00000100:00100000:3.0:1789760787.817811:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7023 00000100:00100000:3.0:1789760787.817821:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 767310156277 time 767352848294 state 0x2 ref 0 00000100:00100000:1.0:1789760787.820190:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760787.820275:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512759296 00000100:00100000:0.0:1789760787.820284:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512759296 00000100:00100000:0.0:1789760787.820331:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 767310156277 time 767352848294 state 0x0 ref 1 00000100:00100000:0.0:1789760787.820345:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 767353108710 time 767352848294 state 0x1 ref 1 00000100:00100000:0.0:1789760787.820416:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 767353108710 time 767384715439 state 0x0 ref 1 00000100:00100000:0.0:1789760787.820429:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febfb89ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512759296:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760787.820470:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28647424, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0xe007 00010000:00010000:0.0:1789760787.820478:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c30068 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.820494:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=767s v=(0 0) 00010000:00010000:0.0:1789760787.820498:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760787.820503:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c30068 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28647424->28651519] (req 28647424->28651519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.820516:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c30068 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28647424->28651519] (req 28647424->28651519) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.829983:0:5836:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:1.0:1789760787.852094:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974313 is committed 00002000:00010000:3.0:1789760787.853104:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28647424 -> 28651520 00010000:00010000:3.0:1789760787.853120:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c30068 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28647424->28651519] (req 28647424->28651519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.853140:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c30068 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28647424->28651519] (req 28647424->28651519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.853148:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c30068 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28647424->28651519] (req 28647424->28651519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.853155:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760787.853160:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c30068 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28647424->28651519] (req 28647424->28651519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.853169:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c30068 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28647424->28651519] (req 28647424->28651519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.853189:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c30068 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28647424->28651519] (req 28647424->28651519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.853197:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c30068 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28647424->28651519] (req 28647424->28651519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760787.853466:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febfb89ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512759296:12345-192.168.203.56@tcp:4:dd.0 Request processed in 33034us (33278us total) trans 4294974313 rc 0/0 00000100:00100000:3.0:1789760787.853487:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7024 00000100:00100000:3.0:1789760787.853497:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 767353108710 time 767384715439 state 0x2 ref 0 00000100:00100000:1.0:1789760787.858507:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760787.858640:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512759424 00000100:00100000:3.0:1789760787.858650:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512759424 00000100:00100000:3.0:1789760787.858967:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 767353108710 time 767384715439 state 0x0 ref 1 00000100:00100000:3.0:1789760787.858982:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 767384975855 time 767384715439 state 0x1 ref 1 00000100:00100000:3.0:1789760787.859120:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 767384975855 time 767423418768 state 0x0 ref 1 00000100:00100000:3.0:1789760787.859133:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febfb89fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512759424:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760787.859355:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28651520, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0xe007 00010000:00010000:3.0:1789760787.859365:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c3006f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.859381:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=767s v=(0 0) 00010000:00010000:3.0:1789760787.859386:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760787.859392:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c3006f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28651520->28655615] (req 28651520->28655615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.859405:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c3006f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28651520->28655615] (req 28651520->28655615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760787.894017:0:3312:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8feac2746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3312:1876699540233088:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1789760787.894167:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760787.894830:0:6707:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540233088 00000100:00100000:2.0:1789760787.894839:0:6707:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540233088 00000100:00100000:2.0:1789760787.894939:0:6707:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 215 00000100:00100000:2.0:1789760787.894952:0:6707:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac403bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3312:x1876699540233088:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1789760787.895000:0:6707:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3859808256 avail 3857711104 00000100:00100000:2.0:1789760787.895962:0:6707:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac403bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3312:x1876699540233088:12345-0@lo:13:osp-pre-1-0.0 Request processed in 1008us (1790us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760787.895972:0:6707:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 215 00000100:00100000:3.0:1789760787.896205:0:3312:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8feac2746300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3312:1876699540233088:0@lo:13:osp-pre-1-0.0 00002000:00010000:3.0:1789760787.905683:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28651520 -> 28655616 00010000:00010000:3.0:1789760787.905728:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c3006f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28651520->28655615] (req 28651520->28655615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.905742:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c3006f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28651520->28655615] (req 28651520->28655615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.905757:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c3006f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28651520->28655615] (req 28651520->28655615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.905765:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760787.905770:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c3006f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28651520->28655615] (req 28651520->28655615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.905780:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c3006f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28651520->28655615] (req 28651520->28655615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.905800:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c3006f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28651520->28655615] (req 28651520->28655615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.905808:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c3006f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28651520->28655615] (req 28651520->28655615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760787.905926:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febfb89fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2360:x1876699512759424:12345-192.168.203.56@tcp:4:dd.0 Request processed in 46789us (47422us total) trans 4294974314 rc 0/0 00000100:00100000:3.0:1789760787.905937:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7025 00000100:00100000:3.0:1789760787.905949:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 767384975855 time 767423418768 state 0x2 ref 0 00000001:00080000:1.0:1789760787.906873:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974314 is committed 00000100:00100000:1.0:1789760787.912020:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760787.916286:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512759552 00000100:00100000:3.0:1789760787.916301:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512759552 00000100:00100000:3.0:1789760787.916367:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 767384975855 time 767423418768 state 0x0 ref 1 00000100:00100000:3.0:1789760787.916391:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 767423679184 time 767423418768 state 0x1 ref 1 00000100:00100000:3.0:1789760787.916427:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 767423679184 time 767480725987 state 0x0 ref 1 00000100:00100000:3.0:1789760787.916440:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac403b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512759552:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760787.916490:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28655616, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0xe007 00010000:00010000:3.0:1789760787.916499:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c30076 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.916791:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=767s v=(0 0) 00010000:00010000:3.0:1789760787.916796:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760787.916802:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c30076 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28655616->28659711] (req 28655616->28659711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.916817:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c30076 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28655616->28659711] (req 28655616->28659711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760787.951271:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974315 is committed 00002000:00010000:3.0:1789760787.951446:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28655616 -> 28659712 00010000:00010000:3.0:1789760787.951461:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c30076 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28655616->28659711] (req 28655616->28659711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.951474:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c30076 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28655616->28659711] (req 28655616->28659711) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.951483:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c30076 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28655616->28659711] (req 28655616->28659711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.951490:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760787.951496:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c30076 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28655616->28659711] (req 28655616->28659711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.951558:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c30076 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28655616->28659711] (req 28655616->28659711) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.951579:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c30076 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28655616->28659711] (req 28655616->28659711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.951587:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c30076 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28655616->28659711] (req 28655616->28659711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760787.951802:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac403b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512759552:12345-192.168.203.56@tcp:4:dd.0 Request processed in 35360us (39786us total) trans 4294974315 rc 0/0 00000100:00100000:3.0:1789760787.951813:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7026 00000100:00100000:3.0:1789760787.951823:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 767423679184 time 767480725987 state 0x2 ref 0 00000100:00100000:0.0:1789760787.955328:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760787.955414:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512759680 00000100:00100000:3.0:1789760787.955425:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512759680 00000100:00100000:3.0:1789760787.955478:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 767423679184 time 767480725987 state 0x0 ref 1 00000100:00100000:3.0:1789760787.955496:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 767480986403 time 767480725987 state 0x1 ref 1 00000100:00100000:3.0:1789760787.955654:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 767480986403 time 767519952246 state 0x0 ref 1 00000100:00100000:3.0:1789760787.955679:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac38b1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512759680:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760787.955764:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28659712, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0xe007 00010000:00010000:3.0:1789760787.955774:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c3007d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.955790:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=767s v=(0 0) 00010000:00010000:3.0:1789760787.955795:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760787.955802:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c3007d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28659712->28663807] (req 28659712->28663807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.955816:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c3007d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28659712->28663807] (req 28659712->28663807) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760787.988089:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974316 is committed 00002000:00010000:1.0:1789760787.988230:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28659712 -> 28663808 00010000:00010000:1.0:1789760787.988245:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c3007d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28659712->28663807] (req 28659712->28663807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.988259:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c3007d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28659712->28663807] (req 28659712->28663807) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.988268:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c3007d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28659712->28663807] (req 28659712->28663807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.988276:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760787.988281:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c3007d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28659712->28663807] (req 28659712->28663807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.988290:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c3007d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28659712->28663807] (req 28659712->28663807) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.988308:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c3007d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28659712->28663807] (req 28659712->28663807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.988317:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c3007d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28659712->28663807] (req 28659712->28663807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760787.988437:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac38b1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512759680:12345-192.168.203.56@tcp:4:dd.0 Request processed in 32766us (33115us total) trans 4294974316 rc 0/0 00000100:00100000:1.0:1789760787.988447:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7027 00000100:00100000:1.0:1789760787.988456:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 767480986403 time 767519952246 state 0x2 ref 0 00000100:00100000:1.0:1789760787.991954:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760787.992044:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512759808 00000100:00100000:3.0:1789760787.992052:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512759808 00000100:00100000:3.0:1789760787.992170:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 767480986403 time 767519952246 state 0x0 ref 1 00000100:00100000:3.0:1789760787.992186:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 767520212662 time 767519952246 state 0x1 ref 1 00000100:00100000:3.0:1789760787.992250:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 767520212662 time 767556548691 state 0x0 ref 1 00000100:00100000:3.0:1789760787.992263:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4038380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512759808:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760787.992308:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28663808, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0xe007 00010000:00010000:3.0:1789760787.992316:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c30084 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.992332:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=767s v=(0 0) 00010000:00010000:3.0:1789760787.992337:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760787.992343:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c30084 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28663808->28667903] (req 28663808->28667903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.992358:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c30084 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28663808->28667903] (req 28663808->28667903) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760788.023775:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974317 is committed 00002000:00010000:3.0:1789760788.023919:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28663808 -> 28667904 00010000:00010000:3.0:1789760788.023936:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c30084 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28663808->28667903] (req 28663808->28667903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.023948:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c30084 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28663808->28667903] (req 28663808->28667903) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.023959:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c30084 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28663808->28667903] (req 28663808->28667903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.023967:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760788.023972:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c30084 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28663808->28667903] (req 28663808->28667903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.023981:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c30084 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28663808->28667903] (req 28663808->28667903) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.024010:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c30084 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28663808->28667903] (req 28663808->28667903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.024019:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c30084 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28663808->28667903] (req 28663808->28667903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760788.024116:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4038380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512759808:12345-192.168.203.56@tcp:4:dd.0 Request processed in 31851us (32165us total) trans 4294974317 rc 0/0 00000100:00100000:3.0:1789760788.024126:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7028 00000100:00100000:3.0:1789760788.024135:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 767520212662 time 767556548691 state 0x2 ref 0 00000100:00100000:2.0:1789760788.026637:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760788.026739:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512759936 00000100:00100000:3.0:1789760788.026750:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512759936 00000100:00100000:3.0:1789760788.026801:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 767520212662 time 767556548691 state 0x0 ref 1 00000100:00100000:3.0:1789760788.026815:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 767556809107 time 767556548691 state 0x1 ref 1 00000100:00100000:3.0:1789760788.026838:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 767556809107 time 767591138003 state 0x0 ref 1 00000100:00100000:3.0:1789760788.026850:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac405ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512759936:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760788.026895:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28667904, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0xe007 00010000:00010000:3.0:1789760788.026904:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c3008b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.026919:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=767s v=(0 0) 00010000:00010000:3.0:1789760788.026924:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760788.026930:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c3008b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28667904->28671999] (req 28667904->28671999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.026943:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c3008b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28667904->28671999] (req 28667904->28671999) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760788.056586:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974318 is committed 00002000:00010000:3.0:1789760788.056650:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28667904 -> 28672000 00010000:00010000:3.0:1789760788.056675:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c3008b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28667904->28671999] (req 28667904->28671999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.056688:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c3008b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28667904->28671999] (req 28667904->28671999) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.056721:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c3008b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28667904->28671999] (req 28667904->28671999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.056729:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760788.056733:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c3008b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28667904->28671999] (req 28667904->28671999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.056741:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c3008b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28667904->28671999] (req 28667904->28671999) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.056760:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c3008b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28667904->28671999] (req 28667904->28671999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.056773:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c3008b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28667904->28671999] (req 28667904->28671999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760788.056858:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac405ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512759936:12345-192.168.203.56@tcp:4:dd.0 Request processed in 30005us (30223us total) trans 4294974318 rc 0/0 00000100:00100000:3.0:1789760788.056868:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7029 00000100:00100000:3.0:1789760788.056878:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 767556809107 time 767591138003 state 0x2 ref 0 00000100:00100000:1.0:1789760788.060532:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760788.060625:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512760064 00000100:00100000:3.0:1789760788.060634:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512760064 00000100:00100000:3.0:1789760788.060762:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 767556809107 time 767591138003 state 0x0 ref 1 00000100:00100000:3.0:1789760788.060777:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 767591398419 time 767591138003 state 0x1 ref 1 00000100:00100000:3.0:1789760788.060849:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 767591398419 time 767625148153 state 0x0 ref 1 00000100:00100000:3.0:1789760788.060863:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4039500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512760064:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760788.060909:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28672000, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0xe007 00010000:00010000:3.0:1789760788.060917:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c30092 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.060930:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=767s v=(0 0) 00010000:00010000:3.0:1789760788.060935:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760788.060941:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c30092 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28672000->28676095] (req 28672000->28676095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.060955:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c30092 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28672000->28676095] (req 28672000->28676095) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760788.082838:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28672000 -> 28676096 00010000:00010000:3.0:1789760788.082858:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c30092 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28672000->28676095] (req 28672000->28676095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.082872:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c30092 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28672000->28676095] (req 28672000->28676095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.082882:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c30092 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28672000->28676095] (req 28672000->28676095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.082888:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760788.082892:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c30092 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28672000->28676095] (req 28672000->28676095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.082900:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c30092 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28672000->28676095] (req 28672000->28676095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.082918:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c30092 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28672000->28676095] (req 28672000->28676095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.082926:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c30092 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28672000->28676095] (req 28672000->28676095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760788.083022:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4039500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2360:x1876699512760064:12345-192.168.203.56@tcp:4:dd.0 Request processed in 22157us (22494us total) trans 4294974319 rc 0/0 00000100:00100000:3.0:1789760788.083033:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7030 00000100:00100000:3.0:1789760788.083043:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 767591398419 time 767625148153 state 0x2 ref 0 00000001:00080000:3.0:1789760788.083078:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974319 is committed 00000100:00100000:3.0:1789760788.085370:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760788.085505:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512760192 00000100:00100000:1.0:1789760788.085514:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512760192 00000100:00100000:1.0:1789760788.085563:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 767591398419 time 767625148153 state 0x0 ref 1 00000100:00100000:1.0:1789760788.085578:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 767625408569 time 767625148153 state 0x1 ref 1 00000100:00100000:1.0:1789760788.085676:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 767625408569 time 767649975247 state 0x0 ref 1 00000100:00100000:1.0:1789760788.085690:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2746300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512760192:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760788.085942:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28676096, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0xe007 00010000:00010000:1.0:1789760788.085952:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c30099 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.085967:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=767s v=(0 0) 00010000:00010000:1.0:1789760788.085971:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760788.085977:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c30099 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28676096->28680191] (req 28676096->28680191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.085998:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c30099 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28676096->28680191] (req 28676096->28680191) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760788.121336:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974320 is committed 00002000:00010000:3.0:1789760788.121422:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28676096 -> 28680192 00010000:00010000:3.0:1789760788.121436:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c30099 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28676096->28680191] (req 28676096->28680191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.121449:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c30099 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28676096->28680191] (req 28676096->28680191) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.121459:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c30099 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28676096->28680191] (req 28676096->28680191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.121466:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760788.121471:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c30099 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28676096->28680191] (req 28676096->28680191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.121481:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c30099 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28676096->28680191] (req 28676096->28680191) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.121499:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c30099 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28676096->28680191] (req 28676096->28680191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.121506:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c30099 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28676096->28680191] (req 28676096->28680191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760788.121590:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2746300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512760192:12345-192.168.203.56@tcp:4:dd.0 Request processed in 35898us (36222us total) trans 4294974320 rc 0/0 00000100:00100000:3.0:1789760788.121599:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7031 00000100:00100000:3.0:1789760788.121609:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 767625408569 time 767649975247 state 0x2 ref 0 00000100:00100000:2.0:1789760788.127821:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760788.127904:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512760320 00000100:00100000:3.0:1789760788.127915:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512760320 00000100:00100000:3.0:1789760788.127977:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 767625408569 time 767649975247 state 0x0 ref 1 00000100:00100000:3.0:1789760788.128001:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 767650235663 time 767649975247 state 0x1 ref 1 00000100:00100000:3.0:1789760788.128027:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 767650235663 time 767692327198 state 0x0 ref 1 00000100:00100000:3.0:1789760788.128047:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4058e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512760320:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760788.128094:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28680192, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0xe007 00010000:00010000:3.0:1789760788.128103:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c300a0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.128118:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=767s v=(0 0) 00010000:00010000:3.0:1789760788.128122:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760788.128127:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c300a0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28680192->28684287] (req 28680192->28684287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.128140:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c300a0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28680192->28684287] (req 28680192->28684287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760788.189101:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28680192 -> 28684288 00010000:00010000:0.0:1789760788.189122:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c300a0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28680192->28684287] (req 28680192->28684287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.189135:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c300a0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28680192->28684287] (req 28680192->28684287) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.189144:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c300a0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28680192->28684287] (req 28680192->28684287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.189153:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760788.189158:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c300a0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28680192->28684287] (req 28680192->28684287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.189167:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c300a0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28680192->28684287] (req 28680192->28684287) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.189184:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c300a0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28680192->28684287] (req 28680192->28684287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.189192:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c300a0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28680192->28684287] (req 28680192->28684287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760788.189279:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4058e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2360:x1876699512760320:12345-192.168.203.56@tcp:4:dd.0 Request processed in 61230us (61465us total) trans 4294974321 rc 0/0 00000100:00100000:0.0:1789760788.189290:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7032 00000100:00100000:0.0:1789760788.189300:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 767650235663 time 767692327198 state 0x2 ref 0 00000001:00080000:2.0:1789760788.190079:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974321 is committed 00000100:00100000:0.0:1789760788.196627:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760788.196730:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512760448 00000100:00100000:3.0:1789760788.196739:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512760448 00000100:00100000:3.0:1789760788.196790:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 767650235663 time 767692327198 state 0x0 ref 1 00000100:00100000:3.0:1789760788.196806:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 767692587614 time 767692327198 state 0x1 ref 1 00000100:00100000:3.0:1789760788.196825:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 767692587614 time 767761125354 state 0x0 ref 1 00000100:00100000:3.0:1789760788.196837:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac38b0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512760448:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760788.196883:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28684288, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0xe007 00010000:00010000:3.0:1789760788.196891:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c300a7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.196906:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=767s v=(0 0) 00010000:00010000:3.0:1789760788.196910:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760788.196916:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c300a7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28684288->28688383] (req 28684288->28688383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.196929:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c300a7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28684288->28688383] (req 28684288->28688383) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760788.232233:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974322 is committed 00002000:00010000:1.0:1789760788.232310:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28684288 -> 28688384 00010000:00010000:1.0:1789760788.232324:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c300a7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28684288->28688383] (req 28684288->28688383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.232337:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c300a7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28684288->28688383] (req 28684288->28688383) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.232352:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c300a7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28684288->28688383] (req 28684288->28688383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.232360:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760788.232365:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c300a7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28684288->28688383] (req 28684288->28688383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.232374:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c300a7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28684288->28688383] (req 28684288->28688383) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.232394:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c300a7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28684288->28688383] (req 28684288->28688383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.232402:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c300a7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28684288->28688383] (req 28684288->28688383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760788.232521:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac38b0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512760448:12345-192.168.203.56@tcp:4:dd.0 Request processed in 35681us (35900us total) trans 4294974322 rc 0/0 00000100:00100000:1.0:1789760788.232532:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7033 00000100:00100000:1.0:1789760788.232541:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 767692587614 time 767761125354 state 0x2 ref 0 00000100:00100000:1.0:1789760788.237515:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760788.237841:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512760576 00000100:00100000:3.0:1789760788.237852:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512760576 00000100:00100000:3.0:1789760788.237911:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 767692587614 time 767761125354 state 0x0 ref 1 00000100:00100000:3.0:1789760788.237925:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 767761385770 time 767761125354 state 0x1 ref 1 00000100:00100000:3.0:1789760788.237957:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 767761385770 time 767802255959 state 0x0 ref 1 00000100:00100000:3.0:1789760788.237968:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac403a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512760576:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760788.238299:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28688384, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0xe007 00010000:00010000:3.0:1789760788.238308:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c300ae lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.238324:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=767s v=(0 0) 00010000:00010000:3.0:1789760788.238329:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760788.238335:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c300ae lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28688384->28692479] (req 28688384->28692479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.238762:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c300ae lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28688384->28692479] (req 28688384->28692479) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760788.289620:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974323 is committed 00002000:00010000:3.0:1789760788.289820:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28688384 -> 28692480 00010000:00010000:3.0:1789760788.289835:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c300ae lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28688384->28692479] (req 28688384->28692479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.289849:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c300ae lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28688384->28692479] (req 28688384->28692479) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.289858:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c300ae lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28688384->28692479] (req 28688384->28692479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.289866:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760788.289871:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c300ae lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28688384->28692479] (req 28688384->28692479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.289880:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c300ae lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28688384->28692479] (req 28688384->28692479) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.289904:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c300ae lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28688384->28692479] (req 28688384->28692479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.289911:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c300ae lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28688384->28692479] (req 28688384->28692479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760788.290038:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac403a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512760576:12345-192.168.203.56@tcp:4:dd.0 Request processed in 52065us (52526us total) trans 4294974323 rc 0/0 00000100:00100000:3.0:1789760788.290049:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7034 00000100:00100000:3.0:1789760788.290058:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 767761385770 time 767802255959 state 0x2 ref 0 00000100:00100000:2.0:1789760788.292654:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760788.292829:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512760704 00000100:00100000:0.0:1789760788.292839:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512760704 00000100:00100000:0.0:1789760788.292890:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 767761385770 time 767802255959 state 0x0 ref 1 00000100:00100000:0.0:1789760788.292904:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 767802516375 time 767802255959 state 0x1 ref 1 00000100:00100000:0.0:1789760788.292974:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 767802516375 time 767857273189 state 0x0 ref 1 00000100:00100000:0.0:1789760788.292996:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4059180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512760704:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760788.293046:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28692480, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0xe007 00010000:00010000:0.0:1789760788.293054:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c300b5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.293070:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=767s v=(0 0) 00010000:00010000:0.0:1789760788.293074:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760788.293080:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c300b5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28692480->28696575] (req 28692480->28696575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.293093:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c300b5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28692480->28696575] (req 28692480->28696575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760788.318574:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28692480 -> 28696576 00010000:00010000:3.0:1789760788.318593:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c300b5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28692480->28696575] (req 28692480->28696575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.318605:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c300b5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28692480->28696575] (req 28692480->28696575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.318614:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c300b5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28692480->28696575] (req 28692480->28696575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.318621:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760788.318626:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c300b5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28692480->28696575] (req 28692480->28696575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.318641:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c300b5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28692480->28696575] (req 28692480->28696575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.318671:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c300b5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28692480->28696575] (req 28692480->28696575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.318680:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c300b5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28692480->28696575] (req 28692480->28696575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760788.318808:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4059180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2359:x1876699512760704:12345-192.168.203.56@tcp:4:dd.0 Request processed in 25818us (26158us total) trans 4294974324 rc 0/0 00000100:00100000:3.0:1789760788.318818:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7035 00000100:00100000:3.0:1789760788.318828:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 767802516375 time 767857273189 state 0x2 ref 0 00000001:00080000:3.0:1789760788.318964:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974324 is committed 00000100:00100000:0.0:1789760788.321639:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760788.322119:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512760832 00000100:00100000:3.0:1789760788.322129:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512760832 00000100:00100000:3.0:1789760788.322180:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 767802516375 time 767857273189 state 0x0 ref 1 00000100:00100000:3.0:1789760788.322201:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 767857533605 time 767857273189 state 0x1 ref 1 00000100:00100000:3.0:1789760788.322225:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 767857533605 time 767886524456 state 0x0 ref 1 00000100:00100000:3.0:1789760788.322237:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac38b2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512760832:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760788.322284:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28696576, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0xe007 00010000:00010000:3.0:1789760788.322294:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c300bc lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.322308:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=767s v=(0 0) 00010000:00010000:3.0:1789760788.322313:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760788.322319:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c300bc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28696576->28700671] (req 28696576->28700671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.322333:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c300bc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28696576->28700671] (req 28696576->28700671) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760788.351254:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28696576 -> 28700672 00010000:00010000:3.0:1789760788.351274:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c300bc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28696576->28700671] (req 28696576->28700671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.351287:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c300bc lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28696576->28700671] (req 28696576->28700671) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.351295:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c300bc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28696576->28700671] (req 28696576->28700671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.351303:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760788.351307:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c300bc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28696576->28700671] (req 28696576->28700671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.351317:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c300bc lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28696576->28700671] (req 28696576->28700671) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.351339:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c300bc lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28696576->28700671] (req 28696576->28700671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.351346:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c300bc lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28696576->28700671] (req 28696576->28700671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760788.351535:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac38b2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2358:x1876699512760832:12345-192.168.203.56@tcp:4:dd.0 Request processed in 29295us (29901us total) trans 4294974325 rc 0/0 00000100:00100000:3.0:1789760788.351545:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7036 00000100:00100000:3.0:1789760788.351554:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 767857533605 time 767886524456 state 0x2 ref 0 00000001:00080000:3.0:1789760788.351591:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974325 is committed 00000100:00100000:3.0:1789760788.354170:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760788.354253:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512760960 00000100:00100000:1.0:1789760788.354262:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512760960 00000100:00100000:1.0:1789760788.354317:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 767857533605 time 767886524456 state 0x0 ref 1 00000100:00100000:1.0:1789760788.354332:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 767886784872 time 767886524456 state 0x1 ref 1 00000100:00100000:1.0:1789760788.354408:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 767886784872 time 767918707275 state 0x0 ref 1 00000100:00100000:1.0:1789760788.354421:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febea48a450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512760960:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760788.354467:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28700672, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0xe007 00010000:00010000:1.0:1789760788.354476:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c300c3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.354492:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=767s v=(0 0) 00010000:00010000:1.0:1789760788.354497:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760788.354502:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c300c3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28700672->28704767] (req 28700672->28704767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.354516:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c300c3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28700672->28704767] (req 28700672->28704767) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760788.384344:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974326 is committed 00002000:00010000:3.0:1789760788.384500:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28700672 -> 28704768 00010000:00010000:3.0:1789760788.384515:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c300c3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28700672->28704767] (req 28700672->28704767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.384529:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c300c3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28700672->28704767] (req 28700672->28704767) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.384538:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c300c3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28700672->28704767] (req 28700672->28704767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.384546:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760788.384550:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c300c3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28700672->28704767] (req 28700672->28704767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.384558:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c300c3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28700672->28704767] (req 28700672->28704767) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.384575:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c300c3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28700672->28704767] (req 28700672->28704767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.384583:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c300c3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28700672->28704767] (req 28700672->28704767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760788.384822:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febea48a450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512760960:12345-192.168.203.56@tcp:4:dd.0 Request processed in 30399us (30656us total) trans 4294974326 rc 0/0 00000100:00100000:3.0:1789760788.384832:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7037 00000100:00100000:3.0:1789760788.384841:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 767886784872 time 767918707275 state 0x2 ref 0 00000100:00100000:3.0:1789760788.389827:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760788.389929:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512761088 00000100:00100000:0.0:1789760788.389940:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512761088 00000100:00100000:0.0:1789760788.390006:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 767886784872 time 767918707275 state 0x0 ref 1 00000100:00100000:0.0:1789760788.390022:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 767918967691 time 767918707275 state 0x1 ref 1 00000100:00100000:0.0:1789760788.390093:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 767918967691 time 767954391812 state 0x0 ref 1 00000100:00100000:0.0:1789760788.390106:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2746300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512761088:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760788.390157:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28704768, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0xe007 00010000:00010000:0.0:1789760788.390167:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c300ca lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.390189:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=767s v=(0 0) 00010000:00010000:0.0:1789760788.390194:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760788.390201:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c300ca lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28704768->28708863] (req 28704768->28708863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.390216:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c300ca lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28704768->28708863] (req 28704768->28708863) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760788.425050:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28704768 -> 28708864 00010000:00010000:0.0:1789760788.425071:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c300ca lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28704768->28708863] (req 28704768->28708863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.425084:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c300ca lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28704768->28708863] (req 28704768->28708863) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.425093:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c300ca lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28704768->28708863] (req 28704768->28708863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.425100:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760788.425105:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c300ca lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28704768->28708863] (req 28704768->28708863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.425114:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c300ca lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28704768->28708863] (req 28704768->28708863) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.425132:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c300ca lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28704768->28708863] (req 28704768->28708863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.425141:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c300ca lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28704768->28708863] (req 28704768->28708863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760788.425223:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2746300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2359:x1876699512761088:12345-192.168.203.56@tcp:4:dd.0 Request processed in 35115us (35403us total) trans 4294974327 rc 0/0 00000100:00100000:0.0:1789760788.425232:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7038 00000100:00100000:0.0:1789760788.425242:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 767918967691 time 767954391812 state 0x2 ref 0 00000001:00080000:2.0:1789760788.427786:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974327 is committed 00000100:00100000:0.0:1789760788.428082:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760788.428153:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512761216 00000100:00100000:3.0:1789760788.428163:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512761216 00000100:00100000:3.0:1789760788.428214:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 767918967691 time 767954391812 state 0x0 ref 1 00000100:00100000:3.0:1789760788.428229:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 767954652228 time 767954391812 state 0x1 ref 1 00000100:00100000:3.0:1789760788.428260:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 767954652228 time 767992559652 state 0x0 ref 1 00000100:00100000:3.0:1789760788.428272:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac38b1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512761216:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760788.428317:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28708864, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0xe007 00010000:00010000:3.0:1789760788.428326:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c300d1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.428342:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=767s v=(0 0) 00010000:00010000:3.0:1789760788.428346:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760788.428352:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c300d1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28708864->28712959] (req 28708864->28712959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.428364:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c300d1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28708864->28712959] (req 28708864->28712959) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760788.459474:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974328 is committed 00002000:00010000:3.0:1789760788.459603:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28708864 -> 28712960 00010000:00010000:3.0:1789760788.459619:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c300d1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28708864->28712959] (req 28708864->28712959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.459632:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c300d1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28708864->28712959] (req 28708864->28712959) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.459641:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c300d1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28708864->28712959] (req 28708864->28712959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.459649:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760788.459772:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c300d1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28708864->28712959] (req 28708864->28712959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.459785:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c300d1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28708864->28712959] (req 28708864->28712959) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.459814:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c300d1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28708864->28712959] (req 28708864->28712959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.459822:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c300d1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28708864->28712959] (req 28708864->28712959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760788.459947:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac38b1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512761216:12345-192.168.203.56@tcp:4:dd.0 Request processed in 31672us (31864us total) trans 4294974328 rc 0/0 00000100:00100000:3.0:1789760788.459958:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7039 00000100:00100000:3.0:1789760788.459967:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 767954652228 time 767992559652 state 0x2 ref 0 00000100:00100000:3.0:1789760788.463016:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760788.463116:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512761344 00000100:00100000:1.0:1789760788.463125:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512761344 00000100:00100000:1.0:1789760788.463179:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 767954652228 time 767992559652 state 0x0 ref 1 00000100:00100000:1.0:1789760788.463193:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 767992820068 time 767992559652 state 0x1 ref 1 00000100:00100000:1.0:1789760788.463267:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 767992820068 time 768027566307 state 0x0 ref 1 00000100:00100000:1.0:1789760788.463282:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2744a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512761344:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760788.463325:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28712960, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0xe007 00010000:00010000:1.0:1789760788.463333:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c300d8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.463348:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=768s v=(0 0) 00010000:00010000:1.0:1789760788.463352:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760788.463359:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c300d8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28712960->28717055] (req 28712960->28717055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.463373:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c300d8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28712960->28717055] (req 28712960->28717055) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760788.489810:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974329 is committed 00002000:00010000:1.0:1789760788.489872:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28712960 -> 28717056 00010000:00010000:1.0:1789760788.489886:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c300d8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28712960->28717055] (req 28712960->28717055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.489900:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c300d8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28712960->28717055] (req 28712960->28717055) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.489908:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c300d8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28712960->28717055] (req 28712960->28717055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.489916:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760788.489921:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c300d8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28712960->28717055] (req 28712960->28717055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.489930:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c300d8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28712960->28717055] (req 28712960->28717055) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.489949:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c300d8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28712960->28717055] (req 28712960->28717055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.489956:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c300d8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28712960->28717055] (req 28712960->28717055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760788.490056:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2744a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512761344:12345-192.168.203.56@tcp:4:dd.0 Request processed in 26772us (27044us total) trans 4294974329 rc 0/0 00000100:00100000:1.0:1789760788.490066:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7040 00000100:00100000:1.0:1789760788.490075:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 767992820068 time 768027566307 state 0x2 ref 0 00000100:00100000:0.0:1789760788.493030:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760788.493120:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512761472 00000100:00100000:2.0:1789760788.493129:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512761472 00000100:00100000:2.0:1789760788.493177:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 767992820068 time 768027566307 state 0x0 ref 1 00000100:00100000:2.0:1789760788.493191:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 768027826723 time 768027566307 state 0x1 ref 1 00000100:00100000:2.0:1789760788.493220:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 768027826723 time 768057520222 state 0x0 ref 1 00000100:00100000:2.0:1789760788.493233:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac38b0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512761472:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760788.493282:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28717056, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0xe007 00010000:00010000:2.0:1789760788.493291:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c300df lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.493307:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=768s v=(0 0) 00010000:00010000:2.0:1789760788.493311:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760788.493318:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c300df lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28717056->28721151] (req 28717056->28721151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.493332:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c300df lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28717056->28721151] (req 28717056->28721151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760788.517221:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974330 is committed 00002000:00010000:1.0:1789760788.517317:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28717056 -> 28721152 00002000:00010000:1.0:1789760788.517322:0:6709:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760787 -> 1789760788 00002000:00010000:1.0:1789760788.517325:0:6709:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760787 -> 1789760788 00010000:00010000:1.0:1789760788.517337:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c300df lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28717056->28721151] (req 28717056->28721151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.517350:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c300df lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28717056->28721151] (req 28717056->28721151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.517360:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c300df lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28717056->28721151] (req 28717056->28721151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.517373:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760788.517385:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c300df lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28717056->28721151] (req 28717056->28721151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.517394:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c300df lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28717056->28721151] (req 28717056->28721151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.517413:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c300df lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28717056->28721151] (req 28717056->28721151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.517420:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c300df lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28717056->28721151] (req 28717056->28721151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760788.517512:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac38b0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512761472:12345-192.168.203.56@tcp:4:dd.0 Request processed in 24277us (24487us total) trans 4294974330 rc 0/0 00000100:00100000:1.0:1789760788.517522:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7041 00000100:00100000:1.0:1789760788.517531:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 768027826723 time 768057520222 state 0x2 ref 0 00000100:00100000:0.0:1789760788.520251:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760788.520325:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512761600 00000100:00100000:3.0:1789760788.520334:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512761600 00000100:00100000:3.0:1789760788.520447:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 768027826723 time 768057520222 state 0x0 ref 1 00000100:00100000:3.0:1789760788.520463:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 768057780638 time 768057520222 state 0x1 ref 1 00000100:00100000:3.0:1789760788.520532:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 768057780638 time 768084830618 state 0x0 ref 1 00000100:00100000:3.0:1789760788.520546:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac38b2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512761600:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760788.520587:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28721152, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0xe007 00010000:00010000:3.0:1789760788.520596:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c300e6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.520616:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=768s v=(0 0) 00010000:00010000:3.0:1789760788.520620:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760788.520627:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c300e6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28721152->28725247] (req 28721152->28725247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.520640:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c300e6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28721152->28725247] (req 28721152->28725247) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760788.548613:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974331 is committed 00002000:00010000:3.0:1789760788.548739:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28721152 -> 28725248 00010000:00010000:3.0:1789760788.548755:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c300e6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28721152->28725247] (req 28721152->28725247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.548768:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c300e6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28721152->28725247] (req 28721152->28725247) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.548776:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c300e6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28721152->28725247] (req 28721152->28725247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.548783:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760788.548788:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c300e6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28721152->28725247] (req 28721152->28725247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.548797:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c300e6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28721152->28725247] (req 28721152->28725247) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.548815:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c300e6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28721152->28725247] (req 28721152->28725247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.548823:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c300e6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28721152->28725247] (req 28721152->28725247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760788.548903:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac38b2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512761600:12345-192.168.203.56@tcp:4:dd.0 Request processed in 28355us (28654us total) trans 4294974331 rc 0/0 00000100:00100000:3.0:1789760788.548914:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7042 00000100:00100000:3.0:1789760788.548924:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 768057780638 time 768084830618 state 0x2 ref 0 00000100:00100000:0.0:1789760788.551493:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760788.551538:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512761728 00000100:00100000:1.0:1789760788.551547:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512761728 00000100:00100000:1.0:1789760788.551590:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 768057780638 time 768084830618 state 0x0 ref 1 00000100:00100000:1.0:1789760788.551601:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 768085091034 time 768084830618 state 0x1 ref 1 00000100:00100000:1.0:1789760788.551624:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 768085091034 time 768115923772 state 0x0 ref 1 00000100:00100000:1.0:1789760788.551637:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac3351180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512761728:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760788.551690:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28725248, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0xe007 00010000:00010000:1.0:1789760788.551826:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c300ed lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.551839:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=768s v=(0 0) 00010000:00010000:1.0:1789760788.551842:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760788.551847:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c300ed lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28725248->28729343] (req 28725248->28729343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.551857:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c300ed lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28725248->28729343] (req 28725248->28729343) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760788.579566:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28725248 -> 28729344 00010000:00010000:3.0:1789760788.579585:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c300ed lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28725248->28729343] (req 28725248->28729343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.579598:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c300ed lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28725248->28729343] (req 28725248->28729343) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.579606:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c300ed lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28725248->28729343] (req 28725248->28729343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.579613:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760788.579618:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c300ed lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28725248->28729343] (req 28725248->28729343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.579627:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c300ed lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28725248->28729343] (req 28725248->28729343) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.579650:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c300ed lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28725248->28729343] (req 28725248->28729343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.579667:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c300ed lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28725248->28729343] (req 28725248->28729343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760788.579784:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac3351180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2359:x1876699512761728:12345-192.168.203.56@tcp:4:dd.0 Request processed in 28144us (28292us total) trans 4294974332 rc 0/0 00000100:00100000:3.0:1789760788.579794:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7043 00000100:00100000:3.0:1789760788.579804:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 768085091034 time 768115923772 state 0x2 ref 0 00000001:00080000:0.0:1789760788.582064:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974332 is committed 00000100:00100000:1.0:1789760788.582796:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760788.582883:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512761856 00000100:00100000:2.0:1789760788.582893:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512761856 00000100:00100000:2.0:1789760788.582940:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 768085091034 time 768115923772 state 0x0 ref 1 00000100:00100000:2.0:1789760788.582954:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 768116184188 time 768115923772 state 0x1 ref 1 00000100:00100000:2.0:1789760788.582996:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 768116184188 time 768147283234 state 0x0 ref 1 00000100:00100000:2.0:1789760788.583009:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4039c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512761856:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760788.583051:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28729344, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0xe007 00010000:00010000:2.0:1789760788.583060:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c300f4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.583074:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=768s v=(0 0) 00010000:00010000:2.0:1789760788.583078:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760788.583084:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c300f4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28729344->28733439] (req 28729344->28733439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.583095:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c300f4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28729344->28733439] (req 28729344->28733439) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760788.610046:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974333 is committed 00002000:00010000:0.0:1789760788.612000:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28729344 -> 28733440 00010000:00010000:0.0:1789760788.612021:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c300f4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28729344->28733439] (req 28729344->28733439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.612036:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c300f4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28729344->28733439] (req 28729344->28733439) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.612045:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c300f4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28729344->28733439] (req 28729344->28733439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.612052:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760788.612058:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c300f4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28729344->28733439] (req 28729344->28733439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.612067:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c300f4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28729344->28733439] (req 28729344->28733439) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.612087:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c300f4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28729344->28733439] (req 28729344->28733439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.612099:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c300f4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28729344->28733439] (req 28729344->28733439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760788.612263:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4039c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512761856:12345-192.168.203.56@tcp:4:dd.0 Request processed in 29252us (29468us total) trans 4294974333 rc 0/0 00000100:00100000:0.0:1789760788.612274:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7044 00000100:00100000:0.0:1789760788.612283:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 768116184188 time 768147283234 state 0x2 ref 0 00000100:00100000:2.0:1789760788.615077:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760788.617556:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512761984 00000100:00100000:3.0:1789760788.617571:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512761984 00000100:00100000:3.0:1789760788.617633:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 768116184188 time 768147283234 state 0x0 ref 1 00000100:00100000:3.0:1789760788.617648:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 768147543650 time 768147283234 state 0x1 ref 1 00000100:00100000:3.0:1789760788.617854:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 768147543650 time 768182152124 state 0x0 ref 1 00000100:00100000:3.0:1789760788.617868:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4059880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512761984:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760788.617912:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28733440, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0xe007 00010000:00010000:3.0:1789760788.617921:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c300fb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.617942:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=768s v=(0 0) 00010000:00010000:3.0:1789760788.617946:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760788.617952:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c300fb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28733440->28737535] (req 28733440->28737535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.617965:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c300fb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28733440->28737535] (req 28733440->28737535) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760788.650639:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974334 is committed 00002000:00010000:3.0:1789760788.650797:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28733440 -> 28737536 00010000:00010000:3.0:1789760788.650814:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c300fb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28733440->28737535] (req 28733440->28737535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.650827:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c300fb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28733440->28737535] (req 28733440->28737535) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.650835:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c300fb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28733440->28737535] (req 28733440->28737535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.650843:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760788.650847:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c300fb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28733440->28737535] (req 28733440->28737535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.650855:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c300fb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28733440->28737535] (req 28733440->28737535) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.650874:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c300fb lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28733440->28737535] (req 28733440->28737535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.650882:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c300fb lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28733440->28737535] (req 28733440->28737535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760788.651025:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4059880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512761984:12345-192.168.203.56@tcp:4:dd.0 Request processed in 33155us (35951us total) trans 4294974334 rc 0/0 00000100:00100000:3.0:1789760788.651036:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7045 00000100:00100000:3.0:1789760788.651045:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 768147543650 time 768182152124 state 0x2 ref 0 00000100:00100000:3.0:1789760788.654618:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760788.654686:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512762112 00000100:00100000:1.0:1789760788.654728:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512762112 00000100:00100000:1.0:1789760788.654782:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 768147543650 time 768182152124 state 0x0 ref 1 00000100:00100000:1.0:1789760788.654797:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 768182412540 time 768182152124 state 0x1 ref 1 00000100:00100000:1.0:1789760788.654818:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 768182412540 time 768219118685 state 0x0 ref 1 00000100:00100000:1.0:1789760788.654831:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2745f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512762112:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760788.654876:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28737536, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0xe007 00010000:00010000:1.0:1789760788.654885:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c30102 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.654901:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=768s v=(0 0) 00010000:00010000:1.0:1789760788.654905:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760788.654911:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c30102 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28737536->28741631] (req 28737536->28741631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.654924:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c30102 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28737536->28741631] (req 28737536->28741631) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760788.690233:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974335 is committed 00002000:00010000:0.0:1789760788.690332:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28737536 -> 28741632 00010000:00010000:0.0:1789760788.690347:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c30102 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28737536->28741631] (req 28737536->28741631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.690359:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c30102 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28737536->28741631] (req 28737536->28741631) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.690369:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c30102 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28737536->28741631] (req 28737536->28741631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.690381:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760788.690386:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c30102 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28737536->28741631] (req 28737536->28741631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.690394:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c30102 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28737536->28741631] (req 28737536->28741631) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.690415:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c30102 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28737536->28741631] (req 28737536->28741631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.690423:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c30102 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28737536->28741631] (req 28737536->28741631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760788.690511:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2745f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512762112:12345-192.168.203.56@tcp:4:dd.0 Request processed in 35677us (35897us total) trans 4294974335 rc 0/0 00000100:00100000:0.0:1789760788.690521:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7046 00000100:00100000:0.0:1789760788.690531:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 768182412540 time 768219118685 state 0x2 ref 0 00000100:00100000:3.0:1789760788.699533:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760788.699617:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512762240 00000100:00100000:0.0:1789760788.699626:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512762240 00000100:00100000:0.0:1789760788.699687:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 768182412540 time 768219118685 state 0x0 ref 1 00000100:00100000:0.0:1789760788.699720:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 768219379101 time 768219118685 state 0x1 ref 1 00000100:00100000:0.0:1789760788.699753:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 768219379101 time 768264052488 state 0x0 ref 1 00000100:00100000:0.0:1789760788.699769:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2745500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512762240:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760788.699819:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28741632, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0xe007 00010000:00010000:0.0:1789760788.699827:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c30109 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.699843:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=768s v=(0 0) 00010000:00010000:0.0:1789760788.699848:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760788.699856:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c30109 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28741632->28745727] (req 28741632->28745727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.699872:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c30109 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28741632->28745727] (req 28741632->28745727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760788.746874:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974336 is committed 00002000:00010000:0.0:1789760788.746977:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28741632 -> 28745728 00010000:00010000:0.0:1789760788.747000:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c30109 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28741632->28745727] (req 28741632->28745727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.747013:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c30109 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28741632->28745727] (req 28741632->28745727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.747022:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c30109 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28741632->28745727] (req 28741632->28745727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.747030:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760788.747034:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c30109 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28741632->28745727] (req 28741632->28745727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.747044:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c30109 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28741632->28745727] (req 28741632->28745727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.747062:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c30109 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28741632->28745727] (req 28741632->28745727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.747070:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c30109 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28741632->28745727] (req 28741632->28745727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760788.747170:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2745500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512762240:12345-192.168.203.56@tcp:4:dd.0 Request processed in 47398us (47641us total) trans 4294974336 rc 0/0 00000100:00100000:0.0:1789760788.747179:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7047 00000100:00100000:0.0:1789760788.747187:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 768219379101 time 768264052488 state 0x2 ref 0 00000100:00100000:2.0:1789760788.750447:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760788.751368:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512762368 00000100:00100000:3.0:1789760788.751439:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512762368 00000100:00100000:3.0:1789760788.751502:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 768219379101 time 768264052488 state 0x0 ref 1 00000100:00100000:3.0:1789760788.751517:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 768264312904 time 768264052488 state 0x1 ref 1 00000100:00100000:3.0:1789760788.751581:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 768264312904 time 768315880580 state 0x0 ref 1 00000100:00100000:3.0:1789760788.751594:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac405bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512762368:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760788.751643:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28745728, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0xe007 00010000:00010000:3.0:1789760788.751663:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c30110 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.751679:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=768s v=(0 0) 00010000:00010000:3.0:1789760788.751683:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760788.751689:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c30110 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28745728->28749823] (req 28745728->28749823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.751725:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c30110 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28745728->28749823] (req 28745728->28749823) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760788.785578:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28745728 -> 28749824 00010000:00010000:0.0:1789760788.785597:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c30110 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28745728->28749823] (req 28745728->28749823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.785611:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c30110 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28745728->28749823] (req 28745728->28749823) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.785620:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c30110 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28745728->28749823] (req 28745728->28749823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.785627:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760788.785632:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c30110 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28745728->28749823] (req 28745728->28749823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.785641:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c30110 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28745728->28749823] (req 28745728->28749823) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.786804:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c30110 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28745728->28749823] (req 28745728->28749823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.786832:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c30110 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28745728->28749823] (req 28745728->28749823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760788.786946:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac405bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2361:x1876699512762368:12345-192.168.203.56@tcp:4:dd.0 Request processed in 35346us (36499us total) trans 4294974337 rc 0/0 00000100:00100000:0.0:1789760788.786958:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7048 00000100:00100000:0.0:1789760788.786967:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 768264312904 time 768315880580 state 0x2 ref 0 00000001:00080000:1.0:1789760788.787419:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974337 is committed 00000100:00100000:3.0:1789760788.790522:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760788.790610:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512762496 00000100:00100000:0.0:1789760788.790621:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512762496 00000100:00100000:0.0:1789760788.790682:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 768264312904 time 768315880580 state 0x0 ref 1 00000100:00100000:0.0:1789760788.790728:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 768316140996 time 768315880580 state 0x1 ref 1 00000100:00100000:0.0:1789760788.790756:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 768316140996 time 768355056099 state 0x0 ref 1 00000100:00100000:0.0:1789760788.790769:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2744e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512762496:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760788.790817:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28749824, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0xe007 00010000:00010000:0.0:1789760788.790827:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c30117 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.790845:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=768s v=(0 0) 00010000:00010000:0.0:1789760788.790849:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760788.790856:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c30117 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28749824->28753919] (req 28749824->28753919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.790869:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c30117 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28749824->28753919] (req 28749824->28753919) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760788.819348:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974338 is committed 00002000:00010000:0.0:1789760788.819509:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28749824 -> 28753920 00010000:00010000:0.0:1789760788.819525:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c30117 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28749824->28753919] (req 28749824->28753919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.819538:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c30117 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28749824->28753919] (req 28749824->28753919) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.819547:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c30117 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28749824->28753919] (req 28749824->28753919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.819554:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760788.819558:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c30117 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28749824->28753919] (req 28749824->28753919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.819567:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c30117 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28749824->28753919] (req 28749824->28753919) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.819592:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c30117 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28749824->28753919] (req 28749824->28753919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.819599:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c30117 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28749824->28753919] (req 28749824->28753919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760788.819803:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2744e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512762496:12345-192.168.203.56@tcp:4:dd.0 Request processed in 29032us (29286us total) trans 4294974338 rc 0/0 00000100:00100000:0.0:1789760788.819814:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7049 00000100:00100000:0.0:1789760788.819824:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 768316140996 time 768355056099 state 0x2 ref 0 00000100:00100000:3.0:1789760788.822639:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760788.822791:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512762624 00000100:00100000:0.0:1789760788.822800:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512762624 00000100:00100000:0.0:1789760788.822854:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 768316140996 time 768355056099 state 0x0 ref 1 00000100:00100000:0.0:1789760788.822869:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 768355316515 time 768355056099 state 0x1 ref 1 00000100:00100000:0.0:1789760788.822893:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 768355316515 time 768387192940 state 0x0 ref 1 00000100:00100000:0.0:1789760788.822905:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac437df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512762624:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760788.822954:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28753920, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0xe007 00010000:00010000:0.0:1789760788.822962:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c3011e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.822978:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=768s v=(0 0) 00010000:00010000:0.0:1789760788.822996:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760788.823003:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c3011e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28753920->28758015] (req 28753920->28758015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.823017:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c3011e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28753920->28758015] (req 28753920->28758015) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.854712:0:5835:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:2.0:1789760788.889155:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974339 is committed 00002000:00010000:0.0:1789760788.889453:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28753920 -> 28758016 00010000:00010000:0.0:1789760788.889470:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c3011e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28753920->28758015] (req 28753920->28758015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.889483:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c3011e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28753920->28758015] (req 28753920->28758015) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.889492:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c3011e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28753920->28758015] (req 28753920->28758015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.889499:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760788.889503:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c3011e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28753920->28758015] (req 28753920->28758015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.889512:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c3011e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28753920->28758015] (req 28753920->28758015) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.890748:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c3011e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28753920->28758015] (req 28753920->28758015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.890768:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c3011e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28753920->28758015] (req 28753920->28758015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760788.890888:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac437df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512762624:12345-192.168.203.56@tcp:4:dd.0 Request processed in 67980us (68254us total) trans 4294974339 rc 0/0 00000100:00100000:0.0:1789760788.890900:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7050 00000100:00100000:0.0:1789760788.890910:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 768355316515 time 768387192940 state 0x2 ref 0 00000100:00100000:3.0:1789760788.893621:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760788.893752:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512762752 00000100:00100000:0.0:1789760788.893762:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512762752 00000100:00100000:0.0:1789760788.893819:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 768355316515 time 768387192940 state 0x0 ref 1 00000100:00100000:0.0:1789760788.893835:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 768387453356 time 768387192940 state 0x1 ref 1 00000100:00100000:0.0:1789760788.893859:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 768387453356 time 768458160126 state 0x0 ref 1 00000100:00100000:0.0:1789760788.893871:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac437e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512762752:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760788.893925:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28758016, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0xe007 00010000:00010000:0.0:1789760788.893934:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c30125 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.893951:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=768s v=(0 0) 00010000:00010000:0.0:1789760788.893955:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760788.893962:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c30125 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28758016->28762111] (req 28758016->28762111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.893975:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c30125 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28758016->28762111] (req 28758016->28762111) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760788.944148:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28758016 -> 28762112 00010000:00010000:3.0:1789760788.944168:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c30125 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28758016->28762111] (req 28758016->28762111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.944183:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c30125 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28758016->28762111] (req 28758016->28762111) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.944194:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c30125 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28758016->28762111] (req 28758016->28762111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.944203:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760788.944212:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c30125 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28758016->28762111] (req 28758016->28762111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.944222:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c30125 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28758016->28762111] (req 28758016->28762111) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.944242:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c30125 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28758016->28762111] (req 28758016->28762111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.944251:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c30125 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28758016->28762111] (req 28758016->28762111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760788.944351:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac437e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2360:x1876699512762752:12345-192.168.203.56@tcp:4:dd.0 Request processed in 50477us (50736us total) trans 4294974340 rc 0/0 00000100:00100000:3.0:1789760788.944363:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7051 00000100:00100000:3.0:1789760788.944373:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 768387453356 time 768458160126 state 0x2 ref 0 00000001:00080000:1.0:1789760788.944454:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974340 is committed 00000100:00100000:3.0:1789760788.946787:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760788.946831:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512762880 00000100:00100000:0.0:1789760788.946840:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512762880 00000100:00100000:0.0:1789760788.946894:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 768387453356 time 768458160126 state 0x0 ref 1 00000100:00100000:0.0:1789760788.946906:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 768458420542 time 768458160126 state 0x1 ref 1 00000100:00100000:0.0:1789760788.946928:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 768458420542 time 768511228147 state 0x0 ref 1 00000100:00100000:0.0:1789760788.946941:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac437f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512762880:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760788.947001:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28762112, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0xe007 00010000:00010000:0.0:1789760788.947010:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c3012c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.947025:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=768s v=(0 0) 00010000:00010000:0.0:1789760788.947030:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760788.947037:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c3012c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28762112->28766207] (req 28762112->28766207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.947050:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c3012c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28762112->28766207] (req 28762112->28766207) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760788.992525:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28762112 -> 28766208 00010000:00010000:3.0:1789760788.992545:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c3012c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28762112->28766207] (req 28762112->28766207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.992558:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c3012c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28762112->28766207] (req 28762112->28766207) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.992568:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c3012c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28762112->28766207] (req 28762112->28766207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760788.992574:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974341 is committed 00010000:00010000:3.0:1789760788.992578:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760788.992585:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c3012c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28762112->28766207] (req 28762112->28766207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.992595:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c3012c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28762112->28766207] (req 28762112->28766207) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.992622:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c3012c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28762112->28766207] (req 28762112->28766207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.992630:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c3012c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28762112->28766207] (req 28762112->28766207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760788.992774:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac437f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512762880:12345-192.168.203.56@tcp:4:dd.0 Request processed in 45830us (45989us total) trans 4294974341 rc 0/0 00000100:00100000:3.0:1789760788.992783:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7052 00000100:00100000:3.0:1789760788.992793:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 768458420542 time 768511228147 state 0x2 ref 0 00000100:00100000:3.0:1789760788.996796:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760788.996872:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512763008 00000100:00100000:0.0:1789760788.996881:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512763008 00000100:00100000:0.0:1789760788.996928:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 768458420542 time 768511228147 state 0x0 ref 1 00000100:00100000:0.0:1789760788.996943:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 768511488563 time 768511228147 state 0x1 ref 1 00000100:00100000:0.0:1789760788.996996:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 768511488563 time 768561295592 state 0x0 ref 1 00000100:00100000:0.0:1789760788.997009:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac437d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512763008:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760788.997056:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28766208, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0xe007 00010000:00010000:0.0:1789760788.997065:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c30133 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.997079:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=768s v=(0 0) 00010000:00010000:0.0:1789760788.997083:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760788.997090:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c30133 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28766208->28770303] (req 28766208->28770303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.997104:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c30133 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28766208->28770303] (req 28766208->28770303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760789.052783:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974342 is committed 00002000:00010000:2.0:1789760789.052881:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28766208 -> 28770304 00010000:00010000:2.0:1789760789.052897:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c30133 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28766208->28770303] (req 28766208->28770303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.052911:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c30133 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28766208->28770303] (req 28766208->28770303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.052921:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c30133 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28766208->28770303] (req 28766208->28770303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.052928:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760789.052934:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c30133 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28766208->28770303] (req 28766208->28770303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.052943:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c30133 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28766208->28770303] (req 28766208->28770303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.052968:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c30133 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28766208->28770303] (req 28766208->28770303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.052976:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d001714a/0xe6aa143190c30133 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28766208->28770303] (req 28766208->28770303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760789.053073:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac437d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512763008:12345-192.168.203.56@tcp:4:dd.0 Request processed in 56062us (56282us total) trans 4294974342 rc 0/0 00000100:00100000:2.0:1789760789.053083:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7053 00000100:00100000:2.0:1789760789.053093:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 768511488563 time 768561295592 state 0x2 ref 0 00000100:00100000:0.0:1789760789.056128:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760789.056216:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512763136 00000100:00100000:3.0:1789760789.056226:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512763136 00000100:00100000:3.0:1789760789.056296:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 768511488563 time 768561295592 state 0x0 ref 1 00000100:00100000:3.0:1789760789.056312:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 768561556008 time 768561295592 state 0x1 ref 1 00000100:00100000:3.0:1789760789.056343:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 768561556008 time 768620643277 state 0x0 ref 1 00000100:00100000:3.0:1789760789.056358:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac3352a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512763136:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760789.056411:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28770304, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0xe007 00010000:00010000:3.0:1789760789.056420:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c3013a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.056439:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=768s v=(0 0) 00010000:00010000:3.0:1789760789.056444:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760789.056449:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c3013a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28770304->28774399] (req 28770304->28774399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.056462:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c3013a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28770304->28774399] (req 28770304->28774399) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:1.0:1789760789.057066:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:0.0:1789760789.125004:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974343 is committed 00002000:00010000:3.0:1789760789.125117:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28770304 -> 28774400 00010000:00010000:3.0:1789760789.125134:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c3013a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28770304->28774399] (req 28770304->28774399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.125158:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c3013a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28770304->28774399] (req 28770304->28774399) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.125169:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c3013a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28770304->28774399] (req 28770304->28774399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.125176:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760789.125181:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c3013a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28770304->28774399] (req 28770304->28774399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.125191:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c3013a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28770304->28774399] (req 28770304->28774399) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.125211:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c3013a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28770304->28774399] (req 28770304->28774399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.125220:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c3013a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28770304->28774399] (req 28770304->28774399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760789.125317:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac3352a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512763136:12345-192.168.203.56@tcp:4:dd.0 Request processed in 68959us (69193us total) trans 4294974343 rc 0/0 00000100:00100000:3.0:1789760789.125329:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7054 00000100:00100000:3.0:1789760789.125339:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 768561556008 time 768620643277 state 0x2 ref 0 00000100:00100000:3.0:1789760789.131402:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760789.131824:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512763264 00000100:00100000:0.0:1789760789.131835:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512763264 00000100:00100000:0.0:1789760789.131893:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 768561556008 time 768620643277 state 0x0 ref 1 00000100:00100000:0.0:1789760789.131909:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 768620903693 time 768620643277 state 0x1 ref 1 00000100:00100000:0.0:1789760789.131945:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 768620903693 time 768696244589 state 0x0 ref 1 00000100:00100000:0.0:1789760789.131959:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac437c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512763264:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760789.132019:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28774400, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0xe007 00010000:00010000:0.0:1789760789.132028:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c30141 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.132045:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=768s v=(0 0) 00010000:00010000:0.0:1789760789.132050:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760789.132056:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c30141 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28774400->28778495] (req 28774400->28778495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.132070:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c30141 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28774400->28778495] (req 28774400->28778495) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760789.189959:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974344 is committed 00002000:00010000:0.0:1789760789.190806:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28774400 -> 28778496 00010000:00010000:0.0:1789760789.190824:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c30141 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28774400->28778495] (req 28774400->28778495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.190838:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c30141 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28774400->28778495] (req 28774400->28778495) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.190848:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c30141 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28774400->28778495] (req 28774400->28778495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.190856:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760789.190861:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c30141 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28774400->28778495] (req 28774400->28778495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.190871:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c30141 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28774400->28778495] (req 28774400->28778495) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.190892:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c30141 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28774400->28778495] (req 28774400->28778495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.190900:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c30141 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28774400->28778495] (req 28774400->28778495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760789.191002:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac437c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512763264:12345-192.168.203.56@tcp:4:dd.0 Request processed in 59040us (59605us total) trans 4294974344 rc 0/0 00000100:00100000:0.0:1789760789.191013:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7055 00000100:00100000:0.0:1789760789.191022:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 768620903693 time 768696244589 state 0x2 ref 0 00000100:00100000:0.0:1789760789.197172:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760789.197372:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512763392 00000100:00100000:2.0:1789760789.197382:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512763392 00000100:00100000:2.0:1789760789.197437:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 768620903693 time 768696244589 state 0x0 ref 1 00000100:00100000:2.0:1789760789.197453:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 768696505005 time 768696244589 state 0x1 ref 1 00000100:00100000:2.0:1789760789.197487:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 768696505005 time 768761786850 state 0x0 ref 1 00000100:00100000:2.0:1789760789.197501:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac3352300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512763392:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760789.197553:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28778496, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0xe007 00010000:00010000:2.0:1789760789.197564:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c30148 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.197581:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=768s v=(0 0) 00010000:00010000:2.0:1789760789.197586:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760789.197593:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c30148 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28778496->28782591] (req 28778496->28782591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.197607:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c30148 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28778496->28782591] (req 28778496->28782591) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760789.284628:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974345 is committed 00002000:00010000:0.0:1789760789.284759:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28778496 -> 28782592 00010000:00010000:0.0:1789760789.284775:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c30148 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28778496->28782591] (req 28778496->28782591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.284787:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c30148 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28778496->28782591] (req 28778496->28782591) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.284796:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c30148 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28778496->28782591] (req 28778496->28782591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.284803:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760789.284808:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c30148 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28778496->28782591] (req 28778496->28782591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.284821:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c30148 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28778496->28782591] (req 28778496->28782591) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.284840:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c30148 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28778496->28782591] (req 28778496->28782591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.284847:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c30148 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28778496->28782591] (req 28778496->28782591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760789.285010:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac3352300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512763392:12345-192.168.203.56@tcp:4:dd.0 Request processed in 87506us (87843us total) trans 4294974345 rc 0/0 00000100:00100000:0.0:1789760789.285021:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7056 00000100:00100000:0.0:1789760789.285032:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 768696505005 time 768761786850 state 0x2 ref 0 00000100:00100000:1.0:1789760789.288472:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760789.289668:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512763520 00000100:00100000:3.0:1789760789.289681:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512763520 00000100:00100000:3.0:1789760789.289997:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 768696505005 time 768761786850 state 0x0 ref 1 00000100:00100000:3.0:1789760789.290014:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 768762047266 time 768761786850 state 0x1 ref 1 00000100:00100000:3.0:1789760789.290143:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 768762047266 time 768854441559 state 0x0 ref 1 00000100:00100000:3.0:1789760789.290159:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4038e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512763520:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760789.290210:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28782592, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0xe007 00010000:00010000:3.0:1789760789.290218:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c3014f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.290234:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=768s v=(0 0) 00010000:00010000:3.0:1789760789.290238:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760789.290245:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c3014f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28782592->28786687] (req 28782592->28786687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.290259:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c3014f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28782592->28786687] (req 28782592->28786687) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760789.336603:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974346 is committed 00002000:00010000:0.0:1789760789.336807:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28782592 -> 28786688 00010000:00010000:0.0:1789760789.336825:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c3014f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28782592->28786687] (req 28782592->28786687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.336837:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c3014f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28782592->28786687] (req 28782592->28786687) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.336846:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c3014f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28782592->28786687] (req 28782592->28786687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.336854:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760789.336860:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c3014f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28782592->28786687] (req 28782592->28786687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.336868:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c3014f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28782592->28786687] (req 28782592->28786687) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.336887:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c3014f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28782592->28786687] (req 28782592->28786687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.336895:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c3014f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28782592->28786687] (req 28782592->28786687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760789.340100:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4038e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512763520:12345-192.168.203.56@tcp:4:dd.0 Request processed in 49935us (51627us total) trans 4294974346 rc 0/0 00000100:00100000:0.0:1789760789.340115:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7057 00000100:00100000:0.0:1789760789.340126:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 768762047266 time 768854441559 state 0x2 ref 0 00000100:00100000:3.0:1789760789.340286:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760789.340462:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512763648 00000100:00100000:0.0:1789760789.340470:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512763648 00000100:00100000:0.0:1789760789.340514:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 768762047266 time 768854441559 state 0x0 ref 1 00000100:00100000:0.0:1789760789.340527:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 768854701975 time 768854441559 state 0x1 ref 1 00000100:00100000:0.0:1789760789.340556:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 768854701975 time 768904855925 state 0x0 ref 1 00000100:00100000:0.0:1789760789.340635:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac28a2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512763648:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760789.340693:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28786688, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0xe007 00010000:00010000:0.0:1789760789.340732:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c30156 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.340748:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=768s v=(0 0) 00010000:00010000:0.0:1789760789.340751:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760789.340757:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c30156 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28786688->28790783] (req 28786688->28790783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.340771:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c30156 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28786688->28790783] (req 28786688->28790783) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760789.374209:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28786688 -> 28790784 00010000:00010000:0.0:1789760789.374229:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c30156 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28786688->28790783] (req 28786688->28790783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.374242:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c30156 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28786688->28790783] (req 28786688->28790783) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.374251:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c30156 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28786688->28790783] (req 28786688->28790783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.374258:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760789.374263:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c30156 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28786688->28790783] (req 28786688->28790783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.374272:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c30156 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28786688->28790783] (req 28786688->28790783) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.374296:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c30156 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28786688->28790783] (req 28786688->28790783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.374304:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c30156 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28786688->28790783] (req 28786688->28790783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760789.374406:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac28a2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2359:x1876699512763648:12345-192.168.203.56@tcp:4:dd.0 Request processed in 33770us (34116us total) trans 4294974347 rc 0/0 00000100:00100000:0.0:1789760789.374416:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7058 00000100:00100000:0.0:1789760789.374427:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 768854701975 time 768904855925 state 0x2 ref 0 00000001:00080000:2.0:1789760789.375613:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974347 is committed 00000100:00100000:1.0:1789760789.378132:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760789.378220:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512763776 00000100:00100000:0.0:1789760789.378230:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512763776 00000100:00100000:0.0:1789760789.378287:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 768854701975 time 768904855925 state 0x0 ref 1 00000100:00100000:0.0:1789760789.378301:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 768905116341 time 768904855925 state 0x1 ref 1 00000100:00100000:0.0:1789760789.378365:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 768905116341 time 768942664028 state 0x0 ref 1 00000100:00100000:0.0:1789760789.378446:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac403b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512763776:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760789.378496:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28790784, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0xe007 00010000:00010000:0.0:1789760789.378505:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c3015d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.378521:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=768s v=(0 0) 00010000:00010000:0.0:1789760789.378526:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760789.378532:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c3015d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28790784->28794879] (req 28790784->28794879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.378545:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c3015d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28790784->28794879] (req 28790784->28794879) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760789.426067:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974348 is committed 00002000:00010000:2.0:1789760789.426166:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28790784 -> 28794880 00010000:00010000:2.0:1789760789.426181:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c3015d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28790784->28794879] (req 28790784->28794879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.426194:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c3015d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28790784->28794879] (req 28790784->28794879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.426202:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c3015d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28790784->28794879] (req 28790784->28794879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.426209:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760789.426214:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c3015d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28790784->28794879] (req 28790784->28794879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.426223:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c3015d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28790784->28794879] (req 28790784->28794879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.426242:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c3015d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28790784->28794879] (req 28790784->28794879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.426254:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c3015d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28790784->28794879] (req 28790784->28794879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760789.426381:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac403b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512763776:12345-192.168.203.56@tcp:4:dd.0 Request processed in 47933us (48253us total) trans 4294974348 rc 0/0 00000100:00100000:2.0:1789760789.426391:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7059 00000100:00100000:2.0:1789760789.426401:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 768905116341 time 768942664028 state 0x2 ref 0 00000100:00100000:1.0:1789760789.432599:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760789.432726:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512763904 00000100:00100000:0.0:1789760789.432736:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512763904 00000100:00100000:0.0:1789760789.433067:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 768905116341 time 768942664028 state 0x0 ref 1 00000100:00100000:0.0:1789760789.433086:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 768942924444 time 768942664028 state 0x1 ref 1 00000100:00100000:0.0:1789760789.433120:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 768942924444 time 768997419198 state 0x0 ref 1 00000100:00100000:0.0:1789760789.433134:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac403a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512763904:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760789.433193:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28794880, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0xe007 00010000:00010000:0.0:1789760789.433202:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c30164 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.433221:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=768s v=(0 0) 00010000:00010000:0.0:1789760789.433226:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760789.433233:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c30164 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28794880->28798975] (req 28794880->28798975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.433248:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c30164 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28794880->28798975] (req 28794880->28798975) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760789.484741:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974349 is committed 00002000:00010000:0.0:1789760789.484761:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28794880 -> 28798976 00010000:00010000:0.0:1789760789.484776:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c30164 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28794880->28798975] (req 28794880->28798975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.484876:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c30164 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28794880->28798975] (req 28794880->28798975) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.484886:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c30164 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28794880->28798975] (req 28794880->28798975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.484893:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760789.484897:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c30164 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28794880->28798975] (req 28794880->28798975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.484907:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c30164 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28794880->28798975] (req 28794880->28798975) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.484928:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c30164 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28794880->28798975] (req 28794880->28798975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.484936:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c30164 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28794880->28798975] (req 28794880->28798975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760789.485070:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac403a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512763904:12345-192.168.203.56@tcp:4:dd.0 Request processed in 51934us (52476us total) trans 4294974349 rc 0/0 00000100:00100000:0.0:1789760789.485084:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7060 00000100:00100000:0.0:1789760789.485094:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 768942924444 time 768997419198 state 0x2 ref 0 00000100:00100000:3.0:1789760789.488178:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760789.488259:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512764032 00000100:00100000:0.0:1789760789.488269:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512764032 00000100:00100000:0.0:1789760789.488332:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 768942924444 time 768997419198 state 0x0 ref 1 00000100:00100000:0.0:1789760789.488348:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 768997679614 time 768997419198 state 0x1 ref 1 00000100:00100000:0.0:1789760789.488494:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 768997679614 time 769052792381 state 0x0 ref 1 00000100:00100000:0.0:1789760789.488508:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac28a0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512764032:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760789.488561:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28798976, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0xe007 00010000:00010000:0.0:1789760789.488570:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c3016b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.488587:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=769s v=(0 0) 00010000:00010000:0.0:1789760789.488591:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760789.488597:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c3016b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28798976->28803071] (req 28798976->28803071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.488610:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c3016b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28798976->28803071] (req 28798976->28803071) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760789.522186:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974350 is committed 00002000:00010000:2.0:1789760789.522236:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28798976 -> 28803072 00002000:00010000:2.0:1789760789.522241:0:6710:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760788 -> 1789760789 00002000:00010000:2.0:1789760789.522244:0:6710:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760788 -> 1789760789 00010000:00010000:2.0:1789760789.522258:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c3016b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28798976->28803071] (req 28798976->28803071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.522271:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c3016b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28798976->28803071] (req 28798976->28803071) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.522285:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c3016b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28798976->28803071] (req 28798976->28803071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.522293:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760789.522298:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c3016b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28798976->28803071] (req 28798976->28803071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.522312:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c3016b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28798976->28803071] (req 28798976->28803071) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.522332:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c3016b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28798976->28803071] (req 28798976->28803071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.522340:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c3016b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28798976->28803071] (req 28798976->28803071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760789.522434:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac28a0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512764032:12345-192.168.203.56@tcp:4:dd.0 Request processed in 33923us (34261us total) trans 4294974350 rc 0/0 00000100:00100000:2.0:1789760789.522444:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7061 00000100:00100000:2.0:1789760789.522453:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 768997679614 time 769052792381 state 0x2 ref 0 00000100:00100000:1.0:1789760789.527324:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760789.527456:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512764160 00000100:00100000:2.0:1789760789.527466:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512764160 00000100:00100000:2.0:1789760789.527522:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 768997679614 time 769052792381 state 0x0 ref 1 00000100:00100000:2.0:1789760789.527537:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 769053052797 time 769052792381 state 0x1 ref 1 00000100:00100000:2.0:1789760789.527568:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 769053052797 time 769091867076 state 0x0 ref 1 00000100:00100000:2.0:1789760789.527581:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4038700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512764160:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760789.527631:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28803072, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0xe007 00010000:00010000:2.0:1789760789.527640:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c30172 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.527668:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=769s v=(0 0) 00010000:00010000:2.0:1789760789.527672:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760789.527678:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c30172 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28803072->28807167] (req 28803072->28807167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.527693:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c30172 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28803072->28807167] (req 28803072->28807167) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760789.557954:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28803072 -> 28807168 00010000:00010000:2.0:1789760789.557973:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c30172 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28803072->28807167] (req 28803072->28807167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.557997:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c30172 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28803072->28807167] (req 28803072->28807167) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.558005:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c30172 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28803072->28807167] (req 28803072->28807167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.558012:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760789.558016:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c30172 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28803072->28807167] (req 28803072->28807167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.558025:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c30172 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28803072->28807167] (req 28803072->28807167) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.558043:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c30172 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28803072->28807167] (req 28803072->28807167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.558051:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c30172 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28803072->28807167] (req 28803072->28807167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760789.558196:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974351 is committed 00000100:00100000:2.0:1789760789.558246:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4038700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512764160:12345-192.168.203.56@tcp:4:dd.0 Request processed in 30663us (30929us total) trans 4294974351 rc 0/0 00000100:00100000:2.0:1789760789.558256:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7062 00000100:00100000:2.0:1789760789.558266:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 769053052797 time 769091867076 state 0x2 ref 0 00000100:00100000:2.0:1789760789.561194:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760789.561259:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512764288 00000100:00100000:0.0:1789760789.561268:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512764288 00000100:00100000:0.0:1789760789.561322:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 769053052797 time 769091867076 state 0x0 ref 1 00000100:00100000:0.0:1789760789.561337:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 769092127492 time 769091867076 state 0x1 ref 1 00000100:00100000:0.0:1789760789.561369:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 769092127492 time 769125668517 state 0x0 ref 1 00000100:00100000:0.0:1789760789.561390:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4058380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512764288:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760789.561439:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28807168, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0xe007 00010000:00010000:0.0:1789760789.561448:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c30179 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.561475:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=769s v=(0 0) 00010000:00010000:0.0:1789760789.561480:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760789.561486:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c30179 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28807168->28811263] (req 28807168->28811263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.561501:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c30179 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28807168->28811263] (req 28807168->28811263) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760789.585598:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974352 is committed 00002000:00010000:2.0:1789760789.585925:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28807168 -> 28811264 00010000:00010000:2.0:1789760789.585942:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c30179 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28807168->28811263] (req 28807168->28811263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.585955:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c30179 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28807168->28811263] (req 28807168->28811263) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.585964:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c30179 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28807168->28811263] (req 28807168->28811263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.585973:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760789.585987:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c30179 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28807168->28811263] (req 28807168->28811263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.585996:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c30179 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28807168->28811263] (req 28807168->28811263) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.586021:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c30179 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28807168->28811263] (req 28807168->28811263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.586030:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c30179 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28807168->28811263] (req 28807168->28811263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760789.586123:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4058380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512764288:12345-192.168.203.56@tcp:4:dd.0 Request processed in 24730us (24933us total) trans 4294974352 rc 0/0 00000100:00100000:2.0:1789760789.586133:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7063 00000100:00100000:2.0:1789760789.586142:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 769092127492 time 769125668517 state 0x2 ref 0 00000100:00100000:1.0:1789760789.590062:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760789.590149:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512764416 00000100:00100000:2.0:1789760789.590159:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512764416 00000100:00100000:2.0:1789760789.590211:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 769092127492 time 769125668517 state 0x0 ref 1 00000100:00100000:2.0:1789760789.590226:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 769125928933 time 769125668517 state 0x1 ref 1 00000100:00100000:2.0:1789760789.590288:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 769125928933 time 769154587702 state 0x0 ref 1 00000100:00100000:2.0:1789760789.590300:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4039f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512764416:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760789.590349:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28811264, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0xe007 00010000:00010000:2.0:1789760789.590358:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c30180 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.590385:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=769s v=(0 0) 00010000:00010000:2.0:1789760789.590390:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760789.590397:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c30180 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28811264->28815359] (req 28811264->28815359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.590411:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c30180 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28811264->28815359] (req 28811264->28815359) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760789.619249:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28811264 -> 28815360 00010000:00010000:2.0:1789760789.619268:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c30180 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28811264->28815359] (req 28811264->28815359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.619282:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c30180 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28811264->28815359] (req 28811264->28815359) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.619291:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c30180 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28811264->28815359] (req 28811264->28815359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.619297:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760789.619302:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c30180 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28811264->28815359] (req 28811264->28815359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.619312:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c30180 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28811264->28815359] (req 28811264->28815359) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.619331:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c30180 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28811264->28815359] (req 28811264->28815359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.619338:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c30180 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28811264->28815359] (req 28811264->28815359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760789.619438:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4039f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2359:x1876699512764416:12345-192.168.203.56@tcp:4:dd.0 Request processed in 29135us (29381us total) trans 4294974353 rc 0/0 00000100:00100000:2.0:1789760789.619449:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7064 00000100:00100000:2.0:1789760789.619458:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 769125928933 time 769154587702 state 0x2 ref 0 00000001:00080000:0.0:1789760789.622067:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974353 is committed 00000100:00100000:1.0:1789760789.627946:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760789.628199:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512764544 00000100:00100000:0.0:1789760789.628209:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512764544 00000100:00100000:0.0:1789760789.628264:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 769125928933 time 769154587702 state 0x0 ref 1 00000100:00100000:0.0:1789760789.628279:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 769154848118 time 769154587702 state 0x1 ref 1 00000100:00100000:0.0:1789760789.628308:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 769154848118 time 769192607768 state 0x0 ref 1 00000100:00100000:0.0:1789760789.628324:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4038000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512764544:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760789.628489:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28815360, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0xe007 00010000:00010000:0.0:1789760789.628498:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c30187 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.628515:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=769s v=(0 0) 00010000:00010000:0.0:1789760789.628520:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760789.628526:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c30187 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28815360->28819455] (req 28815360->28819455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.628539:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c30187 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28815360->28819455] (req 28815360->28819455) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760789.657312:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28815360 -> 28819456 00010000:00010000:0.0:1789760789.657333:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c30187 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28815360->28819455] (req 28815360->28819455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.657345:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c30187 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28815360->28819455] (req 28815360->28819455) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.657354:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c30187 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28815360->28819455] (req 28815360->28819455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.657362:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760789.657442:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c30187 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28815360->28819455] (req 28815360->28819455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.657455:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c30187 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28815360->28819455] (req 28815360->28819455) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.657478:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c30187 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28815360->28819455] (req 28815360->28819455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.657487:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c30187 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28815360->28819455] (req 28815360->28819455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760789.657594:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4038000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2359:x1876699512764544:12345-192.168.203.56@tcp:4:dd.0 Request processed in 29267us (29651us total) trans 4294974354 rc 0/0 00000100:00100000:0.0:1789760789.657606:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7065 00000100:00100000:0.0:1789760789.657616:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 769154848118 time 769192607768 state 0x2 ref 0 00000001:00080000:1.0:1789760789.660229:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974354 is committed 00000100:00100000:0.0:1789760789.667719:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760789.667925:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512764672 00000100:00100000:3.0:1789760789.667935:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512764672 00000100:00100000:3.0:1789760789.668001:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 769154848118 time 769192607768 state 0x0 ref 1 00000100:00100000:3.0:1789760789.668025:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 769192868184 time 769192607768 state 0x1 ref 1 00000100:00100000:3.0:1789760789.668046:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 769192868184 time 769232345826 state 0x0 ref 1 00000100:00100000:3.0:1789760789.668059:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac3353800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512764672:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760789.668106:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28819456, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0xe007 00010000:00010000:3.0:1789760789.668115:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c3018e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.668130:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=769s v=(0 0) 00010000:00010000:3.0:1789760789.668135:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760789.668141:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c3018e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28819456->28823551] (req 28819456->28823551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.668155:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c3018e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28819456->28823551] (req 28819456->28823551) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760789.697141:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974355 is committed 00002000:00010000:2.0:1789760789.697259:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28819456 -> 28823552 00010000:00010000:2.0:1789760789.697275:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c3018e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28819456->28823551] (req 28819456->28823551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.697288:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c3018e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28819456->28823551] (req 28819456->28823551) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.697297:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c3018e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28819456->28823551] (req 28819456->28823551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.697303:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760789.697308:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c3018e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28819456->28823551] (req 28819456->28823551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.697316:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c3018e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28819456->28823551] (req 28819456->28823551) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.697334:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c3018e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28819456->28823551] (req 28819456->28823551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.697346:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c3018e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28819456->28823551] (req 28819456->28823551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760789.697547:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac3353800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512764672:12345-192.168.203.56@tcp:4:dd.0 Request processed in 29485us (29849us total) trans 4294974355 rc 0/0 00000100:00100000:2.0:1789760789.697559:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7066 00000100:00100000:2.0:1789760789.697568:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 769192868184 time 769232345826 state 0x2 ref 0 00000100:00100000:2.0:1789760789.700908:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760789.701016:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512764800 00000100:00100000:0.0:1789760789.701027:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512764800 00000100:00100000:0.0:1789760789.701081:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 769192868184 time 769232345826 state 0x0 ref 1 00000100:00100000:0.0:1789760789.701096:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 769232606242 time 769232345826 state 0x1 ref 1 00000100:00100000:0.0:1789760789.701165:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 769232606242 time 769265464053 state 0x0 ref 1 00000100:00100000:0.0:1789760789.701177:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4058a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512764800:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760789.701227:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28823552, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0xe007 00010000:00010000:0.0:1789760789.701237:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c30195 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.701253:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=769s v=(0 0) 00010000:00010000:0.0:1789760789.701258:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760789.701264:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c30195 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28823552->28827647] (req 28823552->28827647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.701276:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c30195 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28823552->28827647] (req 28823552->28827647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760789.736065:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974356 is committed 00002000:00010000:0.0:1789760789.736250:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28823552 -> 28827648 00010000:00010000:0.0:1789760789.736265:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c30195 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28823552->28827647] (req 28823552->28827647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.736278:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c30195 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28823552->28827647] (req 28823552->28827647) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.736286:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c30195 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28823552->28827647] (req 28823552->28827647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.736293:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760789.736298:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c30195 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28823552->28827647] (req 28823552->28827647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.736308:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c30195 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28823552->28827647] (req 28823552->28827647) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.736331:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c30195 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28823552->28827647] (req 28823552->28827647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.736339:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c30195 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28823552->28827647] (req 28823552->28827647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760789.736537:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4058a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512764800:12345-192.168.203.56@tcp:4:dd.0 Request processed in 35357us (35633us total) trans 4294974356 rc 0/0 00000100:00100000:0.0:1789760789.736547:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7067 00000100:00100000:0.0:1789760789.736556:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 769232606242 time 769265464053 state 0x2 ref 0 00000100:00100000:0.0:1789760789.740224:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760789.740309:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512764928 00000100:00100000:2.0:1789760789.740319:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512764928 00000100:00100000:2.0:1789760789.740434:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 769232606242 time 769265464053 state 0x0 ref 1 00000100:00100000:2.0:1789760789.740451:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 769265724469 time 769265464053 state 0x1 ref 1 00000100:00100000:2.0:1789760789.740519:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 769265724469 time 769304818420 state 0x0 ref 1 00000100:00100000:2.0:1789760789.740530:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febff932a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512764928:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760789.740577:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28827648, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0xe007 00010000:00010000:2.0:1789760789.740585:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c3019c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.740600:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=769s v=(0 0) 00010000:00010000:2.0:1789760789.740605:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760789.740611:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c3019c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28827648->28831743] (req 28827648->28831743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.740624:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c3019c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28827648->28831743] (req 28827648->28831743) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760789.774638:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974357 is committed 00002000:00010000:2.0:1789760789.774795:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28827648 -> 28831744 00010000:00010000:2.0:1789760789.774812:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c3019c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28827648->28831743] (req 28827648->28831743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.774827:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c3019c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28827648->28831743] (req 28827648->28831743) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.774837:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c3019c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28827648->28831743] (req 28827648->28831743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.774845:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760789.774850:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c3019c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28827648->28831743] (req 28827648->28831743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.774860:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c3019c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28827648->28831743] (req 28827648->28831743) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.774879:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c3019c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28827648->28831743] (req 28827648->28831743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.774887:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c3019c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28827648->28831743] (req 28827648->28831743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760789.774994:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febff932a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512764928:12345-192.168.203.56@tcp:4:dd.0 Request processed in 34460us (34773us total) trans 4294974357 rc 0/0 00000100:00100000:2.0:1789760789.775004:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7068 00000100:00100000:2.0:1789760789.775015:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 769265724469 time 769304818420 state 0x2 ref 0 00000100:00100000:2.0:1789760789.778148:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760789.778225:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512765056 00000100:00100000:3.0:1789760789.778234:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512765056 00000100:00100000:3.0:1789760789.778285:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 769265724469 time 769304818420 state 0x0 ref 1 00000100:00100000:3.0:1789760789.778299:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 769305078836 time 769304818420 state 0x1 ref 1 00000100:00100000:3.0:1789760789.778332:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 769305078836 time 769342631363 state 0x0 ref 1 00000100:00100000:3.0:1789760789.778344:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4059500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512765056:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760789.778504:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28831744, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0xe007 00010000:00010000:3.0:1789760789.778513:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c301a3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.778529:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=769s v=(0 0) 00010000:00010000:3.0:1789760789.778534:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760789.778540:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c301a3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28831744->28835839] (req 28831744->28835839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.778554:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c301a3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28831744->28835839] (req 28831744->28835839) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760789.809246:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974358 is committed 00002000:00010000:2.0:1789760789.809331:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28831744 -> 28835840 00010000:00010000:2.0:1789760789.809346:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c301a3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28831744->28835839] (req 28831744->28835839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.809360:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c301a3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28831744->28835839] (req 28831744->28835839) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.809379:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c301a3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28831744->28835839] (req 28831744->28835839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.809387:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760789.809392:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c301a3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28831744->28835839] (req 28831744->28835839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.809402:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c301a3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28831744->28835839] (req 28831744->28835839) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.809420:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c301a3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28831744->28835839] (req 28831744->28835839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.809434:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c301a3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28831744->28835839] (req 28831744->28835839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760789.809523:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4059500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512765056:12345-192.168.203.56@tcp:4:dd.0 Request processed in 31177us (31381us total) trans 4294974358 rc 0/0 00000100:00100000:2.0:1789760789.809531:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7069 00000100:00100000:2.0:1789760789.809538:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 769305078836 time 769342631363 state 0x2 ref 0 00000100:00100000:3.0:1789760789.811920:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760789.812073:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512765184 00000100:00100000:0.0:1789760789.812081:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512765184 00000100:00100000:0.0:1789760789.812128:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 769305078836 time 769342631363 state 0x0 ref 1 00000100:00100000:0.0:1789760789.812140:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 769342891779 time 769342631363 state 0x1 ref 1 00000100:00100000:0.0:1789760789.812163:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 769342891779 time 769376462361 state 0x0 ref 1 00000100:00100000:0.0:1789760789.812175:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac28a3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512765184:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760789.812221:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28835840, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0xe007 00010000:00010000:0.0:1789760789.812230:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c301aa lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.812246:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=769s v=(0 0) 00010000:00010000:0.0:1789760789.812250:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760789.812255:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c301aa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28835840->28839935] (req 28835840->28839935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.812264:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c301aa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28835840->28839935] (req 28835840->28839935) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760789.842027:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28835840 -> 28839936 00010000:00010000:1.0:1789760789.842046:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c301aa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28835840->28839935] (req 28835840->28839935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.842059:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c301aa lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28835840->28839935] (req 28835840->28839935) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.842068:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c301aa lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28835840->28839935] (req 28835840->28839935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.842534:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760789.842540:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c301aa lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28835840->28839935] (req 28835840->28839935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.842551:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c301aa lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28835840->28839935] (req 28835840->28839935) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.842571:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c301aa lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28835840->28839935] (req 28835840->28839935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.842580:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c301aa lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28835840->28839935] (req 28835840->28839935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760789.842679:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac28a3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2358:x1876699512765184:12345-192.168.203.56@tcp:4:dd.0 Request processed in 30500us (30761us total) trans 4294974359 rc 0/0 00000100:00100000:1.0:1789760789.842692:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7070 00000100:00100000:1.0:1789760789.842747:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 769342891779 time 769376462361 state 0x2 ref 0 00000001:00080000:1.0:1789760789.842787:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974359 is committed 00000100:00100000:0.0:1789760789.848064:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760789.848555:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512765312 00000100:00100000:2.0:1789760789.848566:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512765312 00000100:00100000:2.0:1789760789.848625:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 769342891779 time 769376462361 state 0x0 ref 1 00000100:00100000:2.0:1789760789.848640:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 769376722777 time 769376462361 state 0x1 ref 1 00000100:00100000:2.0:1789760789.848679:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 769376722777 time 769412979054 state 0x0 ref 1 00000100:00100000:2.0:1789760789.848691:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febff930e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512765312:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760789.848768:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28839936, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0xe007 00010000:00010000:2.0:1789760789.848778:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c301b1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.848795:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=769s v=(0 0) 00010000:00010000:2.0:1789760789.848800:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760789.848806:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c301b1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28839936->28844031] (req 28839936->28844031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.848825:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c301b1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28839936->28844031] (req 28839936->28844031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.875577:0:7647:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:0.0:1789760789.912880:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974360 is committed 00002000:00010000:1.0:1789760789.913307:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28839936 -> 28844032 00010000:00010000:1.0:1789760789.913324:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c301b1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28839936->28844031] (req 28839936->28844031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.913337:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c301b1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28839936->28844031] (req 28839936->28844031) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.913347:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c301b1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28839936->28844031] (req 28839936->28844031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.913355:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760789.913359:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c301b1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28839936->28844031] (req 28839936->28844031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.913368:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c301b1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28839936->28844031] (req 28839936->28844031) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.913386:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c301b1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28839936->28844031] (req 28839936->28844031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.913394:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003fb4b75f/0xe6aa143190c301b1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28839936->28844031] (req 28839936->28844031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760789.915132:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febff930e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512765312:12345-192.168.203.56@tcp:4:dd.0 Request processed in 66433us (67068us total) trans 4294974360 rc 0/0 00000100:00100000:1.0:1789760789.915145:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7071 00000100:00100000:1.0:1789760789.915155:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 769376722777 time 769412979054 state 0x2 ref 0 00000100:00100000:2.0:1789760789.919355:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760789.919447:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512765440 00000100:00100000:3.0:1789760789.919467:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512765440 00000100:00100000:3.0:1789760789.919521:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 769376722777 time 769412979054 state 0x0 ref 1 00000100:00100000:3.0:1789760789.919538:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 769413239470 time 769412979054 state 0x1 ref 1 00000100:00100000:3.0:1789760789.919567:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 769413239470 time 769483867226 state 0x0 ref 1 00000100:00100000:3.0:1789760789.919580:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac405ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512765440:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760789.919639:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28844032, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0xe007 00010000:00010000:3.0:1789760789.919658:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c301b8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.919675:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=769s v=(0 0) 00010000:00010000:3.0:1789760789.919679:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760789.919686:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c301b8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28844032->28848127] (req 28844032->28848127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.919733:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c301b8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28844032->28848127] (req 28844032->28848127) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760789.968402:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974361 is committed 00002000:00010000:2.0:1789760789.969305:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28844032 -> 28848128 00010000:00010000:2.0:1789760789.969322:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c301b8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28844032->28848127] (req 28844032->28848127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.969335:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c301b8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28844032->28848127] (req 28844032->28848127) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.969344:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c301b8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28844032->28848127] (req 28844032->28848127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.969350:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760789.969356:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c301b8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28844032->28848127] (req 28844032->28848127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.969372:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c301b8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28844032->28848127] (req 28844032->28848127) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.969393:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c301b8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28844032->28848127] (req 28844032->28848127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.969402:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c301b8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28844032->28848127] (req 28844032->28848127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760789.969515:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac405ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512765440:12345-192.168.203.56@tcp:4:dd.0 Request processed in 49933us (50167us total) trans 4294974361 rc 0/0 00000100:00100000:2.0:1789760789.969526:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7072 00000100:00100000:2.0:1789760789.969535:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 769413239470 time 769483867226 state 0x2 ref 0 00000100:00100000:3.0:1789760789.973313:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760789.973387:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512765568 00000100:00100000:1.0:1789760789.973397:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512765568 00000100:00100000:1.0:1789760789.973447:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 769413239470 time 769483867226 state 0x0 ref 1 00000100:00100000:1.0:1789760789.973462:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 769484127642 time 769483867226 state 0x1 ref 1 00000100:00100000:1.0:1789760789.973541:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 769484127642 time 769537838847 state 0x0 ref 1 00000100:00100000:1.0:1789760789.973555:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febe7905f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512765568:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760789.973733:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28848128, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0xe007 00010000:00010000:1.0:1789760789.973743:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c301bf lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.973760:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=769s v=(0 0) 00010000:00010000:1.0:1789760789.973765:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760789.973771:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c301bf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28848128->28852223] (req 28848128->28852223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.973786:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c301bf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28848128->28852223] (req 28848128->28852223) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760790.016509:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28848128 -> 28852224 00010000:00010000:0.0:1789760790.016529:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c301bf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28848128->28852223] (req 28848128->28852223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.016542:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c301bf lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28848128->28852223] (req 28848128->28852223) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.016552:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c301bf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28848128->28852223] (req 28848128->28852223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.016560:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760790.016566:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c301bf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28848128->28852223] (req 28848128->28852223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.016575:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c301bf lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28848128->28852223] (req 28848128->28852223) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.016599:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c301bf lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28848128->28852223] (req 28848128->28852223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.016606:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c301bf lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28848128->28852223] (req 28848128->28852223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760790.016760:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febe7905f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2359:x1876699512765568:12345-192.168.203.56@tcp:4:dd.0 Request processed in 43203us (43451us total) trans 4294974362 rc 0/0 00000100:00100000:0.0:1789760790.016770:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7073 00000100:00100000:0.0:1789760790.016779:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 769484127642 time 769537838847 state 0x2 ref 0 00000001:00080000:3.0:1789760790.016880:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974362 is committed 00000100:00100000:0.0:1789760790.024398:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760790.024487:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512765696 00000100:00100000:1.0:1789760790.024497:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512765696 00000100:00100000:1.0:1789760790.024564:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 769484127642 time 769537838847 state 0x0 ref 1 00000100:00100000:1.0:1789760790.024579:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 769538099263 time 769537838847 state 0x1 ref 1 00000100:00100000:1.0:1789760790.024618:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 769538099263 time 769588910358 state 0x0 ref 1 00000100:00100000:1.0:1789760790.024630:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac38b6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512765696:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760790.024677:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28852224, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0xe007 00010000:00010000:1.0:1789760790.024687:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c301c6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.024726:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=769s v=(0 0) 00010000:00010000:1.0:1789760790.024731:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760790.024737:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c301c6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28852224->28856319] (req 28852224->28856319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.024752:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c301c6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28852224->28856319] (req 28852224->28856319) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760790.060884:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974363 is committed 00002000:00010000:1.0:1789760790.061003:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28852224 -> 28856320 00010000:00010000:1.0:1789760790.061019:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c301c6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28852224->28856319] (req 28852224->28856319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.061040:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c301c6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28852224->28856319] (req 28852224->28856319) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.061049:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c301c6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28852224->28856319] (req 28852224->28856319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.061056:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760790.061060:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c301c6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28852224->28856319] (req 28852224->28856319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.061069:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c301c6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28852224->28856319] (req 28852224->28856319) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.061097:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c301c6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28852224->28856319] (req 28852224->28856319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.061104:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c301c6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28852224->28856319] (req 28852224->28856319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760790.061194:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac38b6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512765696:12345-192.168.203.56@tcp:4:dd.0 Request processed in 36561us (36803us total) trans 4294974363 rc 0/0 00000100:00100000:1.0:1789760790.061204:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7074 00000100:00100000:1.0:1789760790.061214:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 769538099263 time 769588910358 state 0x2 ref 0 00000100:00100000:3.0:1789760790.065739:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760790.065891:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512765824 00000100:00100000:2.0:1789760790.065901:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512765824 00000100:00100000:2.0:1789760790.065957:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 769538099263 time 769588910358 state 0x0 ref 1 00000100:00100000:2.0:1789760790.066053:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 769589170774 time 769588910358 state 0x1 ref 1 00000100:00100000:2.0:1789760790.066092:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 769589170774 time 769630391210 state 0x0 ref 1 00000100:00100000:2.0:1789760790.066105:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febe7905880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512765824:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760790.066155:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28856320, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0xe007 00010000:00010000:2.0:1789760790.066164:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c301cd lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.066180:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=769s v=(0 0) 00010000:00010000:2.0:1789760790.066185:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760790.066191:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c301cd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28856320->28860415] (req 28856320->28860415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.066205:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c301cd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28856320->28860415] (req 28856320->28860415) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760790.108337:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974364 is committed 00002000:00010000:0.0:1789760790.108575:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28856320 -> 28860416 00010000:00010000:0.0:1789760790.108590:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c301cd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28856320->28860415] (req 28856320->28860415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.108602:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c301cd lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28856320->28860415] (req 28856320->28860415) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.108611:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c301cd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28856320->28860415] (req 28856320->28860415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.108619:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760790.108624:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c301cd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28856320->28860415] (req 28856320->28860415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.108639:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c301cd lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28856320->28860415] (req 28856320->28860415) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.108672:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c301cd lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28856320->28860415] (req 28856320->28860415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.108681:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c301cd lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28856320->28860415] (req 28856320->28860415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760790.108798:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febe7905880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512765824:12345-192.168.203.56@tcp:4:dd.0 Request processed in 42690us (43063us total) trans 4294974364 rc 0/0 00000100:00100000:0.0:1789760790.108808:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7075 00000100:00100000:0.0:1789760790.108818:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 769589170774 time 769630391210 state 0x2 ref 0 00000100:00100000:3.0:1789760790.112265:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760790.112451:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512765952 00000100:00100000:0.0:1789760790.112461:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512765952 00000100:00100000:0.0:1789760790.112519:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 769589170774 time 769630391210 state 0x0 ref 1 00000100:00100000:0.0:1789760790.112534:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 769630651626 time 769630391210 state 0x1 ref 1 00000100:00100000:0.0:1789760790.112565:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 769630651626 time 769676864896 state 0x0 ref 1 00000100:00100000:0.0:1789760790.112576:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febe7907100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512765952:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760790.112625:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28860416, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0xe007 00010000:00010000:0.0:1789760790.112634:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c301d4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.112663:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=769s v=(0 0) 00010000:00010000:0.0:1789760790.112668:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760790.112674:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c301d4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28860416->28864511] (req 28860416->28864511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.112688:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c301d4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28860416->28864511] (req 28860416->28864511) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760790.163216:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974365 is committed 00002000:00010000:2.0:1789760790.163402:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28860416 -> 28864512 00010000:00010000:2.0:1789760790.163416:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c301d4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28860416->28864511] (req 28860416->28864511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.163429:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c301d4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28860416->28864511] (req 28860416->28864511) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.163438:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c301d4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28860416->28864511] (req 28860416->28864511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.163445:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760790.163449:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c301d4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28860416->28864511] (req 28860416->28864511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.163458:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c301d4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28860416->28864511] (req 28860416->28864511) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.163479:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c301d4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28860416->28864511] (req 28860416->28864511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.163487:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c301d4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28860416->28864511] (req 28860416->28864511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760790.163574:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febe7907100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512765952:12345-192.168.203.56@tcp:4:dd.0 Request processed in 50995us (51315us total) trans 4294974365 rc 0/0 00000100:00100000:2.0:1789760790.163587:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7076 00000100:00100000:2.0:1789760790.163596:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 769630651626 time 769676864896 state 0x2 ref 0 00000100:00100000:3.0:1789760790.169081:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760790.169177:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512766080 00000100:00100000:1.0:1789760790.169188:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512766080 00000100:00100000:1.0:1789760790.169242:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 769630651626 time 769676864896 state 0x0 ref 1 00000100:00100000:1.0:1789760790.169258:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 769677125312 time 769676864896 state 0x1 ref 1 00000100:00100000:1.0:1789760790.169292:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 769677125312 time 769733592160 state 0x0 ref 1 00000100:00100000:1.0:1789760790.169305:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febe7906300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512766080:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760790.169352:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28864512, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0xe007 00010000:00010000:1.0:1789760790.169362:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c301db lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.169389:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=769s v=(0 0) 00010000:00010000:1.0:1789760790.169394:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760790.169400:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c301db lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28864512->28868607] (req 28864512->28868607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.169414:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c301db lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28864512->28868607] (req 28864512->28868607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760790.209785:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974366 is committed 00002000:00010000:0.0:1789760790.210285:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28864512 -> 28868608 00010000:00010000:0.0:1789760790.210301:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c301db lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28864512->28868607] (req 28864512->28868607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.210314:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c301db lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28864512->28868607] (req 28864512->28868607) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.210324:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c301db lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28864512->28868607] (req 28864512->28868607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.210332:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760790.210337:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c301db lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28864512->28868607] (req 28864512->28868607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.210346:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c301db lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28864512->28868607] (req 28864512->28868607) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.210364:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c301db lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28864512->28868607] (req 28864512->28868607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.210373:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c301db lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28864512->28868607] (req 28864512->28868607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760790.210460:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febe7906300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512766080:12345-192.168.203.56@tcp:4:dd.0 Request processed in 41153us (41384us total) trans 4294974366 rc 0/0 00000100:00100000:0.0:1789760790.210471:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7077 00000100:00100000:0.0:1789760790.210481:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 769677125312 time 769733592160 state 0x2 ref 0 00000100:00100000:3.0:1789760790.212771:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760790.212853:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512766208 00000100:00100000:0.0:1789760790.212862:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512766208 00000100:00100000:0.0:1789760790.212912:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 769677125312 time 769733592160 state 0x0 ref 1 00000100:00100000:0.0:1789760790.212927:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 769733852576 time 769733592160 state 0x1 ref 1 00000100:00100000:0.0:1789760790.212957:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 769733852576 time 769777256646 state 0x0 ref 1 00000100:00100000:0.0:1789760790.212979:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febe7904e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512766208:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760790.213024:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28868608, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0xe007 00010000:00010000:0.0:1789760790.213032:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c301e2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.213048:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=769s v=(0 0) 00010000:00010000:0.0:1789760790.213052:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760790.213058:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c301e2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28868608->28872703] (req 28868608->28872703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.213089:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c301e2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28868608->28872703] (req 28868608->28872703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760790.247495:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28868608 -> 28872704 00010000:00010000:0.0:1789760790.247513:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c301e2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28868608->28872703] (req 28868608->28872703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.247526:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c301e2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28868608->28872703] (req 28868608->28872703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.247535:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c301e2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28868608->28872703] (req 28868608->28872703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.247542:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760790.247547:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c301e2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28868608->28872703] (req 28868608->28872703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.247557:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c301e2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28868608->28872703] (req 28868608->28872703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.247576:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c301e2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28868608->28872703] (req 28868608->28872703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.247584:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c301e2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28868608->28872703] (req 28868608->28872703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760790.247681:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febe7904e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2361:x1876699512766208:12345-192.168.203.56@tcp:4:dd.0 Request processed in 34709us (34914us total) trans 4294974367 rc 0/0 00000100:00100000:0.0:1789760790.247690:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7078 00000100:00100000:0.0:1789760790.247730:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 769733852576 time 769777256646 state 0x2 ref 0 00000001:00080000:2.0:1789760790.248264:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974367 is committed 00000100:00100000:3.0:1789760790.250629:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760790.251150:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512766336 00000100:00100000:2.0:1789760790.251161:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512766336 00000100:00100000:3.0:1789760790.251294:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 769733852576 time 769777256646 state 0x0 ref 1 00000100:00100000:3.0:1789760790.251305:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 769777517062 time 769777256646 state 0x1 ref 1 00000100:00100000:3.0:1789760790.251355:0:6710:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 769777517062 time 769815655847 state 0x0 ref 1 00000100:00100000:3.0:1789760790.251373:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4132680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512766336:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760790.251423:0:6710:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28872704, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0xe007 00010000:00010000:3.0:1789760790.251432:0:6710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c301e9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.251449:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=769s v=(0 0) 00010000:00010000:3.0:1789760790.251453:0:6710:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760790.251459:0:6710:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c301e9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28872704->28876799] (req 28872704->28876799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.251478:0:6710:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c301e9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28872704->28876799] (req 28872704->28876799) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760790.286612:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974368 is committed 00002000:00010000:3.0:1789760790.286761:0:6710:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28872704 -> 28876800 00010000:00010000:3.0:1789760790.286778:0:6710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c301e9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28872704->28876799] (req 28872704->28876799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.286792:0:6710:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c301e9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28872704->28876799] (req 28872704->28876799) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.286807:0:6710:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c301e9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28872704->28876799] (req 28872704->28876799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.286815:0:6710:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760790.286819:0:6710:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c301e9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28872704->28876799] (req 28872704->28876799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.286828:0:6710:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c301e9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28872704->28876799] (req 28872704->28876799) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.286847:0:6710:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c301e9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28872704->28876799] (req 28872704->28876799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.286856:0:6710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c301e9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28872704->28876799] (req 28872704->28876799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760790.286936:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4132680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512766336:12345-192.168.203.56@tcp:4:dd.0 Request processed in 35568us (36313us total) trans 4294974368 rc 0/0 00000100:00100000:3.0:1789760790.286946:0:6710:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7079 00000100:00100000:3.0:1789760790.286956:0:6710:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 769777517062 time 769815655847 state 0x2 ref 0 00000100:00100000:3.0:1789760790.290889:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760790.290997:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512766464 00000100:00100000:0.0:1789760790.291006:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512766464 00000100:00100000:0.0:1789760790.291058:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 769777517062 time 769815655847 state 0x0 ref 1 00000100:00100000:0.0:1789760790.291139:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 769815916263 time 769815655847 state 0x1 ref 1 00000100:00100000:0.0:1789760790.291216:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 769815916263 time 769855515239 state 0x0 ref 1 00000100:00100000:0.0:1789760790.291230:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4130000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512766464:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760790.291275:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28876800, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0xe007 00010000:00010000:0.0:1789760790.291284:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c301f0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.291300:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=769s v=(0 0) 00010000:00010000:0.0:1789760790.291305:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760790.291311:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c301f0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28876800->28880895] (req 28876800->28880895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.291326:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c301f0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28876800->28880895] (req 28876800->28880895) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760790.317986:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974369 is committed 00002000:00010000:0.0:1789760790.318047:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28876800 -> 28880896 00010000:00010000:0.0:1789760790.318062:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c301f0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28876800->28880895] (req 28876800->28880895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.318160:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c301f0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28876800->28880895] (req 28876800->28880895) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.318171:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c301f0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28876800->28880895] (req 28876800->28880895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.318179:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760790.318184:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c301f0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28876800->28880895] (req 28876800->28880895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.318194:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c301f0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28876800->28880895] (req 28876800->28880895) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.318212:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c301f0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28876800->28880895] (req 28876800->28880895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.318220:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c301f0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28876800->28880895] (req 28876800->28880895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760790.318305:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4130000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512766464:12345-192.168.203.56@tcp:4:dd.0 Request processed in 27073us (27421us total) trans 4294974369 rc 0/0 00000100:00100000:0.0:1789760790.318316:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7080 00000100:00100000:0.0:1789760790.318326:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 769815916263 time 769855515239 state 0x2 ref 0 00000100:00100000:1.0:1789760790.327362:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760790.327441:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512766592 00000100:00100000:0.0:1789760790.327450:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512766592 00000100:00100000:0.0:1789760790.327504:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 769815916263 time 769855515239 state 0x0 ref 1 00000100:00100000:0.0:1789760790.327521:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 769855775655 time 769855515239 state 0x1 ref 1 00000100:00100000:0.0:1789760790.327611:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 769855775655 time 769891909572 state 0x0 ref 1 00000100:00100000:0.0:1789760790.327625:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac403b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512766592:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760790.327689:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28880896, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0xe007 00010000:00010000:0.0:1789760790.327725:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c301f7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.327742:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=769s v=(0 0) 00010000:00010000:0.0:1789760790.327746:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760790.327752:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c301f7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28880896->28884991] (req 28880896->28884991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.327765:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c301f7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28880896->28884991] (req 28880896->28884991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760790.361891:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974370 is committed 00002000:00010000:0.0:1789760790.362009:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28880896 -> 28884992 00010000:00010000:0.0:1789760790.362024:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c301f7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28880896->28884991] (req 28880896->28884991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.362037:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c301f7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28880896->28884991] (req 28880896->28884991) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.362045:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c301f7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28880896->28884991] (req 28880896->28884991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.362053:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760790.362058:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c301f7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28880896->28884991] (req 28880896->28884991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.362067:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c301f7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28880896->28884991] (req 28880896->28884991) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.362087:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c301f7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28880896->28884991] (req 28880896->28884991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.362095:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c301f7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28880896->28884991] (req 28880896->28884991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760790.362221:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac403b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512766592:12345-192.168.203.56@tcp:4:dd.0 Request processed in 34594us (34863us total) trans 4294974370 rc 0/0 00000100:00100000:0.0:1789760790.362230:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7081 00000100:00100000:0.0:1789760790.362239:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 769855775655 time 769891909572 state 0x2 ref 0 00000100:00100000:0.0:1789760790.364414:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760790.364469:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512766720 00000100:00100000:3.0:1789760790.364478:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512766720 00000100:00100000:3.0:1789760790.364527:0:6710:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 769855775655 time 769891909572 state 0x0 ref 1 00000100:00100000:3.0:1789760790.364544:0:6710:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 769892169988 time 769891909572 state 0x1 ref 1 00000100:00100000:1.0:1789760790.364606:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 769892169988 time 769928904351 state 0x0 ref 1 00000100:00100000:1.0:1789760790.364630:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac38b4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512766720:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760790.364678:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28884992, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0xe007 00010000:00010000:1.0:1789760790.364687:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c301fe lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.364725:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=769s v=(0 0) 00010000:00010000:1.0:1789760790.364729:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760790.364735:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c301fe lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28884992->28889087] (req 28884992->28889087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.364749:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c301fe lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28884992->28889087] (req 28884992->28889087) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760790.388377:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28884992 -> 28889088 00010000:00010000:0.0:1789760790.388398:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c301fe lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28884992->28889087] (req 28884992->28889087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.388410:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c301fe lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28884992->28889087] (req 28884992->28889087) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.388419:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c301fe lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28884992->28889087] (req 28884992->28889087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.388426:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760790.388431:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c301fe lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28884992->28889087] (req 28884992->28889087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.388440:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c301fe lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28884992->28889087] (req 28884992->28889087) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.388459:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c301fe lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28884992->28889087] (req 28884992->28889087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.388467:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c301fe lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28884992->28889087] (req 28884992->28889087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760790.388531:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974371 is committed 00000100:00100000:0.0:1789760790.388564:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac38b4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512766720:12345-192.168.203.56@tcp:4:dd.0 Request processed in 23942us (24154us total) trans 4294974371 rc 0/0 00000100:00100000:0.0:1789760790.388578:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7082 00000100:00100000:0.0:1789760790.388588:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 769892169988 time 769928904351 state 0x2 ref 0 00000100:00100000:2.0:1789760790.391156:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760790.391242:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512766848 00000100:00100000:0.0:1789760790.391253:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512766848 00000100:00100000:0.0:1789760790.391318:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 769892169988 time 769928904351 state 0x0 ref 1 00000100:00100000:0.0:1789760790.391334:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 769929164767 time 769928904351 state 0x1 ref 1 00000100:00100000:0.0:1789760790.391369:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 769929164767 time 769955659432 state 0x0 ref 1 00000100:00100000:0.0:1789760790.391382:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4058e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512766848:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760790.391427:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28889088, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0xe007 00010000:00010000:0.0:1789760790.391436:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c30205 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.391451:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=769s v=(0 0) 00010000:00010000:0.0:1789760790.391455:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760790.391462:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c30205 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28889088->28893183] (req 28889088->28893183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.391483:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c30205 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28889088->28893183] (req 28889088->28893183) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760790.416627:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974372 is committed 00002000:00010000:0.0:1789760790.416780:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28889088 -> 28893184 00010000:00010000:0.0:1789760790.416794:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c30205 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28889088->28893183] (req 28889088->28893183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.416807:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c30205 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28889088->28893183] (req 28889088->28893183) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.416817:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c30205 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28889088->28893183] (req 28889088->28893183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.416825:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760790.416831:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c30205 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28889088->28893183] (req 28889088->28893183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.416853:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c30205 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28889088->28893183] (req 28889088->28893183) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.416873:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c30205 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28889088->28893183] (req 28889088->28893183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.416881:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c30205 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28889088->28893183] (req 28889088->28893183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760790.416976:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4058e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512766848:12345-192.168.203.56@tcp:4:dd.0 Request processed in 25584us (25813us total) trans 4294974372 rc 0/0 00000100:00100000:0.0:1789760790.416986:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7083 00000100:00100000:0.0:1789760790.416996:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 769929164767 time 769955659432 state 0x2 ref 0 00000100:00100000:1.0:1789760790.422058:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760790.422138:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512766976 00000100:00100000:0.0:1789760790.422148:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512766976 00000100:00100000:0.0:1789760790.422197:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 769929164767 time 769955659432 state 0x0 ref 1 00000100:00100000:0.0:1789760790.422212:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 769955919848 time 769955659432 state 0x1 ref 1 00000100:00100000:0.0:1789760790.422242:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 769955919848 time 769986541500 state 0x0 ref 1 00000100:00100000:0.0:1789760790.422254:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4038380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512766976:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760790.422302:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28893184, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0xe007 00010000:00010000:0.0:1789760790.422311:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c3020c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.422328:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=769s v=(0 0) 00010000:00010000:0.0:1789760790.422333:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760790.422340:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c3020c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28893184->28897279] (req 28893184->28897279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.422355:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c3020c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28893184->28897279] (req 28893184->28897279) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760790.465603:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28893184 -> 28897280 00010000:00010000:2.0:1789760790.465630:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c3020c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28893184->28897279] (req 28893184->28897279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.465652:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c3020c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28893184->28897279] (req 28893184->28897279) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.465662:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c3020c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28893184->28897279] (req 28893184->28897279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.465669:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760790.465674:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c3020c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28893184->28897279] (req 28893184->28897279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.465684:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c3020c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28893184->28897279] (req 28893184->28897279) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.465736:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c3020c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28893184->28897279] (req 28893184->28897279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.465752:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c3020c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28893184->28897279] (req 28893184->28897279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760790.465837:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4038380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2358:x1876699512766976:12345-192.168.203.56@tcp:4:dd.0 Request processed in 43579us (43783us total) trans 4294974373 rc 0/0 00000100:00100000:2.0:1789760790.465848:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7084 00000100:00100000:2.0:1789760790.465857:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 769955919848 time 769986541500 state 0x2 ref 0 00000001:00080000:0.0:1789760790.470729:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974373 is committed 00000100:00100000:1.0:1789760790.471298:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760790.471417:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512767104 00000100:00100000:0.0:1789760790.471427:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512767104 00000100:00100000:0.0:1789760790.471479:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 769955919848 time 769986541500 state 0x0 ref 1 00000100:00100000:0.0:1789760790.471493:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 769986801916 time 769986541500 state 0x1 ref 1 00000100:00100000:0.0:1789760790.471560:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 769986801916 time 770035858892 state 0x0 ref 1 00000100:00100000:0.0:1789760790.471572:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4039500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512767104:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760790.471622:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28897280, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0xe007 00010000:00010000:0.0:1789760790.471630:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c30213 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.471656:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=770s v=(0 0) 00010000:00010000:0.0:1789760790.471661:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760790.471668:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c30213 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28897280->28901375] (req 28897280->28901375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.471680:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c30213 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28897280->28901375] (req 28897280->28901375) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760790.502395:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974374 is committed 00002000:00010000:0.0:1789760790.502557:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28897280 -> 28901376 00010000:00010000:0.0:1789760790.502574:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c30213 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28897280->28901375] (req 28897280->28901375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.502586:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c30213 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28897280->28901375] (req 28897280->28901375) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.502595:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c30213 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28897280->28901375] (req 28897280->28901375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.502602:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760790.502607:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c30213 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28897280->28901375] (req 28897280->28901375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.502615:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c30213 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28897280->28901375] (req 28897280->28901375) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.502634:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c30213 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28897280->28901375] (req 28897280->28901375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.502654:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c30213 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28897280->28901375] (req 28897280->28901375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760790.502775:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4039500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512767104:12345-192.168.203.56@tcp:4:dd.0 Request processed in 31200us (31478us total) trans 4294974374 rc 0/0 00000100:00100000:0.0:1789760790.502785:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7085 00000100:00100000:0.0:1789760790.502795:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 769986801916 time 770035858892 state 0x2 ref 0 00000100:00100000:0.0:1789760790.506946:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760790.507568:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512767232 00000100:00100000:1.0:1789760790.507579:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512767232 00000100:00100000:1.0:1789760790.507632:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 769986801916 time 770035858892 state 0x0 ref 1 00000100:00100000:1.0:1789760790.507646:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 770036119308 time 770035858892 state 0x1 ref 1 00000100:00100000:1.0:1789760790.507679:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 770036119308 time 770071978109 state 0x0 ref 1 00000100:00100000:1.0:1789760790.507691:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac38b5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512767232:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760790.507774:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28901376, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0xe007 00010000:00010000:1.0:1789760790.507782:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c3021a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.507803:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=770s v=(0 0) 00010000:00010000:1.0:1789760790.507808:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760790.507814:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c3021a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28901376->28905471] (req 28901376->28905471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.507828:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c3021a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28901376->28905471] (req 28901376->28905471) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760790.563094:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974375 is committed 00002000:00010000:2.0:1789760790.563170:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28901376 -> 28905472 00002000:00010000:2.0:1789760790.563176:0:6709:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760789 -> 1789760790 00002000:00010000:2.0:1789760790.563180:0:6709:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760789 -> 1789760790 00010000:00010000:2.0:1789760790.563193:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c3021a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28901376->28905471] (req 28901376->28905471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.563207:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c3021a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28901376->28905471] (req 28901376->28905471) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.563216:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c3021a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28901376->28905471] (req 28901376->28905471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.563222:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760790.563227:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c3021a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28901376->28905471] (req 28901376->28905471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.563242:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c3021a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28901376->28905471] (req 28901376->28905471) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.563260:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c3021a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28901376->28905471] (req 28901376->28905471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.563267:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c3021a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28901376->28905471] (req 28901376->28905471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760790.563354:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac38b5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512767232:12345-192.168.203.56@tcp:4:dd.0 Request processed in 55660us (56413us total) trans 4294974375 rc 0/0 00000100:00100000:2.0:1789760790.563364:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7086 00000100:00100000:2.0:1789760790.563373:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 770036119308 time 770071978109 state 0x2 ref 0 00000100:00100000:1.0:1789760790.569567:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760790.569753:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512767360 00000100:00100000:2.0:1789760790.569763:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512767360 00000100:00100000:2.0:1789760790.569811:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 770036119308 time 770071978109 state 0x0 ref 1 00000100:00100000:2.0:1789760790.569826:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 770072238525 time 770071978109 state 0x1 ref 1 00000100:00100000:2.0:1789760790.569856:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 770072238525 time 770134155782 state 0x0 ref 1 00000100:00100000:2.0:1789760790.569867:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac403a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512767360:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760790.569912:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28905472, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0xe007 00010000:00010000:2.0:1789760790.569921:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c30221 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.569937:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=770s v=(0 0) 00010000:00010000:2.0:1789760790.569942:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760790.569949:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c30221 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28905472->28909567] (req 28905472->28909567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.569963:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c30221 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28905472->28909567] (req 28905472->28909567) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760790.607675:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974376 is committed 00002000:00010000:0.0:1789760790.608022:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28905472 -> 28909568 00010000:00010000:0.0:1789760790.608036:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c30221 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28905472->28909567] (req 28905472->28909567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.608048:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c30221 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28905472->28909567] (req 28905472->28909567) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.608057:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c30221 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28905472->28909567] (req 28905472->28909567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.608064:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760790.608068:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c30221 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28905472->28909567] (req 28905472->28909567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.608078:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c30221 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28905472->28909567] (req 28905472->28909567) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.608097:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c30221 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28905472->28909567] (req 28905472->28909567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.608105:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c30221 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28905472->28909567] (req 28905472->28909567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760790.608228:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac403a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512767360:12345-192.168.203.56@tcp:4:dd.0 Request processed in 38357us (38666us total) trans 4294974376 rc 0/0 00000100:00100000:0.0:1789760790.608238:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7087 00000100:00100000:0.0:1789760790.608247:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 770072238525 time 770134155782 state 0x2 ref 0 00000100:00100000:1.0:1789760790.616035:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760790.616980:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512767488 00000100:00100000:0.0:1789760790.616992:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512767488 00000100:00100000:0.0:1789760790.617051:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 770072238525 time 770134155782 state 0x0 ref 1 00000100:00100000:0.0:1789760790.617066:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 770134416198 time 770134155782 state 0x1 ref 1 00000100:00100000:0.0:1789760790.617154:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 770134416198 time 770181452346 state 0x0 ref 1 00000100:00100000:0.0:1789760790.617168:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febea89b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512767488:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760790.617217:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28909568, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0xe007 00010000:00010000:0.0:1789760790.617227:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c30228 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.617244:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=770s v=(0 0) 00010000:00010000:0.0:1789760790.617248:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760790.617253:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c30228 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28909568->28913663] (req 28909568->28913663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.617265:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c30228 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28909568->28913663] (req 28909568->28913663) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760790.650882:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28909568 -> 28913664 00010000:00010000:1.0:1789760790.650902:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c30228 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28909568->28913663] (req 28909568->28913663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.650914:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c30228 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28909568->28913663] (req 28909568->28913663) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.650923:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c30228 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28909568->28913663] (req 28909568->28913663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.650929:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760790.650934:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c30228 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28909568->28913663] (req 28909568->28913663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.650942:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c30228 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28909568->28913663] (req 28909568->28913663) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.650962:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c30228 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28909568->28913663] (req 28909568->28913663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.650981:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c30228 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28909568->28913663] (req 28909568->28913663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760790.651076:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febea89b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2359:x1876699512767488:12345-192.168.203.56@tcp:4:dd.0 Request processed in 33905us (35045us total) trans 4294974377 rc 0/0 00000100:00100000:1.0:1789760790.651086:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7088 00000100:00100000:1.0:1789760790.651095:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 770134416198 time 770181452346 state 0x2 ref 0 00000001:00080000:1.0:1789760790.651136:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974377 is committed 00000100:00100000:2.0:1789760790.655101:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760790.655238:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512767616 00000100:00100000:1.0:1789760790.655248:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512767616 00000100:00100000:1.0:1789760790.655390:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 770134416198 time 770181452346 state 0x0 ref 1 00000100:00100000:1.0:1789760790.655404:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 770181712762 time 770181452346 state 0x1 ref 1 00000100:00100000:1.0:1789760790.655432:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 770181712762 time 770219732236 state 0x0 ref 1 00000100:00100000:1.0:1789760790.655444:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4059180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512767616:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760790.655488:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28913664, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0xe007 00010000:00010000:1.0:1789760790.655497:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c3022f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.655518:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=770s v=(0 0) 00010000:00010000:1.0:1789760790.655523:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760790.655528:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c3022f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28913664->28917759] (req 28913664->28917759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.655542:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c3022f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28913664->28917759] (req 28913664->28917759) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760790.701157:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974378 is committed 00002000:00010000:1.0:1789760790.701345:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28913664 -> 28917760 00010000:00010000:1.0:1789760790.701366:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c3022f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28913664->28917759] (req 28913664->28917759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.701380:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c3022f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28913664->28917759] (req 28913664->28917759) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.701389:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c3022f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28913664->28917759] (req 28913664->28917759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.701396:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760790.701402:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c3022f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28913664->28917759] (req 28913664->28917759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.701412:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c3022f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28913664->28917759] (req 28913664->28917759) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.701435:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c3022f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28913664->28917759] (req 28913664->28917759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.701444:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c3022f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28913664->28917759] (req 28913664->28917759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760790.701539:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4059180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512767616:12345-192.168.203.56@tcp:4:dd.0 Request processed in 46092us (46443us total) trans 4294974378 rc 0/0 00000100:00100000:1.0:1789760790.701549:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7089 00000100:00100000:1.0:1789760790.701558:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 770181712762 time 770219732236 state 0x2 ref 0 00000100:00100000:2.0:1789760790.705354:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760790.705473:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512767744 00000100:00100000:0.0:1789760790.705483:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512767744 00000100:00100000:0.0:1789760790.705536:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 770181712762 time 770219732236 state 0x0 ref 1 00000100:00100000:0.0:1789760790.705551:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 770219992652 time 770219732236 state 0x1 ref 1 00000100:00100000:0.0:1789760790.705614:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 770219992652 time 770269912641 state 0x0 ref 1 00000100:00100000:0.0:1789760790.705627:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febfb374380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512767744:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760790.705681:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28917760, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0xe007 00010000:00010000:0.0:1789760790.705690:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c30236 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.705746:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=770s v=(0 0) 00010000:00010000:0.0:1789760790.705751:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760790.705760:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c30236 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28917760->28921855] (req 28917760->28921855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.705795:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c30236 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28917760->28921855] (req 28917760->28921855) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760790.739255:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974379 is committed 00002000:00010000:0.0:1789760790.743656:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28917760 -> 28921856 00010000:00010000:0.0:1789760790.743676:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c30236 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28917760->28921855] (req 28917760->28921855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.743689:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c30236 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28917760->28921855] (req 28917760->28921855) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.743713:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c30236 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28917760->28921855] (req 28917760->28921855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.743720:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760790.743795:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c30236 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28917760->28921855] (req 28917760->28921855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.743807:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c30236 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28917760->28921855] (req 28917760->28921855) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.743827:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c30236 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28917760->28921855] (req 28917760->28921855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.743835:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c30236 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28917760->28921855] (req 28917760->28921855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760790.743995:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febfb374380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512767744:12345-192.168.203.56@tcp:4:dd.0 Request processed in 38365us (38708us total) trans 4294974379 rc 0/0 00000100:00100000:0.0:1789760790.744007:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7090 00000100:00100000:0.0:1789760790.744016:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 770219992652 time 770269912641 state 0x2 ref 0 00000100:00100000:0.0:1789760790.753341:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760790.755405:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512767872 00000100:00100000:1.0:1789760790.755417:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512767872 00000100:00100000:0.0:1789760790.755566:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 770219992652 time 770269912641 state 0x0 ref 1 00000100:00100000:0.0:1789760790.755583:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 770270173057 time 770269912641 state 0x1 ref 1 00000100:00100000:0.0:1789760790.755613:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 770270173057 time 770319912894 state 0x0 ref 1 00000100:00100000:0.0:1789760790.755626:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac38b5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512767872:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760790.755815:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28921856, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0xe007 00010000:00010000:0.0:1789760790.755824:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c3023d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.755840:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=770s v=(0 0) 00010000:00010000:0.0:1789760790.755845:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760790.755850:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c3023d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28921856->28925951] (req 28921856->28925951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.755864:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c3023d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28921856->28925951] (req 28921856->28925951) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760790.796364:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760790.796400:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760790.796434:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760790.796465:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760790.796529:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512768128 00000100:00100000:3.0:1789760790.796540:0:5847:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512768128 00000100:00100000:3.0:1789760790.796600:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512768384 00000100:00100000:3.0:1789760790.796614:0:5847:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512768384 00000100:00100000:3.0:1789760790.796645:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 519 00000100:00100000:3.0:1789760790.796654:0:5847:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5305500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+8:2361:x1876699512768128:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:3.0:1789760790.796914:0:5847:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5305500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+8:2361:x1876699512768128:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 260us (559us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760790.796924:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 519 00000100:00100000:3.0:1789760790.796953:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 520 00000100:00100000:3.0:1789760790.796961:0:5847:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5307b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:d2c40150-e97b-40da-952b-cb9f18bf7e96+5:2361:x1876699512768384:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:3.0:1789760790.796998:0:5847:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5307b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:d2c40150-e97b-40da-952b-cb9f18bf7e96+5:2361:x1876699512768384:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 38us (597us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760790.797004:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 520 00000100:00100000:0.0:1789760790.797153:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512768000 00000100:00100000:0.0:1789760790.797158:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512768000 00000100:00100000:0.0:1789760790.797196:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 416 00000100:00100000:0.0:1789760790.797203:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febe00c6c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:e6cf7267-9b19-462a-a263-14115df15dab+13:2361:x1876699512768000:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:0.0:1789760790.797231:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febe00c6c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:e6cf7267-9b19-462a-a263-14115df15dab+13:2361:x1876699512768000:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 28us (796us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760790.797238:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 416 00000100:00100000:2.0:1789760790.797317:0:6704:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512768256 00000100:00100000:2.0:1789760790.797322:0:6704:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512768256 00000100:00100000:2.0:1789760790.797357:0:6704:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 196 00000100:00100000:2.0:1789760790.797365:0:6704:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5305f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+8:2361:x1876699512768256:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:2.0:1789760790.797393:0:6704:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5305f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+8:2361:x1876699512768256:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 29us (927us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760790.797400:0:6704:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 196 00000100:00080000:1.0:1789760790.797630:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000001:00080000:3.0:1789760790.806508:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974380 is committed 00002000:00010000:1.0:1789760790.806762:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28921856 -> 28925952 00010000:00010000:1.0:1789760790.806777:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c3023d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28921856->28925951] (req 28921856->28925951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.806790:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c3023d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28921856->28925951] (req 28921856->28925951) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.806799:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c3023d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28921856->28925951] (req 28921856->28925951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.806807:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760790.806813:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c3023d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28921856->28925951] (req 28921856->28925951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.806822:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c3023d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28921856->28925951] (req 28921856->28925951) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.806840:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c3023d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28921856->28925951] (req 28921856->28925951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.806847:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c3023d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28921856->28925951] (req 28921856->28925951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760790.806932:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac38b5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512767872:12345-192.168.203.56@tcp:4:dd.0 Request processed in 51303us (53595us total) trans 4294974380 rc 0/0 00000100:00100000:1.0:1789760790.806942:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7091 00000100:00100000:1.0:1789760790.806952:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 770270173057 time 770319912894 state 0x2 ref 0 00000100:00100000:1.0:1789760790.809397:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760790.809670:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512768512 00000100:00100000:2.0:1789760790.809680:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512768512 00000100:00100000:2.0:1789760790.809768:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 770270173057 time 770319912894 state 0x0 ref 1 00000100:00100000:2.0:1789760790.809783:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 770320173310 time 770319912894 state 0x1 ref 1 00000100:00100000:2.0:1789760790.809811:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 770320173310 time 770374110912 state 0x0 ref 1 00000100:00100000:2.0:1789760790.809826:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5307480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512768512:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760790.809882:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28925952, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0xe007 00010000:00010000:2.0:1789760790.809891:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c30244 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.809907:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=770s v=(0 0) 00010000:00010000:2.0:1789760790.809911:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760790.809918:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c30244 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28925952->28930047] (req 28925952->28930047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.809932:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c30244 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28925952->28930047] (req 28925952->28930047) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760790.840648:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974381 is committed 00002000:00010000:1.0:1789760790.840747:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28925952 -> 28930048 00010000:00010000:1.0:1789760790.840764:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c30244 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28925952->28930047] (req 28925952->28930047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.840777:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c30244 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28925952->28930047] (req 28925952->28930047) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.840785:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c30244 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28925952->28930047] (req 28925952->28930047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.840798:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760790.840803:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c30244 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28925952->28930047] (req 28925952->28930047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.840813:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c30244 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28925952->28930047] (req 28925952->28930047) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.840830:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c30244 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28925952->28930047] (req 28925952->28930047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.840838:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c30244 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28925952->28930047] (req 28925952->28930047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760790.842283:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5307480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512768512:12345-192.168.203.56@tcp:4:dd.0 Request processed in 32449us (32884us total) trans 4294974381 rc 0/0 00000100:00100000:2.0:1789760790.842296:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7092 00000100:00100000:2.0:1789760790.842305:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 770320173310 time 770374110912 state 0x2 ref 0 00000100:00100000:0.0:1789760790.852056:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760790.852148:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512768640 00000100:00100000:2.0:1789760790.852158:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512768640 00000100:00100000:2.0:1789760790.852213:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 770320173310 time 770374110912 state 0x0 ref 1 00000100:00100000:2.0:1789760790.852229:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 770374371328 time 770374110912 state 0x1 ref 1 00000100:00100000:2.0:1789760790.852257:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 770374371328 time 770416557288 state 0x0 ref 1 00000100:00100000:2.0:1789760790.852272:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac38b6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512768640:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760790.852318:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28930048, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0xe007 00010000:00010000:2.0:1789760790.852326:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c3024b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.852341:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=770s v=(0 0) 00010000:00010000:2.0:1789760790.852345:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760790.852351:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c3024b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28930048->28934143] (req 28930048->28934143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.852375:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c3024b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28930048->28934143] (req 28930048->28934143) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760790.883719:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28930048 -> 28934144 00010000:00010000:1.0:1789760790.883740:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c3024b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28930048->28934143] (req 28930048->28934143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.883752:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c3024b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28930048->28934143] (req 28930048->28934143) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.883762:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c3024b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28930048->28934143] (req 28930048->28934143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.883769:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760790.883774:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c3024b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28930048->28934143] (req 28930048->28934143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.883784:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c3024b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28930048->28934143] (req 28930048->28934143) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.883804:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c3024b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28930048->28934143] (req 28930048->28934143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.883812:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c3024b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28930048->28934143] (req 28930048->28934143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760790.883905:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac38b6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2360:x1876699512768640:12345-192.168.203.56@tcp:4:dd.0 Request processed in 31629us (31852us total) trans 4294974382 rc 0/0 00000100:00100000:1.0:1789760790.883916:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7093 00000100:00100000:1.0:1789760790.883926:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 770374371328 time 770416557288 state 0x2 ref 0 00000001:00080000:3.0:1789760790.884526:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974382 is committed 00000100:00100000:0.0:1789760790.888102:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760790.890303:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512768768 00000100:00100000:1.0:1789760790.890316:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512768768 00000100:00100000:1.0:1789760790.890369:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 770374371328 time 770416557288 state 0x0 ref 1 00000100:00100000:1.0:1789760790.890386:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 770416817704 time 770416557288 state 0x1 ref 1 00000100:00100000:1.0:1789760790.890417:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 770416817704 time 770454716136 state 0x0 ref 1 00000100:00100000:1.0:1789760790.890429:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febea48bc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512768768:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760790.890479:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28934144, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0xe007 00010000:00010000:1.0:1789760790.890488:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c30252 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.890505:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=770s v=(0 0) 00010000:00010000:1.0:1789760790.890509:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760790.890515:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c30252 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28934144->28938239] (req 28934144->28938239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.890529:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c30252 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28934144->28938239] (req 28934144->28938239) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.898867:0:5836:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:1.0:1789760790.940573:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28934144 -> 28938240 00010000:00010000:1.0:1789760790.940592:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c30252 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28934144->28938239] (req 28934144->28938239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.940606:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c30252 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28934144->28938239] (req 28934144->28938239) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.940616:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c30252 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28934144->28938239] (req 28934144->28938239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.940623:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760790.940628:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c30252 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28934144->28938239] (req 28934144->28938239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.940638:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c30252 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28934144->28938239] (req 28934144->28938239) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.940662:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c30252 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28934144->28938239] (req 28934144->28938239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.940670:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c30252 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28934144->28938239] (req 28934144->28938239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760790.942194:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974383 is committed 00000100:00100000:0.0:1789760790.942474:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febea48bc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512768768:12345-192.168.203.56@tcp:4:dd.0 Request processed in 52041us (54375us total) trans 4294974383 rc 0/0 00000100:00100000:0.0:1789760790.942487:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7094 00000100:00100000:0.0:1789760790.942497:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 770416817704 time 770454716136 state 0x2 ref 0 00000100:00100000:0.0:1789760790.945452:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760790.945570:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512768896 00000100:00100000:2.0:1789760790.945580:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512768896 00000100:00100000:2.0:1789760790.945638:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 770416817704 time 770454716136 state 0x0 ref 1 00000100:00100000:2.0:1789760790.945653:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 770454976552 time 770454716136 state 0x1 ref 1 00000100:00100000:2.0:1789760790.945841:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 770454976552 time 770510138907 state 0x0 ref 1 00000100:00100000:2.0:1789760790.945854:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac38b6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512768896:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760790.945902:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28938240, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0xe007 00010000:00010000:2.0:1789760790.945912:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c30259 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.945927:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=770s v=(0 0) 00010000:00010000:2.0:1789760790.945931:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760790.945937:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c30259 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28938240->28942335] (req 28938240->28942335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.945950:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c30259 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28938240->28942335] (req 28938240->28942335) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760790.997773:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974384 is committed 00002000:00010000:3.0:1789760790.997834:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28938240 -> 28942336 00010000:00010000:3.0:1789760790.997849:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c30259 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28938240->28942335] (req 28938240->28942335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.997862:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c30259 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28938240->28942335] (req 28938240->28942335) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.997886:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c30259 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28938240->28942335] (req 28938240->28942335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.997894:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760790.997899:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c30259 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28938240->28942335] (req 28938240->28942335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.997908:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c30259 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28938240->28942335] (req 28938240->28942335) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.997932:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c30259 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28938240->28942335] (req 28938240->28942335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.997940:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c30259 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28938240->28942335] (req 28938240->28942335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760790.998031:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac38b6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512768896:12345-192.168.203.56@tcp:4:dd.0 Request processed in 52174us (52583us total) trans 4294974384 rc 0/0 00000100:00100000:3.0:1789760790.998041:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7095 00000100:00100000:3.0:1789760790.998057:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 770454976552 time 770510138907 state 0x2 ref 0 00000100:00100000:1.0:1789760791.000825:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760791.006800:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512769024 00000100:00100000:2.0:1789760791.006816:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512769024 00000100:00080000:1.0:1789760791.007616:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1789760791.007631:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 770454976552 time 770510138907 state 0x0 ref 1 00000100:00100000:3.0:1789760791.007649:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 770510399323 time 770510138907 state 0x1 ref 1 00000100:00100000:3.0:1789760791.007686:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 770510399323 time 770571985347 state 0x0 ref 1 00000100:00100000:3.0:1789760791.007727:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5304700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512769024:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760791.007782:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28942336, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0xe007 00010000:00010000:3.0:1789760791.007792:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c30260 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.007809:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=770s v=(0 0) 00010000:00010000:3.0:1789760791.007813:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760791.007819:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c30260 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28942336->28946431] (req 28942336->28946431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.007834:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c30260 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28942336->28946431] (req 28942336->28946431) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760791.044556:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974385 is committed 00002000:00010000:3.0:1789760791.044726:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28942336 -> 28946432 00010000:00010000:3.0:1789760791.044742:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c30260 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28942336->28946431] (req 28942336->28946431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.044753:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c30260 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28942336->28946431] (req 28942336->28946431) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.044762:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c30260 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28942336->28946431] (req 28942336->28946431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.044769:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760791.044773:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c30260 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28942336->28946431] (req 28942336->28946431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.044782:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c30260 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28942336->28946431] (req 28942336->28946431) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.044801:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c30260 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28942336->28946431] (req 28942336->28946431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.044808:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c30260 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28942336->28946431] (req 28942336->28946431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760791.044931:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5304700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512769024:12345-192.168.203.56@tcp:4:dd.0 Request processed in 37231us (44112us total) trans 4294974385 rc 0/0 00000100:00100000:3.0:1789760791.044943:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7096 00000100:00100000:3.0:1789760791.044952:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 770510399323 time 770571985347 state 0x2 ref 0 00000100:00100000:2.0:1789760791.047529:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760791.047813:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512769152 00000100:00100000:0.0:1789760791.047824:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512769152 00000100:00100000:0.0:1789760791.047884:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 770510399323 time 770571985347 state 0x0 ref 1 00000100:00100000:0.0:1789760791.047899:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 770572245763 time 770571985347 state 0x1 ref 1 00000100:00100000:0.0:1789760791.047961:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 770572245763 time 770612260002 state 0x0 ref 1 00000100:00100000:0.0:1789760791.047983:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febfb374380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512769152:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760791.048036:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28946432, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0xe007 00010000:00010000:0.0:1789760791.048045:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c30267 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.048059:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=770s v=(0 0) 00010000:00010000:0.0:1789760791.048063:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760791.048069:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c30267 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28946432->28950527] (req 28946432->28950527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.048082:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c30267 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28946432->28950527] (req 28946432->28950527) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760791.079588:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28946432 -> 28950528 00010000:00010000:1.0:1789760791.079608:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c30267 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28946432->28950527] (req 28946432->28950527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.079622:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c30267 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28946432->28950527] (req 28946432->28950527) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.079631:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c30267 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28946432->28950527] (req 28946432->28950527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.079780:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760791.079785:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c30267 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28946432->28950527] (req 28946432->28950527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.079796:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c30267 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28946432->28950527] (req 28946432->28950527) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.079819:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c30267 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28946432->28950527] (req 28946432->28950527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.079827:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c30267 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28946432->28950527] (req 28946432->28950527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760791.079912:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febfb374380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2361:x1876699512769152:12345-192.168.203.56@tcp:4:dd.0 Request processed in 31927us (32387us total) trans 4294974386 rc 0/0 00000100:00100000:1.0:1789760791.079922:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7097 00000100:00100000:1.0:1789760791.079933:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 770572245763 time 770612260002 state 0x2 ref 0 00000001:00080000:1.0:1789760791.079988:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974386 is committed 00000100:00100000:0.0:1789760791.084492:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760791.085459:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512769280 00000100:00100000:3.0:1789760791.085471:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512769280 00000100:00100000:3.0:1789760791.085529:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 770572245763 time 770612260002 state 0x0 ref 1 00000100:00100000:3.0:1789760791.085692:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 770612520418 time 770612260002 state 0x1 ref 1 00000100:00100000:3.0:1789760791.085869:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 770612520418 time 770650167255 state 0x0 ref 1 00000100:00100000:3.0:1789760791.085884:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac38b4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512769280:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760791.085934:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28950528, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0xe007 00010000:00010000:3.0:1789760791.085943:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c3026e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.085958:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=770s v=(0 0) 00010000:00010000:3.0:1789760791.085975:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760791.085982:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c3026e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28950528->28954623] (req 28950528->28954623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.086133:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c3026e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28950528->28954623] (req 28950528->28954623) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760791.129207:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974387 is committed 00002000:00010000:1.0:1789760791.129357:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28950528 -> 28954624 00010000:00010000:1.0:1789760791.129374:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c3026e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28950528->28954623] (req 28950528->28954623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.129387:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c3026e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28950528->28954623] (req 28950528->28954623) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.129396:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c3026e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28950528->28954623] (req 28950528->28954623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.129404:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760791.129409:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c3026e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28950528->28954623] (req 28950528->28954623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.129422:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c3026e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28950528->28954623] (req 28950528->28954623) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.129439:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c3026e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28950528->28954623] (req 28950528->28954623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.129446:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c3026e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28950528->28954623] (req 28950528->28954623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760791.129538:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac38b4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512769280:12345-192.168.203.56@tcp:4:dd.0 Request processed in 43653us (45051us total) trans 4294974387 rc 0/0 00000100:00100000:1.0:1789760791.129548:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7098 00000100:00100000:1.0:1789760791.129557:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 770612520418 time 770650167255 state 0x2 ref 0 00000100:00100000:2.0:1789760791.134221:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760791.134306:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512769408 00000100:00100000:3.0:1789760791.134328:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512769408 00000100:00100000:3.0:1789760791.134385:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 770612520418 time 770650167255 state 0x0 ref 1 00000100:00100000:3.0:1789760791.134400:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 770650427671 time 770650167255 state 0x1 ref 1 00000100:00100000:3.0:1789760791.134433:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 770650427671 time 770698732675 state 0x0 ref 1 00000100:00100000:3.0:1789760791.134446:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febffb9e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512769408:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760791.134492:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28954624, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0xe007 00010000:00010000:3.0:1789760791.134501:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c30275 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.134515:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=770s v=(0 0) 00010000:00010000:3.0:1789760791.134519:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760791.134524:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c30275 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28954624->28958719] (req 28954624->28958719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.134538:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c30275 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28954624->28958719] (req 28954624->28958719) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760791.166142:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28954624 -> 28958720 00010000:00010000:3.0:1789760791.166167:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c30275 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28954624->28958719] (req 28954624->28958719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.166183:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c30275 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28954624->28958719] (req 28954624->28958719) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.166193:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c30275 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28954624->28958719] (req 28954624->28958719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.166200:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760791.166206:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c30275 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28954624->28958719] (req 28954624->28958719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.166214:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c30275 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28954624->28958719] (req 28954624->28958719) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.166233:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c30275 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28954624->28958719] (req 28954624->28958719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.166241:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c30275 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28954624->28958719] (req 28954624->28958719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760791.166348:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febffb9e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2361:x1876699512769408:12345-192.168.203.56@tcp:4:dd.0 Request processed in 31899us (32129us total) trans 4294974388 rc 0/0 00000100:00100000:3.0:1789760791.166360:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7099 00000100:00100000:3.0:1789760791.166369:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 770650427671 time 770698732675 state 0x2 ref 0 00000001:00080000:0.0:1789760791.166635:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974388 is committed 00000100:00100000:0.0:1789760791.178102:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760791.178174:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512769536 00000100:00100000:1.0:1789760791.178184:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512769536 00000100:00100000:1.0:1789760791.178231:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 770650427671 time 770698732675 state 0x0 ref 1 00000100:00100000:1.0:1789760791.178245:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 770698993091 time 770698732675 state 0x1 ref 1 00000100:00100000:1.0:1789760791.178272:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 770698993091 time 770742572468 state 0x0 ref 1 00000100:00100000:1.0:1789760791.178284:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac38b5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512769536:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760791.178327:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28958720, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0xe007 00010000:00010000:1.0:1789760791.178335:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c3027c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.178351:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=770s v=(0 0) 00010000:00010000:1.0:1789760791.178692:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760791.178724:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c3027c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28958720->28962815] (req 28958720->28962815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.178739:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c3027c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28958720->28962815] (req 28958720->28962815) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760791.213164:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974389 is committed 00002000:00010000:1.0:1789760791.213261:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28958720 -> 28962816 00010000:00010000:1.0:1789760791.213277:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c3027c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28958720->28962815] (req 28958720->28962815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.213290:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c3027c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28958720->28962815] (req 28958720->28962815) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.213300:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c3027c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28958720->28962815] (req 28958720->28962815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.213306:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760791.213311:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c3027c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28958720->28962815] (req 28958720->28962815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.213320:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c3027c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28958720->28962815] (req 28958720->28962815) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.213338:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c3027c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28958720->28962815] (req 28958720->28962815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.213348:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c3027c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28958720->28962815] (req 28958720->28962815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760791.213438:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac38b5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512769536:12345-192.168.203.56@tcp:4:dd.0 Request processed in 35150us (35341us total) trans 4294974389 rc 0/0 00000100:00100000:1.0:1789760791.213448:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7100 00000100:00100000:1.0:1789760791.213458:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 770698993091 time 770742572468 state 0x2 ref 0 00000100:00100000:1.0:1789760791.220093:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760791.220174:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512769664 00000100:00100000:3.0:1789760791.220184:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512769664 00000100:00100000:3.0:1789760791.220238:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 770698993091 time 770742572468 state 0x0 ref 1 00000100:00100000:3.0:1789760791.220252:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 770742832884 time 770742572468 state 0x1 ref 1 00000100:00100000:3.0:1789760791.220284:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 770742832884 time 770784583453 state 0x0 ref 1 00000100:00100000:3.0:1789760791.220296:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5304380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512769664:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760791.220345:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28962816, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0xe007 00010000:00010000:3.0:1789760791.220354:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c30283 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.220372:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=770s v=(0 0) 00010000:00010000:3.0:1789760791.220377:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760791.220383:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c30283 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28962816->28966911] (req 28962816->28966911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.220398:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c30283 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28962816->28966911] (req 28962816->28966911) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760791.259591:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28962816 -> 28966912 00010000:00010000:2.0:1789760791.259610:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c30283 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28962816->28966911] (req 28962816->28966911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.259623:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c30283 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28962816->28966911] (req 28962816->28966911) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.259631:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c30283 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28962816->28966911] (req 28962816->28966911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.259648:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760791.259653:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c30283 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28962816->28966911] (req 28962816->28966911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.259667:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c30283 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28962816->28966911] (req 28962816->28966911) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.259688:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c30283 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28962816->28966911] (req 28962816->28966911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.259716:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c30283 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28962816->28966911] (req 28962816->28966911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760791.259815:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5304380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2361:x1876699512769664:12345-192.168.203.56@tcp:4:dd.0 Request processed in 39515us (39726us total) trans 4294974390 rc 0/0 00000100:00100000:2.0:1789760791.259827:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7101 00000100:00100000:2.0:1789760791.259918:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 770742832884 time 770784583453 state 0x2 ref 0 00000001:00080000:2.0:1789760791.259959:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974390 is committed 00000100:00100000:2.0:1789760791.264945:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760791.265035:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512769792 00000100:00100000:3.0:1789760791.265044:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512769792 00000100:00100000:3.0:1789760791.265093:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 770742832884 time 770784583453 state 0x0 ref 1 00000100:00100000:3.0:1789760791.265109:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 770784843869 time 770784583453 state 0x1 ref 1 00000100:00100000:3.0:1789760791.265139:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 770784843869 time 770829438211 state 0x0 ref 1 00000100:00100000:3.0:1789760791.265153:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febffb9ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512769792:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760791.265204:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28966912, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0xe007 00010000:00010000:3.0:1789760791.265212:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c3028a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.265229:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=770s v=(0 0) 00010000:00010000:3.0:1789760791.265234:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760791.265241:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c3028a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28966912->28971007] (req 28966912->28971007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.265255:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c3028a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28966912->28971007] (req 28966912->28971007) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760791.312806:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974391 is committed 00002000:00010000:3.0:1789760791.313003:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28966912 -> 28971008 00010000:00010000:3.0:1789760791.313019:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c3028a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28966912->28971007] (req 28966912->28971007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.313038:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c3028a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28966912->28971007] (req 28966912->28971007) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.313047:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c3028a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28966912->28971007] (req 28966912->28971007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.313054:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760791.313059:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c3028a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28966912->28971007] (req 28966912->28971007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.313143:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c3028a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28966912->28971007] (req 28966912->28971007) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.313164:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c3028a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28966912->28971007] (req 28966912->28971007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.313172:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c3028a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28966912->28971007] (req 28966912->28971007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760791.313921:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febffb9ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512769792:12345-192.168.203.56@tcp:4:dd.0 Request processed in 48762us (48976us total) trans 4294974391 rc 0/0 00000100:00100000:3.0:1789760791.313932:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7102 00000100:00100000:3.0:1789760791.313941:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 770784843869 time 770829438211 state 0x2 ref 0 00000100:00100000:3.0:1789760791.317021:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760791.317117:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512769920 00000100:00100000:1.0:1789760791.317127:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512769920 00000100:00100000:1.0:1789760791.317180:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 770784843869 time 770829438211 state 0x0 ref 1 00000100:00100000:1.0:1789760791.317195:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 770829698627 time 770829438211 state 0x1 ref 1 00000100:00100000:1.0:1789760791.317222:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 770829698627 time 770881521581 state 0x0 ref 1 00000100:00100000:1.0:1789760791.317239:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4130700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512769920:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760791.317467:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28971008, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0xe007 00010000:00010000:1.0:1789760791.317475:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c30291 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.317488:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=770s v=(0 0) 00010000:00010000:1.0:1789760791.317491:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760791.317496:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c30291 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28971008->28975103] (req 28971008->28975103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.317507:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c30291 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28971008->28975103] (req 28971008->28975103) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760791.356262:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28971008 -> 28975104 00010000:00010000:0.0:1789760791.356282:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c30291 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28971008->28975103] (req 28971008->28975103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.356296:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c30291 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28971008->28975103] (req 28971008->28975103) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.356305:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c30291 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28971008->28975103] (req 28971008->28975103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.356316:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760791.356321:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c30291 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28971008->28975103] (req 28971008->28975103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.356329:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c30291 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28971008->28975103] (req 28971008->28975103) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.356348:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c30291 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28971008->28975103] (req 28971008->28975103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.356356:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c30291 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28971008->28975103] (req 28971008->28975103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760791.356456:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4130700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2361:x1876699512769920:12345-192.168.203.56@tcp:4:dd.0 Request processed in 39215us (39438us total) trans 4294974392 rc 0/0 00000100:00100000:0.0:1789760791.356466:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7103 00000100:00100000:0.0:1789760791.356486:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 770829698627 time 770881521581 state 0x2 ref 0 00000001:00080000:3.0:1789760791.356624:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974392 is committed 00000100:00100000:0.0:1789760791.358952:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760791.359043:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512770048 00000100:00100000:2.0:1789760791.359054:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512770048 00000100:00100000:2.0:1789760791.359105:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 770829698627 time 770881521581 state 0x0 ref 1 00000100:00100000:2.0:1789760791.359119:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 770881781997 time 770881521581 state 0x1 ref 1 00000100:00100000:2.0:1789760791.359148:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 770881781997 time 770923447999 state 0x0 ref 1 00000100:00100000:2.0:1789760791.359161:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac38b5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512770048:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760791.359213:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28975104, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0xe007 00010000:00010000:2.0:1789760791.359222:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c30298 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.359238:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=770s v=(0 0) 00010000:00010000:2.0:1789760791.359242:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760791.359249:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c30298 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28975104->28979199] (req 28975104->28979199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.359263:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c30298 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28975104->28979199] (req 28975104->28979199) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760791.385504:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974393 is committed 00002000:00010000:3.0:1789760791.385592:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28975104 -> 28979200 00010000:00010000:3.0:1789760791.385607:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c30298 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28975104->28979199] (req 28975104->28979199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.385620:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c30298 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28975104->28979199] (req 28975104->28979199) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.385629:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c30298 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28975104->28979199] (req 28975104->28979199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.385693:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760791.385726:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c30298 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28975104->28979199] (req 28975104->28979199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.385737:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c30298 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28975104->28979199] (req 28975104->28979199) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.385755:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c30298 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28975104->28979199] (req 28975104->28979199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.385763:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c30298 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28975104->28979199] (req 28975104->28979199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760791.385837:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac38b5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512770048:12345-192.168.203.56@tcp:4:dd.0 Request processed in 26674us (26889us total) trans 4294974393 rc 0/0 00000100:00100000:3.0:1789760791.385847:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7104 00000100:00100000:3.0:1789760791.385857:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 770881781997 time 770923447999 state 0x2 ref 0 00000100:00100000:2.0:1789760791.392665:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760791.392768:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512770176 00000100:00100000:3.0:1789760791.392779:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512770176 00000100:00100000:3.0:1789760791.392837:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 770881781997 time 770923447999 state 0x0 ref 1 00000100:00100000:3.0:1789760791.392853:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 770923708415 time 770923447999 state 0x1 ref 1 00000100:00100000:3.0:1789760791.392884:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 770923708415 time 770957183117 state 0x0 ref 1 00000100:00100000:3.0:1789760791.392896:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febea772680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512770176:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760791.392943:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28979200, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0xe007 00010000:00010000:3.0:1789760791.392951:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c3029f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.392979:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=770s v=(0 0) 00010000:00010000:3.0:1789760791.392983:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760791.392989:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c3029f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28979200->28983295] (req 28979200->28983295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.393004:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c3029f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28979200->28983295] (req 28979200->28983295) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760791.429854:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974394 is committed 00002000:00010000:2.0:1789760791.432383:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28979200 -> 28983296 00010000:00010000:2.0:1789760791.432405:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c3029f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28979200->28983295] (req 28979200->28983295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.432417:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c3029f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28979200->28983295] (req 28979200->28983295) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.432425:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c3029f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28979200->28983295] (req 28979200->28983295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.432432:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760791.432437:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c3029f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28979200->28983295] (req 28979200->28983295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.432450:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c3029f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28979200->28983295] (req 28979200->28983295) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.432468:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c3029f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28979200->28983295] (req 28979200->28983295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.432475:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c3029f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28979200->28983295] (req 28979200->28983295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760791.432626:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febea772680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512770176:12345-192.168.203.56@tcp:4:dd.0 Request processed in 39728us (39966us total) trans 4294974394 rc 0/0 00000100:00100000:2.0:1789760791.432646:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7105 00000100:00100000:2.0:1789760791.432657:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 770923708415 time 770957183117 state 0x2 ref 0 00000100:00100000:0.0:1789760791.436764:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760791.436872:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512770304 00000100:00100000:1.0:1789760791.436883:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512770304 00000100:00100000:1.0:1789760791.436934:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 770923708415 time 770957183117 state 0x0 ref 1 00000100:00100000:1.0:1789760791.436950:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 770957443533 time 770957183117 state 0x1 ref 1 00000100:00100000:1.0:1789760791.436991:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 770957443533 time 771001290591 state 0x0 ref 1 00000100:00100000:1.0:1789760791.437004:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac38b6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512770304:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760791.437050:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28983296, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0xe007 00010000:00010000:1.0:1789760791.437059:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c302a6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.437074:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=771s v=(0 0) 00010000:00010000:1.0:1789760791.437078:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760791.437085:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c302a6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28983296->28987391] (req 28983296->28987391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.437102:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c302a6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28983296->28987391] (req 28983296->28987391) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760791.483023:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974395 is committed 00002000:00010000:1.0:1789760791.483127:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28983296 -> 28987392 00010000:00010000:1.0:1789760791.483144:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c302a6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28983296->28987391] (req 28983296->28987391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.483237:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c302a6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28983296->28987391] (req 28983296->28987391) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.483247:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c302a6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28983296->28987391] (req 28983296->28987391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.483256:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760791.483261:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c302a6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28983296->28987391] (req 28983296->28987391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.483270:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c302a6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28983296->28987391] (req 28983296->28987391) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.483291:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c302a6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28983296->28987391] (req 28983296->28987391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.483300:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c302a6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28983296->28987391] (req 28983296->28987391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760791.483443:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac38b6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512770304:12345-192.168.203.56@tcp:4:dd.0 Request processed in 46436us (46682us total) trans 4294974395 rc 0/0 00000100:00100000:1.0:1789760791.483454:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7106 00000100:00100000:1.0:1789760791.483464:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 770957443533 time 771001290591 state 0x2 ref 0 00000100:00100000:2.0:1789760791.499746:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760791.499844:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512770432 00000100:00100000:3.0:1789760791.499854:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512770432 00000100:00100000:3.0:1789760791.499906:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 770957443533 time 771001290591 state 0x0 ref 1 00000100:00100000:3.0:1789760791.499923:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 771001551007 time 771001290591 state 0x1 ref 1 00000100:00100000:3.0:1789760791.500030:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 771001551007 time 771064328199 state 0x0 ref 1 00000100:00100000:3.0:1789760791.500043:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febd1360380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512770432:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760791.500091:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28987392, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0xe007 00010000:00010000:3.0:1789760791.500098:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c302ad lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.500115:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=771s v=(0 0) 00010000:00010000:3.0:1789760791.500119:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760791.500124:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c302ad lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28987392->28991487] (req 28987392->28991487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.500137:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c302ad lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28987392->28991487] (req 28987392->28991487) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760791.542336:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28987392 -> 28991488 00002000:00010000:3.0:1789760791.542346:0:6709:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760790 -> 1789760791 00002000:00010000:3.0:1789760791.542349:0:6709:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760790 -> 1789760791 00010000:00010000:3.0:1789760791.542374:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c302ad lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28987392->28991487] (req 28987392->28991487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.542388:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c302ad lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28987392->28991487] (req 28987392->28991487) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.542403:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c302ad lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28987392->28991487] (req 28987392->28991487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.542410:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760791.542415:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c302ad lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28987392->28991487] (req 28987392->28991487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.542423:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c302ad lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28987392->28991487] (req 28987392->28991487) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.542443:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c302ad lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28987392->28991487] (req 28987392->28991487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.542452:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c302ad lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28987392->28991487] (req 28987392->28991487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760791.542557:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febd1360380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2361:x1876699512770432:12345-192.168.203.56@tcp:4:dd.0 Request processed in 42511us (42815us total) trans 4294974396 rc 0/0 00000100:00100000:3.0:1789760791.542568:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7107 00000100:00100000:3.0:1789760791.542578:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 771001551007 time 771064328199 state 0x2 ref 0 00000001:00080000:1.0:1789760791.543154:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974396 is committed 00000100:00100000:2.0:1789760791.547177:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760791.547255:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512770560 00000100:00100000:3.0:1789760791.547266:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512770560 00000100:00100000:3.0:1789760791.547319:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 771001551007 time 771064328199 state 0x0 ref 1 00000100:00100000:3.0:1789760791.547334:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 771064588615 time 771064328199 state 0x1 ref 1 00000100:00100000:3.0:1789760791.547374:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 771064588615 time 771111674325 state 0x0 ref 1 00000100:00100000:3.0:1789760791.547387:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febc6bd4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512770560:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760791.547433:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28991488, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0xe007 00010000:00010000:3.0:1789760791.547441:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c302b4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.547456:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=771s v=(0 0) 00010000:00010000:3.0:1789760791.547460:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760791.547466:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c302b4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28991488->28995583] (req 28991488->28995583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.547480:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c302b4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28991488->28995583] (req 28991488->28995583) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760791.601287:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974397 is committed 00002000:00010000:0.0:1789760791.601410:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28991488 -> 28995584 00010000:00010000:0.0:1789760791.601425:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c302b4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28991488->28995583] (req 28991488->28995583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.601443:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c302b4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28991488->28995583] (req 28991488->28995583) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.601452:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c302b4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28991488->28995583] (req 28991488->28995583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.601458:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760791.601463:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c302b4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28991488->28995583] (req 28991488->28995583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.601471:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c302b4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28991488->28995583] (req 28991488->28995583) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.601490:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c302b4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28991488->28995583] (req 28991488->28995583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.601498:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c302b4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28991488->28995583] (req 28991488->28995583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760791.601602:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febc6bd4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512770560:12345-192.168.203.56@tcp:4:dd.0 Request processed in 54211us (54429us total) trans 4294974397 rc 0/0 00000100:00100000:0.0:1789760791.601612:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7108 00000100:00100000:0.0:1789760791.601622:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 771064588615 time 771111674325 state 0x2 ref 0 00000100:00100000:0.0:1789760791.605321:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760791.605603:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512770688 00000100:00100000:1.0:1789760791.605613:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512770688 00000100:00100000:1.0:1789760791.605877:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 771064588615 time 771111674325 state 0x0 ref 1 00000100:00100000:1.0:1789760791.605891:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 771111934741 time 771111674325 state 0x1 ref 1 00000100:00100000:1.0:1789760791.605918:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 771111934741 time 771170218478 state 0x0 ref 1 00000100:00100000:1.0:1789760791.605931:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac38b1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512770688:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760791.606037:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28995584, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0xe007 00010000:00010000:1.0:1789760791.606047:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c302bb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.606061:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=771s v=(0 0) 00010000:00010000:1.0:1789760791.606065:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760791.606070:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c302bb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28995584->28999679] (req 28995584->28999679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.606083:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c302bb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28995584->28999679] (req 28995584->28999679) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760791.655461:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974398 is committed 00002000:00010000:1.0:1789760791.655578:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28995584 -> 28999680 00010000:00010000:1.0:1789760791.655594:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c302bb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28995584->28999679] (req 28995584->28999679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.655613:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c302bb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28995584->28999679] (req 28995584->28999679) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.655623:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c302bb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28995584->28999679] (req 28995584->28999679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.655641:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760791.655647:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c302bb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28995584->28999679] (req 28995584->28999679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.655657:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c302bb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28995584->28999679] (req 28995584->28999679) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.655676:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c302bb lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28995584->28999679] (req 28995584->28999679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.655684:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c302bb lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28995584->28999679] (req 28995584->28999679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760791.655810:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac38b1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512770688:12345-192.168.203.56@tcp:4:dd.0 Request processed in 49877us (50620us total) trans 4294974398 rc 0/0 00000100:00100000:1.0:1789760791.655820:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7109 00000100:00100000:1.0:1789760791.655829:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 771111934741 time 771170218478 state 0x2 ref 0 00000100:00100000:1.0:1789760791.670870:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760791.670953:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512770816 00000100:00100000:3.0:1789760791.670970:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512770816 00000100:00100000:3.0:1789760791.671024:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 771111934741 time 771170218478 state 0x0 ref 1 00000100:00100000:3.0:1789760791.671040:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 771170478894 time 771170218478 state 0x1 ref 1 00000100:00100000:3.0:1789760791.671071:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 771170478894 time 771235370446 state 0x0 ref 1 00000100:00100000:3.0:1789760791.671083:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5306680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512770816:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760791.671129:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28999680, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0xe007 00010000:00010000:3.0:1789760791.671138:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c302c2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.671154:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=771s v=(0 0) 00010000:00010000:3.0:1789760791.671167:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760791.671173:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c302c2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28999680->29003775] (req 28999680->29003775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.671186:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c302c2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28999680->29003775] (req 28999680->29003775) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760791.715561:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974399 is committed 00002000:00010000:1.0:1789760791.715899:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28999680 -> 29003776 00010000:00010000:1.0:1789760791.715915:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c302c2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28999680->29003775] (req 28999680->29003775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.715929:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c302c2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28999680->29003775] (req 28999680->29003775) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.715938:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c302c2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28999680->29003775] (req 28999680->29003775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.715946:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760791.715951:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c302c2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28999680->29003775] (req 28999680->29003775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.715969:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c302c2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28999680->29003775] (req 28999680->29003775) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.715989:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c302c2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28999680->29003775] (req 28999680->29003775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.715997:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c302c2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28999680->29003775] (req 28999680->29003775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760791.716126:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5306680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512770816:12345-192.168.203.56@tcp:4:dd.0 Request processed in 45040us (45261us total) trans 4294974399 rc 0/0 00000100:00100000:1.0:1789760791.716138:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7110 00000100:00100000:1.0:1789760791.716148:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 771170478894 time 771235370446 state 0x2 ref 0 00000100:00100000:2.0:1789760791.719249:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760791.719419:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512770944 00000100:00100000:0.0:1789760791.719430:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512770944 00000100:00100000:0.0:1789760791.719483:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 771170478894 time 771235370446 state 0x0 ref 1 00000100:00100000:0.0:1789760791.719499:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 771235630862 time 771235370446 state 0x1 ref 1 00000100:00100000:0.0:1789760791.719568:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 771235630862 time 771283866754 state 0x0 ref 1 00000100:00100000:0.0:1789760791.719581:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2fb5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512770944:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760791.719640:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 29003776, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0xe007 00010000:00010000:0.0:1789760791.719649:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c302c9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.719665:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=771s v=(0 0) 00010000:00010000:0.0:1789760791.719669:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760791.719675:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c302c9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29003776->29007871] (req 29003776->29007871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.719921:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c302c9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29003776->29007871] (req 29003776->29007871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760791.758594:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 29003776 -> 29007872 00010000:00010000:2.0:1789760791.758616:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c302c9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29003776->29007871] (req 29003776->29007871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.758636:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c302c9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29003776->29007871] (req 29003776->29007871) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.758646:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c302c9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29003776->29007871] (req 29003776->29007871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.758655:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760791.758660:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c302c9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29003776->29007871] (req 29003776->29007871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.758669:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c302c9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29003776->29007871] (req 29003776->29007871) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.758687:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c302c9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29003776->29007871] (req 29003776->29007871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.758727:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c302c9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29003776->29007871] (req 29003776->29007871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760791.758814:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2fb5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2361:x1876699512770944:12345-192.168.203.56@tcp:4:dd.0 Request processed in 39229us (39569us total) trans 4294974400 rc 0/0 00000100:00100000:2.0:1789760791.758824:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7111 00000100:00100000:2.0:1789760791.758834:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 771235630862 time 771283866754 state 0x2 ref 0 00000001:00080000:1.0:1789760791.759312:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974400 is committed 00000100:00100000:0.0:1789760791.762305:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760791.762442:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512771072 00000100:00100000:1.0:1789760791.762452:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512771072 00000100:00100000:1.0:1789760791.762508:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 771235630862 time 771283866754 state 0x0 ref 1 00000100:00100000:1.0:1789760791.762523:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 771284127170 time 771283866754 state 0x1 ref 1 00000100:00100000:1.0:1789760791.762553:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 771284127170 time 771326852737 state 0x0 ref 1 00000100:00100000:1.0:1789760791.762566:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac38b0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512771072:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760791.762613:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 29007872, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0xe007 00010000:00010000:1.0:1789760791.762621:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c302d0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.762648:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=771s v=(0 0) 00010000:00010000:1.0:1789760791.762653:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760791.762659:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c302d0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29007872->29011967] (req 29007872->29011967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.762673:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c302d0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29007872->29011967] (req 29007872->29011967) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760791.810898:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974401 is committed 00002000:00010000:1.0:1789760791.811043:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 29007872 -> 29011968 00010000:00010000:1.0:1789760791.811058:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c302d0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29007872->29011967] (req 29007872->29011967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.811076:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c302d0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29007872->29011967] (req 29007872->29011967) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.811085:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c302d0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29007872->29011967] (req 29007872->29011967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.811092:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760791.811096:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c302d0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29007872->29011967] (req 29007872->29011967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.811105:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c302d0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29007872->29011967] (req 29007872->29011967) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.811123:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c302d0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29007872->29011967] (req 29007872->29011967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.811131:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c302d0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29007872->29011967] (req 29007872->29011967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760791.811248:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac38b0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512771072:12345-192.168.203.56@tcp:4:dd.0 Request processed in 48680us (48947us total) trans 4294974401 rc 0/0 00000100:00100000:1.0:1789760791.811257:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7112 00000100:00100000:1.0:1789760791.811266:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 771284127170 time 771326852737 state 0x2 ref 0 00000100:00100000:0.0:1789760791.815407:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760791.815944:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512771200 00000100:00100000:1.0:1789760791.815954:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512771200 00000100:00100000:1.0:1789760791.816015:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 771284127170 time 771326852737 state 0x0 ref 1 00000100:00100000:1.0:1789760791.816030:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 771327113153 time 771326852737 state 0x1 ref 1 00000100:00100000:1.0:1789760791.816094:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 771327113153 time 771380392447 state 0x0 ref 1 00000100:00100000:1.0:1789760791.816107:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac38b2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512771200:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760791.816153:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 29011968, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0xe007 00010000:00010000:1.0:1789760791.816162:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c302d7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.816176:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=771s v=(0 0) 00010000:00010000:1.0:1789760791.816181:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760791.816187:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c302d7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29011968->29016063] (req 29011968->29016063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.816201:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c302d7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29011968->29016063] (req 29011968->29016063) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760791.860047:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974402 is committed 00002000:00010000:1.0:1789760791.860206:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 29011968 -> 29016064 00010000:00010000:1.0:1789760791.860222:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c302d7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29011968->29016063] (req 29011968->29016063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.860234:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c302d7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29011968->29016063] (req 29011968->29016063) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.860250:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c302d7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29011968->29016063] (req 29011968->29016063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.860259:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760791.860263:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c302d7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29011968->29016063] (req 29011968->29016063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.860272:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c302d7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29011968->29016063] (req 29011968->29016063) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.860291:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c302d7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29011968->29016063] (req 29011968->29016063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.860298:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c302d7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29011968->29016063] (req 29011968->29016063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760791.860480:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac38b2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512771200:12345-192.168.203.56@tcp:4:dd.0 Request processed in 44371us (45079us total) trans 4294974402 rc 0/0 00000100:00100000:1.0:1789760791.860490:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7113 00000100:00100000:1.0:1789760791.860500:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 771327113153 time 771380392447 state 0x2 ref 0 00000100:00100000:2.0:1789760791.863990:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760791.864088:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512771328 00000100:00100000:0.0:1789760791.864098:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512771328 00000100:00100000:0.0:1789760791.864152:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 771327113153 time 771380392447 state 0x0 ref 1 00000100:00100000:0.0:1789760791.864167:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 771380652863 time 771380392447 state 0x1 ref 1 00000100:00100000:0.0:1789760791.864236:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 771380652863 time 771428534576 state 0x0 ref 1 00000100:00100000:0.0:1789760791.864249:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2fb6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512771328:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760791.864292:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 29016064, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0xe007 00010000:00010000:0.0:1789760791.864301:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c302de lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.864317:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=771s v=(0 0) 00010000:00010000:0.0:1789760791.864321:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760791.864328:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c302de lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29016064->29020159] (req 29016064->29020159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.864342:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c302de lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29016064->29020159] (req 29016064->29020159) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760791.914723:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 29016064 -> 29020160 00010000:00010000:3.0:1789760791.914745:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c302de lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29016064->29020159] (req 29016064->29020159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.914759:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c302de lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29016064->29020159] (req 29016064->29020159) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.914769:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c302de lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29016064->29020159] (req 29016064->29020159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.914777:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760791.914782:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c302de lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29016064->29020159] (req 29016064->29020159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.914790:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c302de lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29016064->29020159] (req 29016064->29020159) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.914816:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c302de lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29016064->29020159] (req 29016064->29020159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.914824:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c302de lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29016064->29020159] (req 29016064->29020159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760791.914928:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2fb6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2360:x1876699512771328:12345-192.168.203.56@tcp:4:dd.0 Request processed in 50676us (50943us total) trans 4294974403 rc 0/0 00000100:00100000:3.0:1789760791.914939:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7114 00000100:00100000:3.0:1789760791.914948:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 771380652863 time 771428534576 state 0x2 ref 0 00000001:00080000:0.0:1789760791.915035:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974403 is committed 00000100:00100000:0.0:1789760791.917638:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760791.917728:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512771456 00000100:00100000:1.0:1789760791.917738:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512771456 00000100:00100000:1.0:1789760791.917790:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 771380652863 time 771428534576 state 0x0 ref 1 00000100:00100000:1.0:1789760791.917806:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 771428794992 time 771428534576 state 0x1 ref 1 00000100:00100000:1.0:1789760791.917839:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 771428794992 time 771482139291 state 0x0 ref 1 00000100:00100000:1.0:1789760791.917851:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2746300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512771456:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760791.917898:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 29020160, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0xe007 00010000:00010000:1.0:1789760791.917907:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c302e5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.917921:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=771s v=(0 0) 00010000:00010000:1.0:1789760791.917925:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760791.917932:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c302e5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29020160->29024255] (req 29020160->29024255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.918081:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c302e5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29020160->29024255] (req 29020160->29024255) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.925671:0:5835:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:3.0:1789760791.983678:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974404 is committed 00002000:00010000:1.0:1789760791.983855:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 29020160 -> 29024256 00010000:00010000:1.0:1789760791.983870:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c302e5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29020160->29024255] (req 29020160->29024255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.983885:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c302e5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29020160->29024255] (req 29020160->29024255) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.983894:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c302e5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29020160->29024255] (req 29020160->29024255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.983907:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760791.983912:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c302e5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29020160->29024255] (req 29020160->29024255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.983921:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c302e5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29020160->29024255] (req 29020160->29024255) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.983937:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c302e5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29020160->29024255] (req 29020160->29024255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.983944:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c302e5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29020160->29024255] (req 29020160->29024255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760791.984066:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2746300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512771456:12345-192.168.203.56@tcp:4:dd.0 Request processed in 66213us (66432us total) trans 4294974404 rc 0/0 00000100:00100000:1.0:1789760791.984076:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7115 00000100:00100000:1.0:1789760791.984085:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 771428794992 time 771482139291 state 0x2 ref 0 00000100:00100000:1.0:1789760791.987463:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760791.987550:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512771584 00000100:00100000:3.0:1789760791.987560:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512771584 00000100:00100000:3.0:1789760791.987610:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 771428794992 time 771482139291 state 0x0 ref 1 00000100:00100000:3.0:1789760791.987623:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 771482399707 time 771482139291 state 0x1 ref 1 00000100:00100000:3.0:1789760791.987814:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 771482399707 time 771552112392 state 0x0 ref 1 00000100:00100000:3.0:1789760791.987828:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5307100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512771584:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760791.987877:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 29024256, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0xe007 00010000:00010000:3.0:1789760791.987887:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c302ec lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.987909:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=771s v=(0 0) 00010000:00010000:3.0:1789760791.987914:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760791.987921:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c302ec lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29024256->29028351] (req 29024256->29028351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.987935:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c302ec lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29024256->29028351] (req 29024256->29028351) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760792.017563:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 29024256 -> 29028352 00010000:00010000:3.0:1789760792.017583:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c302ec lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29024256->29028351] (req 29024256->29028351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.017597:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c302ec lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29024256->29028351] (req 29024256->29028351) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.017605:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c302ec lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29024256->29028351] (req 29024256->29028351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.017613:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760792.017618:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c302ec lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29024256->29028351] (req 29024256->29028351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.017635:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c302ec lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29024256->29028351] (req 29024256->29028351) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.017655:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c302ec lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29024256->29028351] (req 29024256->29028351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.017663:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c302ec lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29024256->29028351] (req 29024256->29028351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760792.017789:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5307100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2360:x1876699512771584:12345-192.168.203.56@tcp:4:dd.0 Request processed in 29959us (30329us total) trans 4294974405 rc 0/0 00000100:00100000:3.0:1789760792.017798:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7116 00000100:00100000:3.0:1789760792.017807:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 771482399707 time 771552112392 state 0x2 ref 0 00000001:00080000:1.0:1789760792.018369:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974405 is committed 00000100:00100000:2.0:1789760792.021341:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760792.021892:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512771712 00000100:00100000:3.0:1789760792.021904:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512771712 00000100:00100000:3.0:1789760792.021969:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 771482399707 time 771552112392 state 0x0 ref 1 00000100:00100000:3.0:1789760792.021986:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 771552372808 time 771552112392 state 0x1 ref 1 00000100:00100000:3.0:1789760792.022022:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 771552372808 time 771586321357 state 0x0 ref 1 00000100:00100000:3.0:1789760792.022035:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2fba680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512771712:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760792.022082:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 29028352, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0xe007 00010000:00010000:3.0:1789760792.022091:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c302f3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.022106:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=771s v=(0 0) 00010000:00010000:3.0:1789760792.022110:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760792.022117:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c302f3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29028352->29032447] (req 29028352->29032447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.022136:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c302f3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29028352->29032447] (req 29028352->29032447) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760792.050460:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974406 is committed 00002000:00010000:3.0:1789760792.050570:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 29028352 -> 29032448 00010000:00010000:3.0:1789760792.050585:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c302f3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29028352->29032447] (req 29028352->29032447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.050598:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c302f3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29028352->29032447] (req 29028352->29032447) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.050607:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c302f3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29028352->29032447] (req 29028352->29032447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.050613:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760792.050618:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c302f3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29028352->29032447] (req 29028352->29032447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.050639:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c302f3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29028352->29032447] (req 29028352->29032447) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.050660:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c302f3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29028352->29032447] (req 29028352->29032447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.050668:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c302f3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29028352->29032447] (req 29028352->29032447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760792.050866:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2fba680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512771712:12345-192.168.203.56@tcp:4:dd.0 Request processed in 28828us (29529us total) trans 4294974406 rc 0/0 00000100:00100000:3.0:1789760792.050877:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7117 00000100:00100000:3.0:1789760792.050886:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 771552372808 time 771586321357 state 0x2 ref 0 00000100:00100000:2.0:1789760792.053625:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760792.053743:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512771840 00000100:00100000:1.0:1789760792.053753:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512771840 00000100:00100000:1.0:1789760792.053811:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 771552372808 time 771586321357 state 0x0 ref 1 00000100:00100000:1.0:1789760792.053827:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 771586581773 time 771586321357 state 0x1 ref 1 00000100:00100000:1.0:1789760792.053862:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 771586581773 time 771618161098 state 0x0 ref 1 00000100:00100000:1.0:1789760792.053875:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2fb8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512771840:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760792.053922:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 29032448, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0xe007 00010000:00010000:1.0:1789760792.053931:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c302fa lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.053947:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=771s v=(0 0) 00010000:00010000:1.0:1789760792.053951:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760792.053968:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c302fa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29032448->29036543] (req 29032448->29036543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.053982:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c302fa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29032448->29036543] (req 29032448->29036543) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760792.083970:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974407 is committed 00002000:00010000:3.0:1789760792.084056:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 29032448 -> 29036544 00010000:00010000:3.0:1789760792.084070:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c302fa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29032448->29036543] (req 29032448->29036543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.084083:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c302fa lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29032448->29036543] (req 29032448->29036543) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.084093:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c302fa lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29032448->29036543] (req 29032448->29036543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.084100:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760792.084105:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c302fa lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29032448->29036543] (req 29032448->29036543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.084114:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c302fa lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29032448->29036543] (req 29032448->29036543) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.084132:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c302fa lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29032448->29036543] (req 29032448->29036543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.084144:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c302fa lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29032448->29036543] (req 29032448->29036543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760792.084272:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2fb8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512771840:12345-192.168.203.56@tcp:4:dd.0 Request processed in 30394us (30651us total) trans 4294974407 rc 0/0 00000100:00100000:3.0:1789760792.084285:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7118 00000100:00100000:3.0:1789760792.084294:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 771586581773 time 771618161098 state 0x2 ref 0 00000100:00100000:1.0:1789760792.088240:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760792.088330:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512771968 00000100:00100000:3.0:1789760792.088341:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512771968 00000100:00100000:3.0:1789760792.088459:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 771586581773 time 771618161098 state 0x0 ref 1 00000100:00100000:3.0:1789760792.088474:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 771618421514 time 771618161098 state 0x1 ref 1 00000100:00100000:3.0:1789760792.088555:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 771618421514 time 771652853952 state 0x0 ref 1 00000100:00100000:3.0:1789760792.088568:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5304a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512771968:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760792.088619:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 29036544, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0xe007 00010000:00010000:3.0:1789760792.088639:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c30301 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.088655:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=771s v=(0 0) 00010000:00010000:3.0:1789760792.088659:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760792.088665:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c30301 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29036544->29040639] (req 29036544->29040639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.088683:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c30301 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29036544->29040639] (req 29036544->29040639) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760792.118080:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974408 is committed 00002000:00010000:3.0:1789760792.118239:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 29036544 -> 29040640 00010000:00010000:3.0:1789760792.118257:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c30301 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29036544->29040639] (req 29036544->29040639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.118270:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c30301 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29036544->29040639] (req 29036544->29040639) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.118279:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c30301 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29036544->29040639] (req 29036544->29040639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.118287:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760792.118291:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c30301 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29036544->29040639] (req 29036544->29040639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.118301:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c30301 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29036544->29040639] (req 29036544->29040639) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.118318:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c30301 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29036544->29040639] (req 29036544->29040639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.118326:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c30301 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29036544->29040639] (req 29036544->29040639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760792.118529:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5304a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512771968:12345-192.168.203.56@tcp:4:dd.0 Request processed in 29957us (30294us total) trans 4294974408 rc 0/0 00000100:00100000:3.0:1789760792.118540:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7119 00000100:00100000:3.0:1789760792.118550:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 771618421514 time 771652853952 state 0x2 ref 0 00000100:00100000:2.0:1789760792.121951:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760792.122059:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512772096 00000100:00100000:3.0:1789760792.122068:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512772096 00000100:00100000:3.0:1789760792.122123:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 771618421514 time 771652853952 state 0x0 ref 1 00000100:00100000:3.0:1789760792.122138:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 771653114368 time 771652853952 state 0x1 ref 1 00000100:00100000:3.0:1789760792.122206:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 771653114368 time 771686505018 state 0x0 ref 1 00000100:00100000:3.0:1789760792.122218:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2fb8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512772096:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760792.122266:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 29040640, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0xe007 00010000:00010000:3.0:1789760792.122274:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c30308 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.122290:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=771s v=(0 0) 00010000:00010000:3.0:1789760792.122294:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760792.122300:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c30308 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29040640->29044735] (req 29040640->29044735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.122313:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c30308 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29040640->29044735] (req 29040640->29044735) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760792.151502:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974409 is committed 00002000:00010000:3.0:1789760792.151586:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 29040640 -> 29044736 00010000:00010000:3.0:1789760792.151603:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c30308 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29040640->29044735] (req 29040640->29044735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.151617:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c30308 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29040640->29044735] (req 29040640->29044735) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.151821:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c30308 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29040640->29044735] (req 29040640->29044735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.151831:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760792.151837:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c30308 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29040640->29044735] (req 29040640->29044735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.151847:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c30308 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29040640->29044735] (req 29040640->29044735) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.151868:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c30308 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29040640->29044735] (req 29040640->29044735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.151876:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c30308 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29040640->29044735] (req 29040640->29044735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760792.152015:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2fb8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512772096:12345-192.168.203.56@tcp:4:dd.0 Request processed in 29793us (30068us total) trans 4294974409 rc 0/0 00000100:00100000:3.0:1789760792.152030:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7120 00000100:00100000:3.0:1789760792.152039:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 771653114368 time 771686505018 state 0x2 ref 0 00000100:00100000:2.0:1789760792.155408:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760792.155490:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512772224 00000100:00100000:3.0:1789760792.155501:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512772224 00000100:00100000:3.0:1789760792.155554:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 771653114368 time 771686505018 state 0x0 ref 1 00000100:00100000:3.0:1789760792.155569:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 771686765434 time 771686505018 state 0x1 ref 1 00000100:00100000:3.0:1789760792.155602:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 771686765434 time 771719901965 state 0x0 ref 1 00000100:00100000:3.0:1789760792.155614:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4360a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512772224:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760792.155673:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 29044736, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0xe007 00010000:00010000:3.0:1789760792.155682:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c3030f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.155740:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=771s v=(0 0) 00010000:00010000:3.0:1789760792.155744:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760792.155749:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c3030f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29044736->29048831] (req 29044736->29048831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.155764:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c3030f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29044736->29048831] (req 29044736->29048831) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760792.186223:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 29044736 -> 29048832 00010000:00010000:0.0:1789760792.186245:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c3030f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29044736->29048831] (req 29044736->29048831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.186265:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c3030f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29044736->29048831] (req 29044736->29048831) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.186273:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c3030f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29044736->29048831] (req 29044736->29048831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.186280:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760792.186285:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c3030f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29044736->29048831] (req 29044736->29048831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.186293:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c3030f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29044736->29048831] (req 29044736->29048831) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.186312:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c3030f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29044736->29048831] (req 29044736->29048831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.186321:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c3030f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29044736->29048831] (req 29044736->29048831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760792.186432:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4360a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2358:x1876699512772224:12345-192.168.203.56@tcp:4:dd.0 Request processed in 30814us (31029us total) trans 4294974410 rc 0/0 00000100:00100000:0.0:1789760792.186442:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7121 00000100:00100000:0.0:1789760792.186451:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 771686765434 time 771719901965 state 0x2 ref 0 00000100:00100000:1.0:1789760792.189780:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760792.189974:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512772352 00000100:00100000:3.0:1789760792.189984:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512772352 00000100:00100000:3.0:1789760792.190039:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 771686765434 time 771719901965 state 0x0 ref 1 00000100:00100000:3.0:1789760792.190056:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 771720162381 time 771719901965 state 0x1 ref 1 00000100:00100000:3.0:1789760792.190087:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 771720162381 time 771754387386 state 0x0 ref 1 00000100:00100000:3.0:1789760792.190100:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5305880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2358:x1876699512772352:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760792.190147:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 29048832, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0xe007 00010000:00010000:3.0:1789760792.190156:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c30316 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.190172:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=771s v=(0 0) 00010000:00010000:3.0:1789760792.190178:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760792.190184:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c30316 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29048832->29052927] (req 29048832->29052927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.190199:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c30316 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29048832->29052927] (req 29048832->29052927) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760792.190906:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974410 is committed 00000001:00080000:1.0:1789760792.221840:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974411 is committed 00002000:00010000:3.0:1789760792.222094:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 29048832 -> 29052928 00010000:00010000:3.0:1789760792.222109:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c30316 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29048832->29052927] (req 29048832->29052927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.222122:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c30316 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29048832->29052927] (req 29048832->29052927) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.222131:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c30316 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29048832->29052927] (req 29048832->29052927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.222140:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760792.222145:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c30316 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29048832->29052927] (req 29048832->29052927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.222154:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c30316 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29048832->29052927] (req 29048832->29052927) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.222173:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c30316 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29048832->29052927] (req 29048832->29052927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.222181:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c30316 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29048832->29052927] (req 29048832->29052927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760792.222379:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5305880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512772352:12345-192.168.203.56@tcp:4:dd.0 Request processed in 32276us (32603us total) trans 4294974411 rc 0/0 00000100:00100000:3.0:1789760792.222390:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7122 00000100:00100000:3.0:1789760792.222400:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 771720162381 time 771754387386 state 0x2 ref 0 00000100:00100000:3.0:1789760792.225884:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760792.226298:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512772480 00000100:00100000:0.0:1789760792.226310:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512772480 00000100:00100000:0.0:1789760792.226801:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 771720162381 time 771754387386 state 0x0 ref 1 00000100:00100000:0.0:1789760792.226818:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 771754647802 time 771754387386 state 0x1 ref 1 00000100:00100000:0.0:1789760792.226851:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 771754647802 time 771791150704 state 0x0 ref 1 00000100:00100000:0.0:1789760792.226866:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4131f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512772480:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760792.226917:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 29052928, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0xe007 00010000:00010000:0.0:1789760792.226926:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c3031d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.226941:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=771s v=(0 0) 00010000:00010000:0.0:1789760792.226945:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760792.226952:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c3031d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29052928->29057023] (req 29052928->29057023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.226978:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c3031d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29052928->29057023] (req 29052928->29057023) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760792.252355:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974412 is committed 00002000:00010000:2.0:1789760792.252506:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 29052928 -> 29057024 00010000:00010000:2.0:1789760792.252523:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c3031d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29052928->29057023] (req 29052928->29057023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.252537:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c3031d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29052928->29057023] (req 29052928->29057023) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.252546:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c3031d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29052928->29057023] (req 29052928->29057023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.252552:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760792.252558:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c3031d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29052928->29057023] (req 29052928->29057023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.252567:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c3031d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29052928->29057023] (req 29052928->29057023) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.252586:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c3031d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29052928->29057023] (req 29052928->29057023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.252592:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c3031d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29052928->29057023] (req 29052928->29057023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760792.252836:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4131f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512772480:12345-192.168.203.56@tcp:4:dd.0 Request processed in 25968us (26956us total) trans 4294974412 rc 0/0 00000100:00100000:2.0:1789760792.252846:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7123 00000100:00100000:2.0:1789760792.252857:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 771754647802 time 771791150704 state 0x2 ref 0 00000100:00100000:2.0:1789760792.257043:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760792.257233:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512772608 00000100:00100000:0.0:1789760792.257243:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512772608 00000100:00100000:0.0:1789760792.257297:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 771754647802 time 771791150704 state 0x0 ref 1 00000100:00100000:0.0:1789760792.257313:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 771791411120 time 771791150704 state 0x1 ref 1 00000100:00100000:0.0:1789760792.257339:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 771791411120 time 771821638954 state 0x0 ref 1 00000100:00100000:0.0:1789760792.257357:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4362d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512772608:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760792.257405:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 29057024, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0xe007 00010000:00010000:0.0:1789760792.257415:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c30324 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.257433:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=771s v=(0 0) 00010000:00010000:0.0:1789760792.257437:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760792.257444:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c30324 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29057024->29061119] (req 29057024->29061119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.257458:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c30324 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29057024->29061119] (req 29057024->29061119) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760792.298513:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974413 is committed 00002000:00010000:2.0:1789760792.298918:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 29057024 -> 29061120 00010000:00010000:2.0:1789760792.298934:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c30324 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29057024->29061119] (req 29057024->29061119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.298947:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c30324 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29057024->29061119] (req 29057024->29061119) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.298966:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c30324 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29057024->29061119] (req 29057024->29061119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.298975:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760792.298980:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c30324 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29057024->29061119] (req 29057024->29061119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.298988:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c30324 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29057024->29061119] (req 29057024->29061119) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.299006:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c30324 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29057024->29061119] (req 29057024->29061119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.299014:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c30324 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29057024->29061119] (req 29057024->29061119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760792.299313:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4362d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512772608:12345-192.168.203.56@tcp:4:dd.0 Request processed in 41954us (42274us total) trans 4294974413 rc 0/0 00000100:00100000:2.0:1789760792.299325:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7124 00000100:00100000:2.0:1789760792.299335:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 771791411120 time 771821638954 state 0x2 ref 0 00000100:00100000:0.0:1789760792.305320:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760792.305486:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512772736 00000100:00100000:3.0:1789760792.305495:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512772736 00000100:00100000:3.0:1789760792.305550:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 771791411120 time 771821638954 state 0x0 ref 1 00000100:00100000:3.0:1789760792.305566:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 771821899370 time 771821638954 state 0x1 ref 1 00000100:00100000:3.0:1789760792.305607:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 771821899370 time 771869906675 state 0x0 ref 1 00000100:00100000:3.0:1789760792.305620:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2744a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512772736:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760792.305668:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 29061120, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0xe007 00010000:00010000:3.0:1789760792.305677:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c3032b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.305693:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=771s v=(0 0) 00010000:00010000:3.0:1789760792.305734:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760792.305740:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c3032b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29061120->29065215] (req 29061120->29065215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.305755:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c3032b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29061120->29065215] (req 29061120->29065215) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760792.374835:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974414 is committed 00002000:00010000:3.0:1789760792.376061:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 29061120 -> 29065216 00010000:00010000:3.0:1789760792.376199:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c3032b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29061120->29065215] (req 29061120->29065215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.376212:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c3032b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29061120->29065215] (req 29061120->29065215) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.376221:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c3032b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29061120->29065215] (req 29061120->29065215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.376229:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760792.376234:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c3032b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29061120->29065215] (req 29061120->29065215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.376244:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c3032b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29061120->29065215] (req 29061120->29065215) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.376263:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c3032b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29061120->29065215] (req 29061120->29065215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.376270:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c3032b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29061120->29065215] (req 29061120->29065215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760792.376423:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2744a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512772736:12345-192.168.203.56@tcp:4:dd.0 Request processed in 70800us (71109us total) trans 4294974414 rc 0/0 00000100:00100000:3.0:1789760792.376433:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7125 00000100:00100000:3.0:1789760792.376442:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 771821899370 time 771869906675 state 0x2 ref 0 00000100:00100000:3.0:1789760792.390276:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760792.391036:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512772864 00000100:00100000:2.0:1789760792.391047:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512772864 00000100:00100000:2.0:1789760792.391108:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 771821899370 time 771869906675 state 0x0 ref 1 00000100:00100000:2.0:1789760792.391124:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 771870167091 time 771869906675 state 0x1 ref 1 00000100:00100000:2.0:1789760792.391159:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 771870167091 time 771955458844 state 0x0 ref 1 00000100:00100000:2.0:1789760792.391172:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4130380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512772864:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760792.391222:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 29065216, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0xe007 00010000:00010000:2.0:1789760792.391231:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c30332 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.391247:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=771s v=(0 0) 00010000:00010000:2.0:1789760792.391251:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760792.391258:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c30332 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29065216->29069311] (req 29065216->29069311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.391271:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c30332 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29065216->29069311] (req 29065216->29069311) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760792.432744:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974415 is committed 00002000:00010000:3.0:1789760792.432995:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 29065216 -> 29069312 00010000:00010000:3.0:1789760792.433012:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c30332 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29065216->29069311] (req 29065216->29069311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.433023:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c30332 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29065216->29069311] (req 29065216->29069311) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.433032:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c30332 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29065216->29069311] (req 29065216->29069311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.433046:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760792.433052:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c30332 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29065216->29069311] (req 29065216->29069311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.433060:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c30332 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29065216->29069311] (req 29065216->29069311) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.433081:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c30332 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29065216->29069311] (req 29065216->29069311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.433090:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c30332 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29065216->29069311] (req 29065216->29069311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760792.433306:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4130380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512772864:12345-192.168.203.56@tcp:4:dd.0 Request processed in 42131us (43034us total) trans 4294974415 rc 0/0 00000100:00100000:3.0:1789760792.433316:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7126 00000100:00100000:3.0:1789760792.433326:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 771870167091 time 771955458844 state 0x2 ref 0 00000100:00100000:0.0:1789760792.443192:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760792.443344:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512772992 00000100:00100000:2.0:1789760792.443353:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512772992 00000100:00100000:2.0:1789760792.443406:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 771870167091 time 771955458844 state 0x0 ref 1 00000100:00100000:2.0:1789760792.443421:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 771955719260 time 771955458844 state 0x1 ref 1 00000100:00100000:2.0:1789760792.443448:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 771955719260 time 772007747878 state 0x0 ref 1 00000100:00100000:2.0:1789760792.443460:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2745f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512772992:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760792.443508:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 29069312, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0xe007 00010000:00010000:2.0:1789760792.443518:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c30339 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.443534:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=772s v=(0 0) 00010000:00010000:2.0:1789760792.443539:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760792.443546:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c30339 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29069312->29073407] (req 29069312->29073407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.443563:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c30339 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29069312->29073407] (req 29069312->29073407) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760792.495514:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974416 is committed 00002000:00010000:3.0:1789760792.495653:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 29069312 -> 29073408 00010000:00010000:3.0:1789760792.495669:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c30339 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29069312->29073407] (req 29069312->29073407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.495682:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c30339 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29069312->29073407] (req 29069312->29073407) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.495729:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c30339 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29069312->29073407] (req 29069312->29073407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.495738:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760792.495743:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c30339 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29069312->29073407] (req 29069312->29073407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.495753:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c30339 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29069312->29073407] (req 29069312->29073407) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.495772:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c30339 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29069312->29073407] (req 29069312->29073407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.495780:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c30339 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29069312->29073407] (req 29069312->29073407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760792.495864:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2745f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512772992:12345-192.168.203.56@tcp:4:dd.0 Request processed in 52401us (52746us total) trans 4294974416 rc 0/0 00000100:00100000:3.0:1789760792.495874:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7127 00000100:00100000:3.0:1789760792.495884:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 771955719260 time 772007747878 state 0x2 ref 0 00000100:00100000:3.0:1789760792.498912:0:3312:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8feac4130000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3312:1876699540233600:0@lo:400:kworker.0 00000100:00100000:3.0:1789760792.498994:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760792.499046:0:3312:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1789760792.499052:0:3312:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8feac4132680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3312:1876699540233472:0@lo:400:kworker.0 00000100:00100000:3.0:1789760792.499081:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760792.499112:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540233600 00000100:00100000:3.0:1789760792.499120:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540233600 00000100:00100000:3.0:1789760792.499313:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 197 00000100:00100000:3.0:1789760792.499326:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febfb61a450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3312:x1876699540233600:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760792.499378:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febfb61a450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3312:x1876699540233600:12345-0@lo:400:kworker.0 Request processed in 56us (383us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760792.499386:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 197 00000100:00100000:3.0:1789760792.499615:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540233472 00000100:00100000:3.0:1789760792.499619:0:5847:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540233472 00000100:00100000:3.0:1789760792.499662:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 521 00000100:00100000:3.0:1789760792.499670:0:5847:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febfb8d4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3312:x1876699540233472:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760792.499726:0:5847:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febfb8d4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3312:x1876699540233472:12345-0@lo:400:kworker.0 Request processed in 57us (644us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760792.499733:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 521 00000100:00080000:3.0:1789760792.499748:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:3.0:1789760792.499752:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1789760792.499782:0:3312:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8feac4132680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3312:1876699540233472:0@lo:400:kworker.0 00000100:00100000:3.0:1789760792.499793:0:3312:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8feac4130000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3312:1876699540233600:0@lo:400:kworker.0 00000100:00100000:1.0:1789760792.500108:0:3309:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8feac4133b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:6df98d54-eb52-4c5c-8fc9-ee9d74f90f98:3309:1876699540233216:0@lo:400:kworker.0 00000100:00100000:1.0:1789760792.500280:0:3309:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760792.500315:0:3309:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8feac2875c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3309:1876699540233728:0@lo:400:kworker.0 00000100:00100000:1.0:1789760792.500341:0:3309:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760792.500359:0:3309:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:1.0:1789760792.500365:0:3309:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8feac4133800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3309:1876699540233344:0@lo:400:kworker.0 00000100:00100000:1.0:1789760792.500385:0:3309:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760792.500401:0:3309:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8feac2877100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3309:1876699540233856:0@lo:400:kworker.0 00000100:00100000:1.0:1789760792.500432:0:3309:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760792.500459:0:10102:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540233856 00000100:00100000:1.0:1789760792.500463:0:10102:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540233856 00000100:00100000:1.0:1789760792.500492:0:10102:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 198 00000100:00100000:1.0:1789760792.500497:0:10102:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5305180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3309:x1876699540233856:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760792.500531:0:10102:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5305180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3309:x1876699540233856:12345-0@lo:400:kworker.0 Request processed in 34us (98us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760792.500537:0:10102:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 198 00000100:00100000:1.0:1789760792.500559:0:3309:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8feac2877100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3309:1876699540233856:0@lo:400:kworker.0 00000100:00100000:2.0:1789760792.500651:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540233216 00000100:00100000:2.0:1789760792.500655:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540233216 00000100:00100000:2.0:1789760792.500687:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 417 00000100:00100000:2.0:1789760792.500716:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febe00c4850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:6df98d54-eb52-4c5c-8fc9-ee9d74f90f98+19:3309:x1876699540233216:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760792.500754:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febe00c4850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:6df98d54-eb52-4c5c-8fc9-ee9d74f90f98+19:3309:x1876699540233216:12345-0@lo:400:kworker.0 Request processed in 60us (609us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760792.500759:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 417 00000100:00100000:2.0:1789760792.500785:0:8229:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540233728 00000100:00100000:2.0:1789760792.500789:0:8229:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540233728 00000100:00100000:2.0:1789760792.500810:0:8229:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540233344 00000100:00100000:2.0:1789760792.500812:0:8229:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540233344 00000100:00100000:2.0:1789760792.500827:0:8229:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 522 00000100:00100000:2.0:1789760792.500832:0:8229:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5306300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3309:x1876699540233728:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760792.500861:0:8229:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5306300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3309:x1876699540233728:12345-0@lo:400:kworker.0 Request processed in 29us (519us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760792.500867:0:8229:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 522 00000100:00100000:2.0:1789760792.500879:0:8229:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 523 00000100:00100000:2.0:1789760792.500885:0:8229:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5307800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3309:x1876699540233344:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760792.500911:0:8229:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5307800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3309:x1876699540233344:12345-0@lo:400:kworker.0 Request processed in 26us (524us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760792.500916:0:8229:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 523 00000100:00100000:2.0:1789760792.500944:0:3309:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8feac4133800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3309:1876699540233344:0@lo:400:kworker.0 00000100:00100000:2.0:1789760792.500963:0:3309:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8feac4133b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:6df98d54-eb52-4c5c-8fc9-ee9d74f90f98:3309:1876699540233216:0@lo:400:kworker.0 00000100:00100000:2.0:1789760792.500974:0:3309:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8feac2875c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3309:1876699540233728:0@lo:400:kworker.0 00000100:00080000:3.0:1789760792.501372:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:3.0:1789760792.501377:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1789760792.501388:0:3311:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8feac4362a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3311:1876699540233984:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1789760792.501432:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760792.501752:0:6706:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540233984 00000100:00100000:2.0:1789760792.501758:0:6706:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540233984 00000100:00100000:2.0:1789760792.501791:0:6706:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 216 00000100:00100000:2.0:1789760792.501799:0:6706:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febfb8d6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3311:x1876699540233984:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1789760792.501815:0:6706:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 526843904 pending 0 free 3829399552 avail 3826253824 00000100:00100000:2.0:1789760792.501853:0:6706:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febfb8d6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3311:x1876699540233984:12345-0@lo:13:osp-pre-0-0.0 Request processed in 54us (420us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760792.501859:0:6706:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 216 00000100:00100000:3.0:1789760792.502007:0:3311:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8feac4362a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3311:1876699540233984:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789760792.502321:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760792.502380:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512773120 00000100:00100000:3.0:1789760792.502385:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512773120 00000100:00100000:3.0:1789760792.502431:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 771955719260 time 772007747878 state 0x0 ref 1 00000100:00100000:3.0:1789760792.502444:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 772008008294 time 772007747878 state 0x1 ref 1 00000100:00100000:3.0:1789760792.502471:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 772008008294 time 772066770285 state 0x0 ref 1 00000100:00100000:3.0:1789760792.502482:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4363100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512773120:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760792.502523:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 29073408, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0xe007 00010000:00010000:3.0:1789760792.502532:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c30340 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.502550:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=772s v=(0 0) 00010000:00010000:3.0:1789760792.502554:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760792.502561:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c30340 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29073408->29077503] (req 29073408->29077503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.502576:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c30340 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29073408->29077503] (req 29073408->29077503) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760792.563121:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 29073408 -> 29077504 00002000:00010000:3.0:1789760792.563131:0:6709:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760791 -> 1789760792 00002000:00010000:3.0:1789760792.563133:0:6709:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760791 -> 1789760792 00010000:00010000:3.0:1789760792.563147:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c30340 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29073408->29077503] (req 29073408->29077503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.563160:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c30340 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29073408->29077503] (req 29073408->29077503) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.563175:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c30340 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29073408->29077503] (req 29073408->29077503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.563183:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760792.563188:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c30340 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29073408->29077503] (req 29073408->29077503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.563198:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c30340 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29073408->29077503] (req 29073408->29077503) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.563217:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c30340 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29073408->29077503] (req 29073408->29077503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.563224:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c30340 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29073408->29077503] (req 29073408->29077503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760792.563318:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4363100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2360:x1876699512773120:12345-192.168.203.56@tcp:4:dd.0 Request processed in 60833us (60994us total) trans 4294974417 rc 0/0 00000100:00100000:3.0:1789760792.563329:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7128 00000100:00100000:3.0:1789760792.563339:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 772008008294 time 772066770285 state 0x2 ref 0 00000001:00080000:1.0:1789760792.563424:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974417 is committed 00000100:00100000:1.0:1789760792.566200:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760792.566304:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512773248 00000100:00100000:3.0:1789760792.566315:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512773248 00000100:00100000:3.0:1789760792.566368:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 772008008294 time 772066770285 state 0x0 ref 1 00000100:00100000:3.0:1789760792.566383:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 772067030701 time 772066770285 state 0x1 ref 1 00000100:00100000:3.0:1789760792.566459:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 772067030701 time 772130758579 state 0x0 ref 1 00000100:00100000:3.0:1789760792.566474:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5305f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512773248:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760792.566519:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 29077504, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0xe007 00010000:00010000:3.0:1789760792.566528:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c30347 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.566543:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=772s v=(0 0) 00010000:00010000:3.0:1789760792.566548:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760792.566554:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c30347 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29077504->29081599] (req 29077504->29081599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.566568:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c30347 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29077504->29081599] (req 29077504->29081599) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760792.595670:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974418 is committed 00002000:00010000:3.0:1789760792.595947:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 29077504 -> 29081600 00010000:00010000:3.0:1789760792.595969:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c30347 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29077504->29081599] (req 29077504->29081599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.595983:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c30347 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29077504->29081599] (req 29077504->29081599) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.595992:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c30347 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29077504->29081599] (req 29077504->29081599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.595999:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760792.596005:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c30347 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29077504->29081599] (req 29077504->29081599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.596015:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c30347 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29077504->29081599] (req 29077504->29081599) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.596034:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c30347 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29077504->29081599] (req 29077504->29081599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.596043:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c30347 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29077504->29081599] (req 29077504->29081599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760792.596133:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5305f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512773248:12345-192.168.203.56@tcp:4:dd.0 Request processed in 29657us (29937us total) trans 4294974418 rc 0/0 00000100:00100000:3.0:1789760792.596143:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7129 00000100:00100000:3.0:1789760792.596155:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 772067030701 time 772130758579 state 0x2 ref 0 00000100:00100000:2.0:1789760792.600194:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760792.600362:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512773376 00000100:00100000:3.0:1789760792.600373:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512773376 00000100:00100000:3.0:1789760792.600439:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 772067030701 time 772130758579 state 0x0 ref 1 00000100:00100000:3.0:1789760792.600454:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 772131018995 time 772130758579 state 0x1 ref 1 00000100:00100000:3.0:1789760792.600486:0:6708:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 772131018995 time 772164786163 state 0x0 ref 1 00000100:00100000:3.0:1789760792.600499:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4362a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512773376:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760792.600545:0:6708:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 29081600, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0xe007 00010000:00010000:3.0:1789760792.600554:0:6708:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c3034e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.600579:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=772s v=(0 0) 00010000:00010000:3.0:1789760792.600584:0:6708:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760792.600591:0:6708:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c3034e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29081600->29085695] (req 29081600->29085695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.600604:0:6708:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c3034e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29081600->29085695] (req 29081600->29085695) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760792.646103:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974419 is committed 00002000:00010000:3.0:1789760792.646388:0:6708:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 29081600 -> 29085696 00010000:00010000:3.0:1789760792.646404:0:6708:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c3034e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29081600->29085695] (req 29081600->29085695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.646417:0:6708:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c3034e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29081600->29085695] (req 29081600->29085695) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.646425:0:6708:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c3034e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29081600->29085695] (req 29081600->29085695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.646431:0:6708:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760792.646435:0:6708:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c3034e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29081600->29085695] (req 29081600->29085695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.646444:0:6708:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c3034e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29081600->29085695] (req 29081600->29085695) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.646466:0:6708:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c3034e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29081600->29085695] (req 29081600->29085695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.646475:0:6708:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c3034e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29081600->29085695] (req 29081600->29085695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6708 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760792.646560:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4362a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512773376:12345-192.168.203.56@tcp:4:dd.0 Request processed in 46060us (46372us total) trans 4294974419 rc 0/0 00000100:00100000:3.0:1789760792.646567:0:6708:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7130 00000100:00100000:3.0:1789760792.646575:0:6708:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 772131018995 time 772164786163 state 0x2 ref 0 00000100:00100000:2.0:1789760792.648972:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760792.649263:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512773504 00000100:00100000:3.0:1789760792.649273:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512773504 00000100:00100000:3.0:1789760792.649330:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 772131018995 time 772164786163 state 0x0 ref 1 00000100:00100000:3.0:1789760792.649405:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 772165046579 time 772164786163 state 0x1 ref 1 00000100:00100000:3.0:1789760792.649439:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 772165046579 time 772213738581 state 0x0 ref 1 00000100:00100000:3.0:1789760792.649452:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4360700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512773504:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760792.649497:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 29085696, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0xe007 00010000:00010000:3.0:1789760792.649505:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c30355 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.649520:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=772s v=(0 0) 00010000:00010000:3.0:1789760792.649524:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760792.649530:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c30355 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29085696->29089791] (req 29085696->29089791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.649544:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c30355 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29085696->29089791] (req 29085696->29089791) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760792.674471:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974420 is committed 00002000:00010000:3.0:1789760792.674720:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 29085696 -> 29089792 00010000:00010000:3.0:1789760792.674736:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c30355 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29085696->29089791] (req 29085696->29089791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.674750:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c30355 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29085696->29089791] (req 29085696->29089791) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.674759:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c30355 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29085696->29089791] (req 29085696->29089791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.674767:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760792.674771:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c30355 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29085696->29089791] (req 29085696->29089791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.674781:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c30355 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29085696->29089791] (req 29085696->29089791) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.674798:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c30355 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29085696->29089791] (req 29085696->29089791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.674806:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c30355 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29085696->29089791] (req 29085696->29089791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760792.674925:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4360700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512773504:12345-192.168.203.56@tcp:4:dd.0 Request processed in 25471us (25966us total) trans 4294974420 rc 0/0 00000100:00100000:3.0:1789760792.674934:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7131 00000100:00100000:3.0:1789760792.674943:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 772165046579 time 772213738581 state 0x2 ref 0 00000100:00100000:0.0:1789760792.678458:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760792.678879:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512773632 00000100:00100000:3.0:1789760792.678889:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512773632 00000100:00100000:3.0:1789760792.678946:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 772165046579 time 772213738581 state 0x0 ref 1 00000100:00100000:3.0:1789760792.678971:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 772213998997 time 772213738581 state 0x1 ref 1 00000100:00100000:3.0:1789760792.679035:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 772213998997 time 772243334393 state 0x0 ref 1 00000100:00100000:3.0:1789760792.679049:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2745500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512773632:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760792.679097:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 29089792, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0xe007 00010000:00010000:3.0:1789760792.679107:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c3035c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.679123:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=772s v=(0 0) 00010000:00010000:3.0:1789760792.679127:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760792.679134:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c3035c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29089792->29093887] (req 29089792->29093887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.679148:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c3035c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29089792->29093887] (req 29089792->29093887) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760792.713208:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974421 is committed 00002000:00010000:2.0:1789760792.714224:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 29089792 -> 29093888 00010000:00010000:2.0:1789760792.714240:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c3035c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29089792->29093887] (req 29089792->29093887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.714252:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c3035c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29089792->29093887] (req 29089792->29093887) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.714261:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c3035c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29089792->29093887] (req 29089792->29093887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.714269:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760792.714274:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c3035c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29089792->29093887] (req 29089792->29093887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.714282:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c3035c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29089792->29093887] (req 29089792->29093887) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.714302:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c3035c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29089792->29093887] (req 29089792->29093887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.714309:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c3035c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29089792->29093887] (req 29089792->29093887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760792.714422:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2745500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512773632:12345-192.168.203.56@tcp:4:dd.0 Request processed in 35371us (35968us total) trans 4294974421 rc 0/0 00000100:00100000:2.0:1789760792.714432:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7132 00000100:00100000:2.0:1789760792.714442:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 772213998997 time 772243334393 state 0x2 ref 0 00000100:00100000:2.0:1789760792.717465:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760792.717610:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512773760 00000100:00100000:3.0:1789760792.717684:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512773760 00000100:00100000:3.0:1789760792.717770:0:6708:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 772213998997 time 772243334393 state 0x0 ref 1 00000100:00100000:3.0:1789760792.717786:0:6708:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 772243594809 time 772243334393 state 0x1 ref 1 00000100:00100000:1.0:1789760792.717915:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 772243594809 time 772282211627 state 0x0 ref 1 00000100:00100000:1.0:1789760792.717932:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febfb89ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512773760:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760792.717991:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 29093888, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0xe007 00010000:00010000:1.0:1789760792.718001:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c30363 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.718016:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=772s v=(0 0) 00010000:00010000:1.0:1789760792.718020:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760792.718027:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c30363 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29093888->29097983] (req 29093888->29097983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.718156:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c30363 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29093888->29097983] (req 29093888->29097983) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760792.751690:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974422 is committed 00002000:00010000:3.0:1789760792.751872:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 29093888 -> 29097984 00010000:00010000:3.0:1789760792.751888:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c30363 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29093888->29097983] (req 29093888->29097983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.751902:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c30363 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29093888->29097983] (req 29093888->29097983) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.751912:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c30363 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29093888->29097983] (req 29093888->29097983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.751920:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760792.751925:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c30363 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29093888->29097983] (req 29093888->29097983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.751934:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c30363 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29093888->29097983] (req 29093888->29097983) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.752027:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c30363 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29093888->29097983] (req 29093888->29097983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.752041:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c30363 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29093888->29097983] (req 29093888->29097983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760792.752205:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febfb89ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512773760:12345-192.168.203.56@tcp:4:dd.0 Request processed in 34273us (34746us total) trans 4294974422 rc 0/0 00000100:00100000:3.0:1789760792.752216:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7133 00000100:00100000:3.0:1789760792.752226:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 772243594809 time 772282211627 state 0x2 ref 0 00000100:00100000:0.0:1789760792.755775:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760792.755840:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512773888 00000100:00100000:3.0:1789760792.755851:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512773888 00000100:00100000:3.0:1789760792.755903:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 772243594809 time 772282211627 state 0x0 ref 1 00000100:00100000:3.0:1789760792.755923:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 772282472043 time 772282211627 state 0x1 ref 1 00000100:00100000:3.0:1789760792.755969:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 772282472043 time 772320269107 state 0x0 ref 1 00000100:00100000:3.0:1789760792.755983:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2744e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512773888:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760792.756031:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 29097984, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0xe007 00010000:00010000:3.0:1789760792.756041:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c3036a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.756065:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=772s v=(0 0) 00010000:00010000:3.0:1789760792.756070:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760792.756077:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c3036a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29097984->29102079] (req 29097984->29102079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.756100:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c3036a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29097984->29102079] (req 29097984->29102079) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760792.795378:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974423 is committed 00002000:00010000:3.0:1789760792.795438:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 29097984 -> 29102080 00010000:00010000:3.0:1789760792.795453:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c3036a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29097984->29102079] (req 29097984->29102079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.795465:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c3036a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29097984->29102079] (req 29097984->29102079) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.795475:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c3036a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29097984->29102079] (req 29097984->29102079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.795482:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760792.795487:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c3036a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29097984->29102079] (req 29097984->29102079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.795496:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c3036a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29097984->29102079] (req 29097984->29102079) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.795515:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c3036a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29097984->29102079] (req 29097984->29102079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.795522:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c3036a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29097984->29102079] (req 29097984->29102079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760792.795618:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2744e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512773888:12345-192.168.203.56@tcp:4:dd.0 Request processed in 39633us (39845us total) trans 4294974423 rc 0/0 00000100:00100000:3.0:1789760792.795636:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7134 00000100:00100000:3.0:1789760792.795647:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 772282472043 time 772320269107 state 0x2 ref 0 00000100:00100000:1.0:1789760792.798967:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760792.799051:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512774016 00000100:00100000:2.0:1789760792.799061:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512774016 00000100:00100000:2.0:1789760792.799114:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 772282472043 time 772320269107 state 0x0 ref 1 00000100:00100000:2.0:1789760792.799129:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 772320529523 time 772320269107 state 0x1 ref 1 00000100:00100000:2.0:1789760792.799364:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 772320529523 time 772363651310 state 0x0 ref 1 00000100:00100000:2.0:1789760792.799378:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5307480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512774016:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760792.799425:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 29102080, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0xe007 00010000:00010000:2.0:1789760792.799434:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c30371 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.799451:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=772s v=(0 0) 00010000:00010000:2.0:1789760792.799455:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760792.799462:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c30371 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29102080->29106175] (req 29102080->29106175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.799482:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c30371 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29102080->29106175] (req 29102080->29106175) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760792.844619:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974424 is committed 00002000:00010000:3.0:1789760792.844678:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 29102080 -> 29106176 00010000:00010000:3.0:1789760792.844718:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c30371 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29102080->29106175] (req 29102080->29106175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.844732:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c30371 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29102080->29106175] (req 29102080->29106175) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.844741:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c30371 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29102080->29106175] (req 29102080->29106175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.844748:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760792.844753:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c30371 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29102080->29106175] (req 29102080->29106175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.844762:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c30371 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29102080->29106175] (req 29102080->29106175) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.844779:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c30371 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29102080->29106175] (req 29102080->29106175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.844791:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c30371 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29102080->29106175] (req 29102080->29106175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760792.844871:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5307480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512774016:12345-192.168.203.56@tcp:4:dd.0 Request processed in 45492us (45928us total) trans 4294974424 rc 0/0 00000100:00100000:3.0:1789760792.844881:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7135 00000100:00100000:3.0:1789760792.844890:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 772320529523 time 772363651310 state 0x2 ref 0 00000100:00100000:2.0:1789760792.848170:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760792.848321:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512774144 00000100:00100000:3.0:1789760792.848332:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512774144 00000100:00100000:3.0:1789760792.848388:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 772320529523 time 772363651310 state 0x0 ref 1 00000100:00100000:3.0:1789760792.848405:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 772363911726 time 772363651310 state 0x1 ref 1 00000100:00100000:3.0:1789760792.848433:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 772363911726 time 772412732471 state 0x0 ref 1 00000100:00100000:3.0:1789760792.848448:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febfb89fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512774144:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760792.848501:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 29106176, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0xe007 00010000:00010000:3.0:1789760792.848510:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c30378 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.848527:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=772s v=(0 0) 00010000:00010000:3.0:1789760792.848532:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760792.848538:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c30378 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29106176->29110271] (req 29106176->29110271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.848557:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c30378 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29106176->29110271] (req 29106176->29110271) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760792.906018:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974425 is committed 00002000:00010000:3.0:1789760792.906190:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 29106176 -> 29110272 00010000:00010000:3.0:1789760792.906206:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c30378 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29106176->29110271] (req 29106176->29110271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.906219:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c30378 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29106176->29110271] (req 29106176->29110271) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.906229:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c30378 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29106176->29110271] (req 29106176->29110271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.906237:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760792.906242:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c30378 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29106176->29110271] (req 29106176->29110271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.906251:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c30378 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29106176->29110271] (req 29106176->29110271) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.906270:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c30378 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29106176->29110271] (req 29106176->29110271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.906277:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c30378 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29106176->29110271] (req 29106176->29110271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760792.906443:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febfb89fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512774144:12345-192.168.203.56@tcp:4:dd.0 Request processed in 57993us (58278us total) trans 4294974425 rc 0/0 00000100:00100000:3.0:1789760792.906454:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7136 00000100:00100000:3.0:1789760792.906464:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 772363911726 time 772412732471 state 0x2 ref 0 00000100:00100000:0.0:1789760792.908966:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760792.909041:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512774272 00000100:00100000:3.0:1789760792.909051:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512774272 00000100:00100000:3.0:1789760792.909104:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 772363911726 time 772412732471 state 0x0 ref 1 00000100:00100000:3.0:1789760792.909119:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 772412992887 time 772412732471 state 0x1 ref 1 00000100:00100000:3.0:1789760792.909149:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 772412992887 time 772473448987 state 0x0 ref 1 00000100:00100000:3.0:1789760792.909162:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2747100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512774272:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760792.909208:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 29110272, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0xe007 00010000:00010000:3.0:1789760792.909217:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c3037f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.909233:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=772s v=(0 0) 00010000:00010000:3.0:1789760792.909237:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760792.909245:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c3037f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29110272->29114367] (req 29110272->29114367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.909258:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c3037f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29110272->29114367] (req 29110272->29114367) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760792.945039:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 29110272 -> 29114368 00010000:00010000:0.0:1789760792.945060:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c3037f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29110272->29114367] (req 29110272->29114367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.945074:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c3037f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29110272->29114367] (req 29110272->29114367) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.945088:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c3037f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29110272->29114367] (req 29110272->29114367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.945095:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760792.945100:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c3037f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29110272->29114367] (req 29110272->29114367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.945109:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c3037f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29110272->29114367] (req 29110272->29114367) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.945127:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c3037f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29110272->29114367] (req 29110272->29114367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.945134:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c3037f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29110272->29114367] (req 29110272->29114367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760792.945223:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2747100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2360:x1876699512774272:12345-192.168.203.56@tcp:4:dd.0 Request processed in 36058us (36269us total) trans 4294974426 rc 0/0 00000100:00100000:0.0:1789760792.945233:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7137 00000100:00100000:0.0:1789760792.945242:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 772412992887 time 772473448987 state 0x2 ref 0 00000001:00080000:0.0:1789760792.945285:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974426 is committed 00010000:00010000:2.0:1789760792.947058:0:7647:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789760792.954230:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760792.954293:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512774400 00000100:00100000:3.0:1789760792.954303:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512774400 00000100:00100000:3.0:1789760792.954380:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 772412992887 time 772473448987 state 0x0 ref 1 00000100:00100000:3.0:1789760792.954396:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 772473709403 time 772473448987 state 0x1 ref 1 00000100:00100000:3.0:1789760792.954426:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 772473709403 time 772518726040 state 0x0 ref 1 00000100:00100000:3.0:1789760792.954441:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac531f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512774400:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760792.954493:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 29114368, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0xe007 00010000:00010000:3.0:1789760792.954502:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c30386 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.954519:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=772s v=(0 0) 00010000:00010000:3.0:1789760792.954523:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760792.954530:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c30386 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29114368->29118463] (req 29114368->29118463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.954545:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c30386 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29114368->29118463] (req 29114368->29118463) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760792.997598:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 29114368 -> 29118464 00010000:00010000:3.0:1789760792.997617:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c30386 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29114368->29118463] (req 29114368->29118463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.997635:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c30386 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29114368->29118463] (req 29114368->29118463) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.997643:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c30386 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29114368->29118463] (req 29114368->29118463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.997650:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760792.997656:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c30386 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29114368->29118463] (req 29114368->29118463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.997664:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c30386 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29114368->29118463] (req 29114368->29118463) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.997684:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c30386 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29114368->29118463] (req 29114368->29118463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.997692:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c30386 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29114368->29118463] (req 29114368->29118463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760792.997801:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac531f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2361:x1876699512774400:12345-192.168.203.56@tcp:4:dd.0 Request processed in 43359us (43576us total) trans 4294974427 rc 0/0 00000100:00100000:3.0:1789760792.997811:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7138 00000100:00100000:3.0:1789760792.997821:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 772473709403 time 772518726040 state 0x2 ref 0 00000001:00080000:1.0:1789760792.997890:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974427 is committed 00000100:00100000:2.0:1789760793.000936:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760793.001027:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512774528 00000100:00100000:3.0:1789760793.001037:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512774528 00000100:00100000:3.0:1789760793.001106:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 772473709403 time 772518726040 state 0x0 ref 1 00000100:00100000:3.0:1789760793.001121:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 772518986456 time 772518726040 state 0x1 ref 1 00000100:00100000:3.0:1789760793.001157:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 772518986456 time 772565457445 state 0x0 ref 1 00000100:00100000:3.0:1789760793.001170:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac287b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512774528:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760793.001218:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 29118464, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0xe007 00010000:00010000:3.0:1789760793.001227:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c3038d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.001242:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=772s v=(0 0) 00010000:00010000:3.0:1789760793.001246:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760793.001252:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c3038d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29118464->29122559] (req 29118464->29122559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.001266:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c3038d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29118464->29122559] (req 29118464->29122559) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1789760793.002266:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1789760793.012321:0:3312:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8feac2745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3312:1876699540234112:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789760793.012388:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760793.012521:0:6706:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540234112 00000100:00100000:1.0:1789760793.012529:0:6706:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540234112 00000100:00100000:1.0:1789760793.012592:0:6706:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 217 00000100:00100000:1.0:1789760793.012602:0:6706:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac287aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3312:x1876699540234112:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1789760793.012636:0:6706:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3859808256 avail 3857711104 00000100:00100000:1.0:1789760793.015315:0:6706:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac287aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3312:x1876699540234112:12345-0@lo:13:osp-pre-1-0.0 Request processed in 2699us (2911us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760793.015315:0:3312:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8feac2745880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3312:1876699540234112:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1789760793.015326:0:6706:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 217 00000001:00080000:1.0:1789760793.036435:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974428 is committed 00002000:00010000:3.0:1789760793.036556:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 29118464 -> 29122560 00010000:00010000:3.0:1789760793.036571:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c3038d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29118464->29122559] (req 29118464->29122559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.036584:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c3038d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29118464->29122559] (req 29118464->29122559) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.036597:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c3038d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29118464->29122559] (req 29118464->29122559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.036603:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760793.036608:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c3038d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29118464->29122559] (req 29118464->29122559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.036617:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c3038d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29118464->29122559] (req 29118464->29122559) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.036723:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c3038d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29118464->29122559] (req 29118464->29122559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.036732:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c3038d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29118464->29122559] (req 29118464->29122559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760793.036963:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac287b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512774528:12345-192.168.203.56@tcp:4:dd.0 Request processed in 35782us (36022us total) trans 4294974428 rc 0/0 00000100:00100000:3.0:1789760793.036974:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7139 00000100:00100000:3.0:1789760793.036984:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 772518986456 time 772565457445 state 0x2 ref 0 00000100:00100000:0.0:1789760793.041505:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760793.041608:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512774656 00000100:00100000:3.0:1789760793.041618:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512774656 00000100:00100000:3.0:1789760793.041835:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 772518986456 time 772565457445 state 0x0 ref 1 00000100:00100000:3.0:1789760793.041913:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 772565717861 time 772565457445 state 0x1 ref 1 00000100:00100000:3.0:1789760793.041988:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 772565717861 time 772606286316 state 0x0 ref 1 00000100:00100000:3.0:1789760793.042002:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2745880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512774656:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760793.042051:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 29122560, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0xe007 00010000:00010000:3.0:1789760793.042060:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c30394 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.042075:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=772s v=(0 0) 00010000:00010000:3.0:1789760793.042080:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760793.042086:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c30394 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29122560->29126655] (req 29122560->29126655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.042100:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c30394 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29122560->29126655] (req 29122560->29126655) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760793.075022:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974429 is committed 00002000:00010000:3.0:1789760793.075141:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 29122560 -> 29126656 00010000:00010000:3.0:1789760793.075156:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c30394 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29122560->29126655] (req 29122560->29126655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.075169:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c30394 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29122560->29126655] (req 29122560->29126655) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.075178:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c30394 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29122560->29126655] (req 29122560->29126655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.075186:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760793.075192:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c30394 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29122560->29126655] (req 29122560->29126655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.075201:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c30394 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29122560->29126655] (req 29122560->29126655) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.075222:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c30394 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29122560->29126655] (req 29122560->29126655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.075229:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c30394 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29122560->29126655] (req 29122560->29126655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760793.075363:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2745880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512774656:12345-192.168.203.56@tcp:4:dd.0 Request processed in 33358us (33862us total) trans 4294974429 rc 0/0 00000100:00100000:3.0:1789760793.075374:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7140 00000100:00100000:3.0:1789760793.075385:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 772565717861 time 772606286316 state 0x2 ref 0 00000100:00100000:2.0:1789760793.078925:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760793.079063:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512774784 00000100:00100000:3.0:1789760793.079073:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512774784 00000100:00100000:3.0:1789760793.079129:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 772565717861 time 772606286316 state 0x0 ref 1 00000100:00100000:3.0:1789760793.079144:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 772606546732 time 772606286316 state 0x1 ref 1 00000100:00100000:3.0:1789760793.079229:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 772606546732 time 772643526948 state 0x0 ref 1 00000100:00100000:3.0:1789760793.079242:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac287b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512774784:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760793.079293:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 29126656, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0xe007 00010000:00010000:3.0:1789760793.079302:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c3039b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.079318:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=772s v=(0 0) 00010000:00010000:3.0:1789760793.079322:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760793.079328:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c3039b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29126656->29130751] (req 29126656->29130751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.079351:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c3039b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29126656->29130751] (req 29126656->29130751) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760793.116583:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 29126656 -> 29130752 00010000:00010000:3.0:1789760793.116603:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c3039b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29126656->29130751] (req 29126656->29130751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.116617:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c3039b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29126656->29130751] (req 29126656->29130751) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.116636:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c3039b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29126656->29130751] (req 29126656->29130751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.116643:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760793.116647:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c3039b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29126656->29130751] (req 29126656->29130751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.116656:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c3039b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29126656->29130751] (req 29126656->29130751) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.116681:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c3039b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29126656->29130751] (req 29126656->29130751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.116688:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c3039b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29126656->29130751] (req 29126656->29130751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760793.116803:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac287b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2361:x1876699512774784:12345-192.168.203.56@tcp:4:dd.0 Request processed in 37557us (37949us total) trans 4294974430 rc 0/0 00000100:00100000:3.0:1789760793.116826:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7141 00000100:00100000:3.0:1789760793.116835:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 772606546732 time 772643526948 state 0x2 ref 0 00000100:00100000:2.0:1789760793.120574:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760793.120654:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512774912 00000100:00100000:3.0:1789760793.120665:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512774912 00000100:00100000:3.0:1789760793.120752:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 772606546732 time 772643526948 state 0x0 ref 1 00000100:00100000:3.0:1789760793.120768:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 772643787364 time 772643526948 state 0x1 ref 1 00000100:00100000:3.0:1789760793.120799:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 772643787364 time 772685098484 state 0x0 ref 1 00000100:00100000:3.0:1789760793.120813:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2878000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2358:x1876699512774912:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760793.120928:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 29130752, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0xe007 00010000:00010000:3.0:1789760793.120938:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c303a2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.120965:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=772s v=(0 0) 00010000:00010000:3.0:1789760793.120971:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760793.120978:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c303a2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29130752->29134847] (req 29130752->29134847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.120993:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c303a2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29130752->29134847] (req 29130752->29134847) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760793.121604:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974430 is committed 00002000:00010000:1.0:1789760793.191389:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 29130752 -> 29134848 00010000:00010000:1.0:1789760793.191408:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c303a2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29130752->29134847] (req 29130752->29134847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.191422:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c303a2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29130752->29134847] (req 29130752->29134847) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.191437:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c303a2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29130752->29134847] (req 29130752->29134847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.191444:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760793.191449:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c303a2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29130752->29134847] (req 29130752->29134847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.191459:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c303a2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29130752->29134847] (req 29130752->29134847) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.191477:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c303a2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29130752->29134847] (req 29130752->29134847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.191484:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c303a2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29130752->29134847] (req 29130752->29134847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760793.191694:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2878000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2358:x1876699512774912:12345-192.168.203.56@tcp:4:dd.0 Request processed in 70879us (71125us total) trans 4294974431 rc 0/0 00000100:00100000:1.0:1789760793.191749:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7142 00000100:00100000:1.0:1789760793.191759:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 772643787364 time 772685098484 state 0x2 ref 0 00000001:00080000:1.0:1789760793.191802:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974431 is committed 00000100:00100000:2.0:1789760793.195232:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760793.195312:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512775040 00000100:00100000:3.0:1789760793.195322:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512775040 00000100:00100000:3.0:1789760793.195374:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 772643787364 time 772685098484 state 0x0 ref 1 00000100:00100000:3.0:1789760793.195388:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 772685358900 time 772685098484 state 0x1 ref 1 00000100:00100000:3.0:1789760793.195418:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 772685358900 time 772759717610 state 0x0 ref 1 00000100:00100000:3.0:1789760793.195431:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2879180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512775040:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760793.195479:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 29134848, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0xe007 00010000:00010000:3.0:1789760793.195488:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c303a9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.195504:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=772s v=(0 0) 00010000:00010000:3.0:1789760793.195509:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760793.195516:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c303a9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29134848->29138943] (req 29134848->29138943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.195529:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c303a9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29134848->29138943] (req 29134848->29138943) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760793.243445:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974432 is committed 00002000:00010000:3.0:1789760793.243514:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 29134848 -> 29138944 00010000:00010000:3.0:1789760793.243530:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c303a9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29134848->29138943] (req 29134848->29138943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.243543:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c303a9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29134848->29138943] (req 29134848->29138943) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.243552:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c303a9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29134848->29138943] (req 29134848->29138943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.243559:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760793.243564:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c303a9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29134848->29138943] (req 29134848->29138943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.243573:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c303a9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29134848->29138943] (req 29134848->29138943) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.243590:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c303a9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29134848->29138943] (req 29134848->29138943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.243597:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c303a9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29134848->29138943] (req 29134848->29138943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760793.243692:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2879180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512775040:12345-192.168.203.56@tcp:4:dd.0 Request processed in 48259us (48465us total) trans 4294974432 rc 0/0 00000100:00100000:3.0:1789760793.243729:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7143 00000100:00100000:3.0:1789760793.243738:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 772685358900 time 772759717610 state 0x2 ref 0 00000100:00100000:2.0:1789760793.249884:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760793.249975:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512775168 00000100:00100000:3.0:1789760793.249985:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512775168 00000100:00100000:3.0:1789760793.250036:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 772685358900 time 772759717610 state 0x0 ref 1 00000100:00100000:3.0:1789760793.250051:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 772759978026 time 772759717610 state 0x1 ref 1 00000100:00100000:3.0:1789760793.250079:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 772759978026 time 772814379310 state 0x0 ref 1 00000100:00100000:3.0:1789760793.250091:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2878380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512775168:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760793.250135:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 29138944, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0xe007 00010000:00010000:3.0:1789760793.250144:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c303b0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.250159:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=772s v=(0 0) 00010000:00010000:3.0:1789760793.250163:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760793.250170:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c303b0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29138944->29143039] (req 29138944->29143039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.250183:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c303b0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29138944->29143039] (req 29138944->29143039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760793.284357:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974433 is committed 00002000:00010000:2.0:1789760793.284441:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 29138944 -> 29143040 00010000:00010000:2.0:1789760793.284456:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c303b0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29138944->29143039] (req 29138944->29143039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.284470:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c303b0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29138944->29143039] (req 29138944->29143039) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.284479:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c303b0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29138944->29143039] (req 29138944->29143039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.284486:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760793.284491:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c303b0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29138944->29143039] (req 29138944->29143039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.284501:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c303b0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29138944->29143039] (req 29138944->29143039) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.284521:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c303b0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29138944->29143039] (req 29138944->29143039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.284529:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c303b0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29138944->29143039] (req 29138944->29143039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760793.284776:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2878380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512775168:12345-192.168.203.56@tcp:4:dd.0 Request processed in 34682us (34895us total) trans 4294974433 rc 0/0 00000100:00100000:2.0:1789760793.284787:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7144 00000100:00100000:2.0:1789760793.284796:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 772759978026 time 772814379310 state 0x2 ref 0 00000100:00100000:0.0:1789760793.287535:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760793.287750:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512775296 00000100:00100000:1.0:1789760793.287760:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512775296 00000100:00100000:1.0:1789760793.287815:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 772759978026 time 772814379310 state 0x0 ref 1 00000100:00100000:1.0:1789760793.287830:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 772814639726 time 772814379310 state 0x1 ref 1 00000100:00100000:1.0:1789760793.287865:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 772814639726 time 772852164564 state 0x0 ref 1 00000100:00100000:1.0:1789760793.287878:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac437d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512775296:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760793.287925:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 29143040, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0xe007 00010000:00010000:1.0:1789760793.287934:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c303b7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.287960:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=772s v=(0 0) 00010000:00010000:1.0:1789760793.287965:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760793.287972:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c303b7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29143040->29147135] (req 29143040->29147135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.287987:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c303b7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29143040->29147135] (req 29143040->29147135) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760793.322383:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974434 is committed 00002000:00010000:1.0:1789760793.322492:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 29143040 -> 29147136 00010000:00010000:1.0:1789760793.322574:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c303b7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29143040->29147135] (req 29143040->29147135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.322588:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c303b7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29143040->29147135] (req 29143040->29147135) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.322596:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c303b7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29143040->29147135] (req 29143040->29147135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.322605:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760793.322610:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c303b7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29143040->29147135] (req 29143040->29147135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.322678:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c303b7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29143040->29147135] (req 29143040->29147135) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.322716:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c303b7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29143040->29147135] (req 29143040->29147135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.322725:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c303b7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29143040->29147135] (req 29143040->29147135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760793.322853:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac437d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512775296:12345-192.168.203.56@tcp:4:dd.0 Request processed in 34972us (35322us total) trans 4294974434 rc 0/0 00000100:00100000:1.0:1789760793.322863:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7145 00000100:00100000:1.0:1789760793.322872:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 772814639726 time 772852164564 state 0x2 ref 0 00000100:00100000:0.0:1789760793.327865:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760793.328073:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512775424 00000100:00100000:3.0:1789760793.328083:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512775424 00000100:00100000:3.0:1789760793.328198:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 772814639726 time 772852164564 state 0x0 ref 1 00000100:00100000:3.0:1789760793.328213:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 772852424980 time 772852164564 state 0x1 ref 1 00000100:00100000:3.0:1789760793.328276:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 772852424980 time 772892575454 state 0x0 ref 1 00000100:00100000:3.0:1789760793.328288:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac437df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512775424:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760793.328329:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 29147136, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0xe007 00010000:00010000:3.0:1789760793.328403:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c303be lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.328420:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=772s v=(0 0) 00010000:00010000:3.0:1789760793.328425:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760793.328432:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c303be lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29147136->29151231] (req 29147136->29151231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.328445:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c303be lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29147136->29151231] (req 29147136->29151231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760793.388275:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 29147136 -> 29151232 00010000:00010000:3.0:1789760793.388294:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c303be lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29147136->29151231] (req 29147136->29151231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.388307:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c303be lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29147136->29151231] (req 29147136->29151231) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.388323:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c303be lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29147136->29151231] (req 29147136->29151231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.388331:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760793.388336:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c303be lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29147136->29151231] (req 29147136->29151231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.388346:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c303be lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29147136->29151231] (req 29147136->29151231) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.388365:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c303be lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29147136->29151231] (req 29147136->29151231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.388373:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c303be lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29147136->29151231] (req 29147136->29151231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760793.388464:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac437df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2360:x1876699512775424:12345-192.168.203.56@tcp:4:dd.0 Request processed in 60173us (60602us total) trans 4294974435 rc 0/0 00000100:00100000:3.0:1789760793.388474:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7146 00000100:00100000:3.0:1789760793.388483:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 772852424980 time 772892575454 state 0x2 ref 0 00000001:00080000:1.0:1789760793.388554:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974435 is committed 00000100:00100000:2.0:1789760793.397377:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760793.397436:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512775552 00000100:00100000:3.0:1789760793.397453:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512775552 00000100:00100000:3.0:1789760793.397502:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 772852424980 time 772892575454 state 0x0 ref 1 00000100:00100000:3.0:1789760793.397516:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 772892835870 time 772892575454 state 0x1 ref 1 00000100:00100000:3.0:1789760793.397554:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 772892835870 time 772961853968 state 0x0 ref 1 00000100:00100000:3.0:1789760793.397566:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2879c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512775552:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760793.397614:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 29151232, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0xe007 00010000:00010000:3.0:1789760793.397629:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c303c5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.397645:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=772s v=(0 0) 00010000:00010000:3.0:1789760793.397649:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760793.397655:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c303c5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29151232->29155327] (req 29151232->29155327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.397669:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c303c5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29151232->29155327] (req 29151232->29155327) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760793.423826:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974436 is committed 00002000:00010000:3.0:1789760793.423936:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 29151232 -> 29155328 00010000:00010000:3.0:1789760793.423960:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c303c5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29151232->29155327] (req 29151232->29155327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.423973:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c303c5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29151232->29155327] (req 29151232->29155327) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.423983:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c303c5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29151232->29155327] (req 29151232->29155327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.423991:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760793.423996:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c303c5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29151232->29155327] (req 29151232->29155327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.424006:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c303c5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29151232->29155327] (req 29151232->29155327) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.424024:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c303c5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29151232->29155327] (req 29151232->29155327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.424031:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c303c5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29151232->29155327] (req 29151232->29155327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760793.424126:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2879c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512775552:12345-192.168.203.56@tcp:4:dd.0 Request processed in 26556us (26752us total) trans 4294974436 rc 0/0 00000100:00100000:3.0:1789760793.424136:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7147 00000100:00100000:3.0:1789760793.424144:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 772892835870 time 772961853968 state 0x2 ref 0 00000100:00100000:2.0:1789760793.427102:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760793.427180:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512775680 00000100:00100000:1.0:1789760793.427190:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512775680 00000100:00100000:1.0:1789760793.427242:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 772892835870 time 772961853968 state 0x0 ref 1 00000100:00100000:1.0:1789760793.427258:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 772962114384 time 772961853968 state 0x1 ref 1 00000100:00100000:1.0:1789760793.427332:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 772962114384 time 772991630963 state 0x0 ref 1 00000100:00100000:1.0:1789760793.427344:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac287bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512775680:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760793.427399:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 29155328, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0xe007 00010000:00010000:1.0:1789760793.427408:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c303cc lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.427423:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=772s v=(0 0) 00010000:00010000:1.0:1789760793.427428:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760793.427434:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c303cc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29155328->29159423] (req 29155328->29159423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.427448:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c303cc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29155328->29159423] (req 29155328->29159423) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760793.463493:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974437 is committed 00002000:00010000:3.0:1789760793.463601:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 29155328 -> 29159424 00010000:00010000:3.0:1789760793.463627:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c303cc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29155328->29159423] (req 29155328->29159423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.463642:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c303cc lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29155328->29159423] (req 29155328->29159423) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.463651:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c303cc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29155328->29159423] (req 29155328->29159423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.463665:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760793.463670:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c303cc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29155328->29159423] (req 29155328->29159423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.463680:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c303cc lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29155328->29159423] (req 29155328->29159423) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.463733:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c303cc lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29155328->29159423] (req 29155328->29159423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.463742:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c303cc lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29155328->29159423] (req 29155328->29159423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760793.463830:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac287bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512775680:12345-192.168.203.56@tcp:4:dd.0 Request processed in 36483us (36733us total) trans 4294974437 rc 0/0 00000100:00100000:3.0:1789760793.463840:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7148 00000100:00100000:3.0:1789760793.463849:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 772962114384 time 772991630963 state 0x2 ref 0 00000100:00100000:1.0:1789760793.468161:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760793.468228:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512775808 00000100:00100000:3.0:1789760793.468239:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512775808 00000100:00100000:3.0:1789760793.468295:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 772962114384 time 772991630963 state 0x0 ref 1 00000100:00100000:3.0:1789760793.468311:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 772991891379 time 772991630963 state 0x1 ref 1 00000100:00100000:3.0:1789760793.468341:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 772991891379 time 773032641055 state 0x0 ref 1 00000100:00100000:3.0:1789760793.468357:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac531ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512775808:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760793.468405:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 29159424, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0xe007 00010000:00010000:3.0:1789760793.468413:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c303d3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.468427:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=773s v=(0 0) 00010000:00010000:3.0:1789760793.468431:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760793.468436:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c303d3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29159424->29163519] (req 29159424->29163519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.468449:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c303d3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29159424->29163519] (req 29159424->29163519) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760793.496733:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974438 is committed 00002000:00010000:3.0:1789760793.497193:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 29159424 -> 29163520 00010000:00010000:3.0:1789760793.497208:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c303d3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29159424->29163519] (req 29159424->29163519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.497221:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c303d3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29159424->29163519] (req 29159424->29163519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.497230:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c303d3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29159424->29163519] (req 29159424->29163519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.497238:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760793.497243:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c303d3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29159424->29163519] (req 29159424->29163519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.497253:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c303d3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29159424->29163519] (req 29159424->29163519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.497274:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c303d3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29159424->29163519] (req 29159424->29163519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.497281:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c303d3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29159424->29163519] (req 29159424->29163519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760793.497374:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac531ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512775808:12345-192.168.203.56@tcp:4:dd.0 Request processed in 29014us (29217us total) trans 4294974438 rc 0/0 00000100:00100000:3.0:1789760793.497383:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7149 00000100:00100000:3.0:1789760793.497392:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 772991891379 time 773032641055 state 0x2 ref 0 00000100:00100000:1.0:1789760793.508059:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760793.508363:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512775936 00000100:00100000:3.0:1789760793.508372:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512775936 00000100:00100000:3.0:1789760793.508428:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 772991891379 time 773032641055 state 0x0 ref 1 00000100:00100000:3.0:1789760793.508444:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 773032901471 time 773032641055 state 0x1 ref 1 00000100:00100000:3.0:1789760793.508481:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 773032901471 time 773072780479 state 0x0 ref 1 00000100:00100000:3.0:1789760793.508493:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac531c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512775936:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760793.508550:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 29163520, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0xe007 00010000:00010000:3.0:1789760793.508558:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c303da lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.508574:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=773s v=(0 0) 00010000:00010000:3.0:1789760793.508578:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760793.508584:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c303da lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29163520->29167615] (req 29163520->29167615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.508598:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c303da lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29163520->29167615] (req 29163520->29167615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760793.552024:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 29163520 -> 29167616 00002000:00010000:3.0:1789760793.552032:0:13062:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760792 -> 1789760793 00002000:00010000:3.0:1789760793.552035:0:13062:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760792 -> 1789760793 00010000:00010000:3.0:1789760793.552048:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c303da lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29163520->29167615] (req 29163520->29167615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.552062:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c303da lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29163520->29167615] (req 29163520->29167615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.552071:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c303da lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29163520->29167615] (req 29163520->29167615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.552079:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760793.552084:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c303da lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29163520->29167615] (req 29163520->29167615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.552094:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c303da lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29163520->29167615] (req 29163520->29167615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.552113:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c303da lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29163520->29167615] (req 29163520->29167615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.552121:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c303da lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29163520->29167615] (req 29163520->29167615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760793.552212:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac531c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2360:x1876699512775936:12345-192.168.203.56@tcp:4:dd.0 Request processed in 43717us (44160us total) trans 4294974439 rc 0/0 00000100:00100000:3.0:1789760793.552222:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7150 00000100:00100000:3.0:1789760793.552231:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 773032901471 time 773072780479 state 0x2 ref 0 00000100:00100000:2.0:1789760793.554730:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760793.554813:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512776064 00000100:00100000:3.0:1789760793.554823:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512776064 00000100:00100000:3.0:1789760793.554874:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 773032901471 time 773072780479 state 0x0 ref 1 00000100:00100000:3.0:1789760793.554889:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 773073040895 time 773072780479 state 0x1 ref 1 00000100:00100000:3.0:1789760793.554911:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 773073040895 time 773119211950 state 0x0 ref 1 00000100:00100000:3.0:1789760793.554924:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febea48a050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2360:x1876699512776064:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760793.554979:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 29167616, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0xe007 00010000:00010000:3.0:1789760793.554988:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c303e1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.555004:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=773s v=(0 0) 00010000:00010000:3.0:1789760793.555009:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760793.555015:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c303e1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29167616->29171711] (req 29167616->29171711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.555029:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c303e1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29167616->29171711] (req 29167616->29171711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760793.561282:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974439 is committed 00000001:00080000:1.0:1789760793.590590:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974440 is committed 00002000:00010000:3.0:1789760793.590749:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 29167616 -> 29171712 00010000:00010000:3.0:1789760793.590763:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c303e1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29167616->29171711] (req 29167616->29171711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.590776:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c303e1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29167616->29171711] (req 29167616->29171711) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.590784:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c303e1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29167616->29171711] (req 29167616->29171711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.590791:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760793.590796:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c303e1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29167616->29171711] (req 29167616->29171711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.590804:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c303e1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29167616->29171711] (req 29167616->29171711) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.590823:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c303e1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29167616->29171711] (req 29167616->29171711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.590835:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c303e1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29167616->29171711] (req 29167616->29171711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760793.590928:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febea48a050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512776064:12345-192.168.203.56@tcp:4:dd.0 Request processed in 36001us (36234us total) trans 4294974440 rc 0/0 00000100:00100000:3.0:1789760793.590937:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7151 00000100:00100000:3.0:1789760793.590958:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 773073040895 time 773119211950 state 0x2 ref 0 00000100:00100000:2.0:1789760793.594271:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760793.594575:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512776192 00000100:00100000:3.0:1789760793.594585:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512776192 00000100:00100000:3.0:1789760793.594652:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 773073040895 time 773119211950 state 0x0 ref 1 00000100:00100000:3.0:1789760793.594667:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 773119472366 time 773119211950 state 0x1 ref 1 00000100:00100000:3.0:1789760793.594730:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 773119472366 time 773158996774 state 0x0 ref 1 00000100:00100000:3.0:1789760793.594745:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac287b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512776192:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760793.594789:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 29171712, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0xe007 00010000:00010000:3.0:1789760793.594798:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c303e8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.594814:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=773s v=(0 0) 00010000:00010000:3.0:1789760793.594818:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760793.594825:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c303e8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29171712->29175807] (req 29171712->29175807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.594843:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c303e8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29171712->29175807] (req 29171712->29175807) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760793.628029:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 29171712 -> 29175808 00010000:00010000:3.0:1789760793.628050:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c303e8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29171712->29175807] (req 29171712->29175807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.628063:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c303e8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29171712->29175807] (req 29171712->29175807) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.628071:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c303e8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29171712->29175807] (req 29171712->29175807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.628079:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760793.628085:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c303e8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29171712->29175807] (req 29171712->29175807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.628093:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c303e8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29171712->29175807] (req 29171712->29175807) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.628111:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c303e8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29171712->29175807] (req 29171712->29175807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.628118:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c303e8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29171712->29175807] (req 29171712->29175807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760793.628204:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974441 is committed 00000100:00100000:3.0:1789760793.628218:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac287b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512776192:12345-192.168.203.56@tcp:4:dd.0 Request processed in 33473us (33953us total) trans 4294974441 rc 0/0 00000100:00100000:3.0:1789760793.628228:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7152 00000100:00100000:3.0:1789760793.628238:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 773119472366 time 773158996774 state 0x2 ref 0 00000100:00100000:1.0:1789760793.632445:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760793.632542:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512776320 00000100:00100000:3.0:1789760793.632552:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512776320 00000100:00100000:3.0:1789760793.632604:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 773119472366 time 773158996774 state 0x0 ref 1 00000100:00100000:3.0:1789760793.632686:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 773159257190 time 773158996774 state 0x1 ref 1 00000100:00100000:3.0:1789760793.632782:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 773159257190 time 773197081420 state 0x0 ref 1 00000100:00100000:3.0:1789760793.632796:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac531ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512776320:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760793.632844:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 29175808, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0xe007 00010000:00010000:3.0:1789760793.632854:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c303ef lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.632870:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=773s v=(0 0) 00010000:00010000:3.0:1789760793.632876:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760793.632882:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c303ef lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29175808->29179903] (req 29175808->29179903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.632896:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c303ef lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29175808->29179903] (req 29175808->29179903) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760793.658963:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 29175808 -> 29179904 00010000:00010000:3.0:1789760793.658983:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c303ef lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29175808->29179903] (req 29175808->29179903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.658996:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c303ef lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29175808->29179903] (req 29175808->29179903) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.659005:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c303ef lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29175808->29179903] (req 29175808->29179903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.659012:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760793.659017:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c303ef lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29175808->29179903] (req 29175808->29179903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.659026:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c303ef lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29175808->29179903] (req 29175808->29179903) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.659055:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c303ef lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29175808->29179903] (req 29175808->29179903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.659064:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c303ef lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29175808->29179903] (req 29175808->29179903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760793.659158:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac531ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2360:x1876699512776320:12345-192.168.203.56@tcp:4:dd.0 Request processed in 26359us (26826us total) trans 4294974442 rc 0/0 00000100:00100000:3.0:1789760793.659173:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7153 00000100:00100000:3.0:1789760793.659183:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 773159257190 time 773197081420 state 0x2 ref 0 00000001:00080000:0.0:1789760793.662973:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974442 is committed 00000100:00100000:1.0:1789760793.663204:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760793.663284:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512776448 00000100:00100000:3.0:1789760793.663292:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512776448 00000100:00100000:3.0:1789760793.663364:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 773159257190 time 773197081420 state 0x0 ref 1 00000100:00100000:3.0:1789760793.663380:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 773197341836 time 773197081420 state 0x1 ref 1 00000100:00100000:3.0:1789760793.663413:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 773197341836 time 773227711996 state 0x0 ref 1 00000100:00100000:3.0:1789760793.663427:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac531dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512776448:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760793.663477:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 29179904, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0xe007 00010000:00010000:3.0:1789760793.663486:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c303f6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.663501:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=773s v=(0 0) 00010000:00010000:3.0:1789760793.663506:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760793.663512:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c303f6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29179904->29183999] (req 29179904->29183999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.663525:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c303f6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29179904->29183999] (req 29179904->29183999) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760793.691551:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 29179904 -> 29184000 00010000:00010000:3.0:1789760793.691571:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c303f6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29179904->29183999] (req 29179904->29183999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.691585:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c303f6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29179904->29183999] (req 29179904->29183999) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.691594:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c303f6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29179904->29183999] (req 29179904->29183999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.691601:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760793.691606:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c303f6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29179904->29183999] (req 29179904->29183999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.691627:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c303f6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29179904->29183999] (req 29179904->29183999) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.691646:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c303f6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29179904->29183999] (req 29179904->29183999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.691654:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c303f6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29179904->29183999] (req 29179904->29183999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760793.691781:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac531dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2361:x1876699512776448:12345-192.168.203.56@tcp:4:dd.0 Request processed in 28352us (28577us total) trans 4294974443 rc 0/0 00000100:00100000:3.0:1789760793.691791:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7154 00000100:00100000:3.0:1789760793.691800:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 773197341836 time 773227711996 state 0x2 ref 0 00000001:00080000:1.0:1789760793.691985:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974443 is committed 00000100:00100000:1.0:1789760793.694920:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760793.695012:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512776576 00000100:00100000:3.0:1789760793.695023:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512776576 00000100:00100000:3.0:1789760793.695088:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 773197341836 time 773227711996 state 0x0 ref 1 00000100:00100000:3.0:1789760793.695104:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 773227972412 time 773227711996 state 0x1 ref 1 00000100:00100000:3.0:1789760793.695141:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 773227972412 time 773259440598 state 0x0 ref 1 00000100:00100000:3.0:1789760793.695153:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac531d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512776576:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760793.695201:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 29184000, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0xe007 00010000:00010000:3.0:1789760793.695210:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c303fd lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.695226:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=773s v=(0 0) 00010000:00010000:3.0:1789760793.695231:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760793.695238:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c303fd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29184000->29188095] (req 29184000->29188095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.695251:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c303fd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29184000->29188095] (req 29184000->29188095) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760793.723832:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974444 is committed 00002000:00010000:1.0:1789760793.723940:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 29184000 -> 29188096 00010000:00010000:1.0:1789760793.723972:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c303fd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29184000->29188095] (req 29184000->29188095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.723986:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c303fd lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29184000->29188095] (req 29184000->29188095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.723995:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c303fd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29184000->29188095] (req 29184000->29188095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.724002:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760793.724007:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c303fd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29184000->29188095] (req 29184000->29188095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.724016:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c303fd lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29184000->29188095] (req 29184000->29188095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.724033:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c303fd lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29184000->29188095] (req 29184000->29188095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.724040:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c303fd lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29184000->29188095] (req 29184000->29188095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760793.724132:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac531d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512776576:12345-192.168.203.56@tcp:4:dd.0 Request processed in 28976us (29218us total) trans 4294974444 rc 0/0 00000100:00100000:1.0:1789760793.724142:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7155 00000100:00100000:1.0:1789760793.724152:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 773227972412 time 773259440598 state 0x2 ref 0 00000100:00100000:1.0:1789760793.727123:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760793.727197:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512776704 00000100:00100000:3.0:1789760793.727206:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512776704 00000100:00100000:3.0:1789760793.727259:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 773227972412 time 773259440598 state 0x0 ref 1 00000100:00100000:3.0:1789760793.727275:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 773259701014 time 773259440598 state 0x1 ref 1 00000100:00100000:3.0:1789760793.727302:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 773259701014 time 773291601932 state 0x0 ref 1 00000100:00100000:3.0:1789760793.727317:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac531ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512776704:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760793.727363:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 29188096, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0xe007 00010000:00010000:3.0:1789760793.727371:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c30404 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.727387:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=773s v=(0 0) 00010000:00010000:3.0:1789760793.727392:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760793.727399:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c30404 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29188096->29192191] (req 29188096->29192191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.727413:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c30404 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29188096->29192191] (req 29188096->29192191) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760793.751097:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974445 is committed 00002000:00010000:1.0:1789760793.751247:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 29188096 -> 29192192 00010000:00010000:1.0:1789760793.751265:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c30404 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29188096->29192191] (req 29188096->29192191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.751278:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c30404 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29188096->29192191] (req 29188096->29192191) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.751287:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c30404 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29188096->29192191] (req 29188096->29192191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.751295:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760793.751300:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c30404 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29188096->29192191] (req 29188096->29192191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.751310:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c30404 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29188096->29192191] (req 29188096->29192191) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.751328:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c30404 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29188096->29192191] (req 29188096->29192191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.751346:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e45aeab9/0xe6aa143190c30404 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29188096->29192191] (req 29188096->29192191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760793.751439:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac531ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512776704:12345-192.168.203.56@tcp:4:dd.0 Request processed in 24118us (24318us total) trans 4294974445 rc 0/0 00000100:00100000:1.0:1789760793.751452:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7156 00000100:00100000:1.0:1789760793.751462:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 773259701014 time 773291601932 state 0x2 ref 0 00000100:00100000:2.0:1789760793.754217:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760793.754308:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512776832 00000100:00100000:3.0:1789760793.754317:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512776832 00000100:00100000:3.0:1789760793.754428:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 773259701014 time 773291601932 state 0x0 ref 1 00000100:00100000:3.0:1789760793.754442:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 773291862348 time 773291601932 state 0x1 ref 1 00000100:00100000:3.0:1789760793.754476:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 773291862348 time 773318775051 state 0x0 ref 1 00000100:00100000:3.0:1789760793.754488:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac287b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512776832:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760793.754535:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 29192192, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0xe007 00010000:00010000:3.0:1789760793.754544:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c3040b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.754565:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=773s v=(0 0) 00010000:00010000:3.0:1789760793.754569:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760793.754575:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c3040b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29192192->29196287] (req 29192192->29196287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.754589:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c3040b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29192192->29196287] (req 29192192->29196287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760793.787860:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974446 is committed 00002000:00010000:3.0:1789760793.787941:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 29192192 -> 29196288 00010000:00010000:3.0:1789760793.787968:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c3040b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29192192->29196287] (req 29192192->29196287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.787979:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c3040b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29192192->29196287] (req 29192192->29196287) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.787988:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c3040b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29192192->29196287] (req 29192192->29196287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.787995:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760793.787999:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c3040b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29192192->29196287] (req 29192192->29196287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.788009:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c3040b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29192192->29196287] (req 29192192->29196287) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.788027:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c3040b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29192192->29196287] (req 29192192->29196287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.788035:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c3040b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29192192->29196287] (req 29192192->29196287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760793.788157:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac287b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512776832:12345-192.168.203.56@tcp:4:dd.0 Request processed in 33665us (33944us total) trans 4294974446 rc 0/0 00000100:00100000:3.0:1789760793.788168:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7157 00000100:00100000:3.0:1789760793.788177:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 773291862348 time 773318775051 state 0x2 ref 0 00000100:00100000:2.0:1789760793.791570:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760793.791847:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512776960 00000100:00100000:1.0:1789760793.791858:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512776960 00000100:00100000:1.0:1789760793.791912:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 773291862348 time 773318775051 state 0x0 ref 1 00000100:00100000:1.0:1789760793.791928:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 773319035467 time 773318775051 state 0x1 ref 1 00000100:00100000:1.0:1789760793.792004:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 773319035467 time 773356303199 state 0x0 ref 1 00000100:00100000:1.0:1789760793.792016:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac287b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512776960:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760793.792062:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 29196288, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0xe007 00010000:00010000:1.0:1789760793.792070:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c30412 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.792086:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=773s v=(0 0) 00010000:00010000:1.0:1789760793.792090:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760793.792097:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c30412 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29196288->29200383] (req 29196288->29200383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.792111:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c30412 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29196288->29200383] (req 29196288->29200383) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760793.819559:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 29196288 -> 29200384 00010000:00010000:3.0:1789760793.819579:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c30412 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29196288->29200383] (req 29196288->29200383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.819592:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c30412 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29196288->29200383] (req 29196288->29200383) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.819601:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c30412 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29196288->29200383] (req 29196288->29200383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.819608:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760793.819613:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c30412 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29196288->29200383] (req 29196288->29200383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.819633:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c30412 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29196288->29200383] (req 29196288->29200383) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.819655:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c30412 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29196288->29200383] (req 29196288->29200383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.819662:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c30412 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29196288->29200383] (req 29196288->29200383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760793.819786:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac287b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2360:x1876699512776960:12345-192.168.203.56@tcp:4:dd.0 Request processed in 27767us (28220us total) trans 4294974447 rc 0/0 00000100:00100000:3.0:1789760793.819797:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7158 00000100:00100000:3.0:1789760793.819807:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 773319035467 time 773356303199 state 0x2 ref 0 00000001:00080000:2.0:1789760793.820839:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974447 is committed 00000100:00100000:1.0:1789760793.822259:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760793.822400:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512777088 00000100:00100000:3.0:1789760793.822410:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512777088 00000100:00100000:3.0:1789760793.822461:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 773319035467 time 773356303199 state 0x0 ref 1 00000100:00100000:3.0:1789760793.822475:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 773356563615 time 773356303199 state 0x1 ref 1 00000100:00100000:3.0:1789760793.822509:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 773356563615 time 773386808891 state 0x0 ref 1 00000100:00100000:3.0:1789760793.822523:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac531f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512777088:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760793.822572:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 29200384, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0xe007 00010000:00010000:3.0:1789760793.822580:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c30419 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.822595:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=773s v=(0 0) 00010000:00010000:3.0:1789760793.822600:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760793.822606:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c30419 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29200384->29204479] (req 29200384->29204479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.822814:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c30419 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29200384->29204479] (req 29200384->29204479) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760793.847120:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 29200384 -> 29204480 00010000:00010000:2.0:1789760793.847140:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c30419 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29200384->29204479] (req 29200384->29204479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.847155:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c30419 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29200384->29204479] (req 29200384->29204479) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.847165:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c30419 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29200384->29204479] (req 29200384->29204479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.847172:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760793.847178:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c30419 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29200384->29204479] (req 29200384->29204479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.847187:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c30419 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29200384->29204479] (req 29200384->29204479) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.847205:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c30419 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29200384->29204479] (req 29200384->29204479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.847213:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000027ceadfd/0xe6aa143190c30419 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29200384->29204479] (req 29200384->29204479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760793.847303:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac531f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2361:x1876699512777088:12345-192.168.203.56@tcp:4:dd.0 Request processed in 24778us (25047us total) trans 4294974448 rc 0/0 00000100:00100000:2.0:1789760793.847313:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7159 00000100:00100000:2.0:1789760793.847323:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 773356563615 time 773386808891 state 0x2 ref 0 00000001:00080000:3.0:1789760793.847508:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974448 is committed 00000100:00100000:3.0:1789760793.856445:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760793.857039:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512777216 00000100:00100000:1.0:1789760793.857050:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512777216 00000100:00100000:1.0:1789760793.857105:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 773356563615 time 773386808891 state 0x0 ref 1 00000100:00100000:1.0:1789760793.857121:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 773387069307 time 773386808891 state 0x1 ref 1 00000100:00100000:1.0:1789760793.857197:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 773387069307 time 773421496092 state 0x0 ref 1 00000100:00100000:1.0:1789760793.857210:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febfb374380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512777216:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760793.857258:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 29204480, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0xe007 00010000:00010000:1.0:1789760793.857277:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c30420 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.857292:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=773s v=(0 0) 00010000:00010000:1.0:1789760793.857296:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760793.857302:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c30420 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29204480->29208575] (req 29204480->29208575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.857315:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c30420 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29204480->29208575] (req 29204480->29208575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760793.900012:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974449 is committed 00002000:00010000:3.0:1789760793.900097:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 29204480 -> 29208576 00010000:00010000:3.0:1789760793.900114:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c30420 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29204480->29208575] (req 29204480->29208575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.900128:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c30420 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29204480->29208575] (req 29204480->29208575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.900138:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c30420 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29204480->29208575] (req 29204480->29208575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.900145:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760793.900150:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c30420 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29204480->29208575] (req 29204480->29208575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.900160:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c30420 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29204480->29208575] (req 29204480->29208575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.900179:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c30420 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29204480->29208575] (req 29204480->29208575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.900187:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c30420 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29204480->29208575] (req 29204480->29208575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760793.900317:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febfb374380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512777216:12345-192.168.203.56@tcp:4:dd.0 Request processed in 43104us (43877us total) trans 4294974449 rc 0/0 00000100:00100000:3.0:1789760793.900328:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7160 00000100:00100000:3.0:1789760793.900337:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 773387069307 time 773421496092 state 0x2 ref 0 00000100:00100000:1.0:1789760793.908115:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760793.908210:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512777344 00000100:00100000:3.0:1789760793.908221:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512777344 00000100:00100000:3.0:1789760793.908275:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 773387069307 time 773421496092 state 0x0 ref 1 00000100:00100000:3.0:1789760793.908292:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 773421756508 time 773421496092 state 0x1 ref 1 00000100:00100000:3.0:1789760793.908326:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 773421756508 time 773472625165 state 0x0 ref 1 00000100:00100000:3.0:1789760793.908348:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac531df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512777344:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760793.908398:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 29208576, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0xe007 00010000:00010000:3.0:1789760793.908407:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c30427 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.908422:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=773s v=(0 0) 00010000:00010000:3.0:1789760793.908426:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760793.908432:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c30427 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29208576->29212671] (req 29208576->29212671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.908465:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c30427 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29208576->29212671] (req 29208576->29212671) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760793.957888:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974450 is committed 00002000:00010000:3.0:1789760793.958034:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 29208576 -> 29212672 00010000:00010000:3.0:1789760793.958050:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c30427 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29208576->29212671] (req 29208576->29212671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.958062:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c30427 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29208576->29212671] (req 29208576->29212671) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.958071:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c30427 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29208576->29212671] (req 29208576->29212671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.958078:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760793.958083:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c30427 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29208576->29212671] (req 29208576->29212671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.958092:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c30427 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29208576->29212671] (req 29208576->29212671) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.958111:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c30427 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29208576->29212671] (req 29208576->29212671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.958119:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c30427 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29208576->29212671] (req 29208576->29212671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760793.958367:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac531df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512777344:12345-192.168.203.56@tcp:4:dd.0 Request processed in 50025us (50258us total) trans 4294974450 rc 0/0 00000100:00100000:3.0:1789760793.958378:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7161 00000100:00100000:3.0:1789760793.958388:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 773421756508 time 773472625165 state 0x2 ref 0 00000100:00100000:1.0:1789760793.964190:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760793.964268:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512777472 00000100:00100000:2.0:1789760793.964279:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512777472 00000100:00100000:2.0:1789760793.964332:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 773421756508 time 773472625165 state 0x0 ref 1 00000100:00100000:2.0:1789760793.964347:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 773472885581 time 773472625165 state 0x1 ref 1 00000100:00100000:2.0:1789760793.964381:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 773472885581 time 773528680421 state 0x0 ref 1 00000100:00100000:2.0:1789760793.964395:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac531c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512777472:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760793.964444:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 29212672, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0xe007 00010000:00010000:2.0:1789760793.964454:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c3042e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.964470:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=773s v=(0 0) 00010000:00010000:2.0:1789760793.964475:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760793.964480:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c3042e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29212672->29216767] (req 29212672->29216767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.964494:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c3042e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29212672->29216767] (req 29212672->29216767) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.972061:0:5836:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:1.0:1789760794.006041:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974451 is committed 00002000:00010000:3.0:1789760794.006059:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 29212672 -> 29216768 00010000:00010000:3.0:1789760794.006075:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c3042e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29212672->29216767] (req 29212672->29216767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.006089:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c3042e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29212672->29216767] (req 29212672->29216767) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.006098:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c3042e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29212672->29216767] (req 29212672->29216767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.006106:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760794.006110:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c3042e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29212672->29216767] (req 29212672->29216767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.006119:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c3042e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29212672->29216767] (req 29212672->29216767) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.006138:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c3042e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29212672->29216767] (req 29212672->29216767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.006152:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000029e3d12c/0xe6aa143190c3042e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29212672->29216767] (req 29212672->29216767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760794.006285:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac531c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512777472:12345-192.168.203.56@tcp:4:dd.0 Request processed in 41888us (42100us total) trans 4294974451 rc 0/0 00000100:00100000:3.0:1789760794.006295:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7162 00000100:00100000:3.0:1789760794.006303:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 773472885581 time 773528680421 state 0x2 ref 0 00000100:00100000:0.0:1789760794.008420:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760794.008494:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512777600 00000100:00100000:3.0:1789760794.008502:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512777600 00000100:00100000:3.0:1789760794.008544:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 773472885581 time 773528680421 state 0x0 ref 1 00000100:00100000:3.0:1789760794.008556:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 773528940837 time 773528680421 state 0x1 ref 1 00000100:00100000:3.0:1789760794.008600:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 773528940837 time 773572900882 state 0x0 ref 1 00000100:00100000:3.0:1789760794.008609:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac437e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512777600:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760794.008647:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 29216768, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0xe007 00010000:00010000:3.0:1789760794.008653:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c30435 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.008665:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=773s v=(0 0) 00010000:00010000:3.0:1789760794.008668:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760794.008672:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c30435 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29216768->29220863] (req 29216768->29220863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.008686:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c30435 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29216768->29220863] (req 29216768->29220863) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760794.038766:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974452 is committed 00002000:00010000:2.0:1789760794.039601:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 29216768 -> 29220864 00010000:00010000:2.0:1789760794.039627:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c30435 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29216768->29220863] (req 29216768->29220863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.039640:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c30435 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29216768->29220863] (req 29216768->29220863) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.039656:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c30435 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29216768->29220863] (req 29216768->29220863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.039663:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760794.039668:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c30435 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29216768->29220863] (req 29216768->29220863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.039677:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c30435 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29216768->29220863] (req 29216768->29220863) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.039725:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c30435 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29216768->29220863] (req 29216768->29220863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.039735:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d137ffd2/0xe6aa143190c30435 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29216768->29220863] (req 29216768->29220863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760794.039823:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac437e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512777600:12345-192.168.203.56@tcp:4:dd.0 Request processed in 31210us (31407us total) trans 4294974452 rc 0/0 00000100:00100000:2.0:1789760794.039833:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7163 00000100:00100000:2.0:1789760794.039843:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 773528940837 time 773572900882 state 0x2 ref 0 00000100:00100000:1.0:1789760794.043260:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760794.043353:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512777728 00000100:00100000:3.0:1789760794.043362:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512777728 00000100:00100000:3.0:1789760794.043415:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 773528940837 time 773572900882 state 0x0 ref 1 00000100:00100000:3.0:1789760794.043431:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 773573161298 time 773572900882 state 0x1 ref 1 00000100:00100000:3.0:1789760794.043462:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 773573161298 time 773607762467 state 0x0 ref 1 00000100:00100000:3.0:1789760794.043474:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac531d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512777728:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760794.043519:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 29220864, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0xe007 00010000:00010000:3.0:1789760794.043528:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c3043c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.043544:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=773s v=(0 0) 00010000:00010000:3.0:1789760794.043549:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760794.043556:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c3043c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29220864->29224959] (req 29220864->29224959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.043570:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c3043c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29220864->29224959] (req 29220864->29224959) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760794.086064:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974453 is committed 00002000:00010000:1.0:1789760794.086121:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 29220864 -> 29224960 00010000:00010000:1.0:1789760794.086138:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c3043c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29220864->29224959] (req 29220864->29224959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.086152:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c3043c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29220864->29224959] (req 29220864->29224959) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.086161:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c3043c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29220864->29224959] (req 29220864->29224959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.086168:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760794.086174:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c3043c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29220864->29224959] (req 29220864->29224959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.086185:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c3043c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29220864->29224959] (req 29220864->29224959) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.086206:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c3043c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29220864->29224959] (req 29220864->29224959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.086215:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f404f9e9/0xe6aa143190c3043c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29220864->29224959] (req 29220864->29224959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760794.086306:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac531d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512777728:12345-192.168.203.56@tcp:4:dd.0 Request processed in 42828us (43048us total) trans 4294974453 rc 0/0 00000100:00100000:1.0:1789760794.086316:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7164 00000100:00100000:1.0:1789760794.086327:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 773573161298 time 773607762467 state 0x2 ref 0 00000100:00100000:0.0:1789760794.091495:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760794.092925:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512777856 00000100:00100000:3.0:1789760794.092937:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512777856 00000100:00100000:3.0:1789760794.093006:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 773573161298 time 773607762467 state 0x0 ref 1 00000100:00100000:3.0:1789760794.093023:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 773608022883 time 773607762467 state 0x1 ref 1 00000100:00100000:3.0:1789760794.093054:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 773608022883 time 773657353334 state 0x0 ref 1 00000100:00100000:3.0:1789760794.093067:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac437f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512777856:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760794.093124:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 29224960, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0xe007 00010000:00010000:3.0:1789760794.093134:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c30443 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.093151:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=773s v=(0 0) 00010000:00010000:3.0:1789760794.093156:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760794.093163:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c30443 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29224960->29229055] (req 29224960->29229055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.093178:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c30443 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29224960->29229055] (req 29224960->29229055) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760794.128843:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974454 is committed 00002000:00010000:1.0:1789760794.128974:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 29224960 -> 29229056 00010000:00010000:1.0:1789760794.128990:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c30443 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29224960->29229055] (req 29224960->29229055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.129003:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c30443 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29224960->29229055] (req 29224960->29229055) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.129011:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c30443 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29224960->29229055] (req 29224960->29229055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.129019:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760794.129023:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c30443 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29224960->29229055] (req 29224960->29229055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.129033:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c30443 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29224960->29229055] (req 29224960->29229055) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.129051:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c30443 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29224960->29229055] (req 29224960->29229055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.129060:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005400bea2/0xe6aa143190c30443 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29224960->29229055] (req 29224960->29229055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760794.129153:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac437f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512777856:12345-192.168.203.56@tcp:4:dd.0 Request processed in 36084us (37662us total) trans 4294974454 rc 0/0 00000100:00100000:1.0:1789760794.129163:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7165 00000100:00100000:1.0:1789760794.129172:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 773608022883 time 773657353334 state 0x2 ref 0 00000100:00100000:0.0:1789760794.132363:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760794.132692:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512777984 00000100:00100000:2.0:1789760794.132883:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512777984 00000100:00100000:2.0:1789760794.132936:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 773608022883 time 773657353334 state 0x0 ref 1 00000100:00100000:2.0:1789760794.132976:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 773657613750 time 773657353334 state 0x1 ref 1 00000100:00100000:2.0:1789760794.133005:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 773657613750 time 773697304658 state 0x0 ref 1 00000100:00100000:2.0:1789760794.133018:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac437d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512777984:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760794.133064:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 29229056, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0xe007 00010000:00010000:2.0:1789760794.133073:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c3044a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.133090:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=773s v=(0 0) 00010000:00010000:2.0:1789760794.133094:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760794.133100:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c3044a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29229056->29233151] (req 29229056->29233151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.133114:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c3044a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29229056->29233151] (req 29229056->29233151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760794.169151:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974455 is committed 00002000:00010000:0.0:1789760794.169257:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 29229056 -> 29233152 00010000:00010000:0.0:1789760794.169273:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c3044a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29229056->29233151] (req 29229056->29233151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.169286:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c3044a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29229056->29233151] (req 29229056->29233151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.169295:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c3044a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29229056->29233151] (req 29229056->29233151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.169302:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760794.169306:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c3044a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29229056->29233151] (req 29229056->29233151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.169315:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c3044a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29229056->29233151] (req 29229056->29233151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.169334:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c3044a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29229056->29233151] (req 29229056->29233151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.169346:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c3044a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29229056->29233151] (req 29229056->29233151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760794.169434:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac437d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512777984:12345-192.168.203.56@tcp:4:dd.0 Request processed in 36413us (37074us total) trans 4294974455 rc 0/0 00000100:00100000:0.0:1789760794.169443:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7166 00000100:00100000:0.0:1789760794.169452:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 773657613750 time 773697304658 state 0x2 ref 0 00000100:00100000:2.0:1789760794.173513:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760794.173661:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512778112 00000100:00100000:1.0:1789760794.173671:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512778112 00000100:00100000:1.0:1789760794.173763:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 773657613750 time 773697304658 state 0x0 ref 1 00000100:00100000:1.0:1789760794.173777:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 773697565074 time 773697304658 state 0x1 ref 1 00000100:00100000:1.0:1789760794.173798:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 773697565074 time 773738098703 state 0x0 ref 1 00000100:00100000:1.0:1789760794.173809:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2878000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512778112:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760794.173856:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 29233152, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0xe007 00010000:00010000:1.0:1789760794.173865:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c30451 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.173880:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=773s v=(0 0) 00010000:00010000:1.0:1789760794.173885:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760794.173891:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c30451 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29233152->29237247] (req 29233152->29237247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.173904:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c30451 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29233152->29237247] (req 29233152->29237247) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760794.202983:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 29233152 -> 29237248 00010000:00010000:1.0:1789760794.203004:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c30451 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29233152->29237247] (req 29233152->29237247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.203018:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c30451 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29233152->29237247] (req 29233152->29237247) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.203028:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c30451 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29233152->29237247] (req 29233152->29237247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.203035:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760794.203040:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c30451 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29233152->29237247] (req 29233152->29237247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.203050:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c30451 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29233152->29237247] (req 29233152->29237247) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.203069:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c30451 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29233152->29237247] (req 29233152->29237247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.203078:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c30451 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29233152->29237247] (req 29233152->29237247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760794.203164:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2878000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2358:x1876699512778112:12345-192.168.203.56@tcp:4:dd.0 Request processed in 29351us (29655us total) trans 4294974456 rc 0/0 00000100:00100000:1.0:1789760794.203174:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7167 00000100:00100000:1.0:1789760794.203183:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 773697565074 time 773738098703 state 0x2 ref 0 00000001:00080000:3.0:1789760794.203398:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974456 is committed 00000100:00100000:2.0:1789760794.206842:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760794.206898:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512778240 00000100:00100000:1.0:1789760794.206908:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512778240 00000100:00100000:1.0:1789760794.206988:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 773697565074 time 773738098703 state 0x0 ref 1 00000100:00100000:1.0:1789760794.207003:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 773738359119 time 773738098703 state 0x1 ref 1 00000100:00100000:1.0:1789760794.207034:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 773738359119 time 773771334120 state 0x0 ref 1 00000100:00100000:1.0:1789760794.207045:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2879180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512778240:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760794.207091:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 29237248, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0xe007 00010000:00010000:1.0:1789760794.207100:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c30458 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.207117:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=773s v=(0 0) 00010000:00010000:1.0:1789760794.207121:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760794.207127:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c30458 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29237248->29241343] (req 29237248->29241343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.207141:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c30458 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29237248->29241343] (req 29237248->29241343) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760794.245136:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974457 is committed 00002000:00010000:1.0:1789760794.245225:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 29237248 -> 29241344 00010000:00010000:1.0:1789760794.245242:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c30458 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29237248->29241343] (req 29237248->29241343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.245256:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c30458 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29237248->29241343] (req 29237248->29241343) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.245265:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c30458 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29237248->29241343] (req 29237248->29241343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.245273:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760794.245278:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c30458 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29237248->29241343] (req 29237248->29241343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.245287:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c30458 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29237248->29241343] (req 29237248->29241343) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.245307:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c30458 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29237248->29241343] (req 29237248->29241343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.245315:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c30458 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29237248->29241343] (req 29237248->29241343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760794.245486:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2879180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512778240:12345-192.168.203.56@tcp:4:dd.0 Request processed in 38436us (38648us total) trans 4294974457 rc 0/0 00000100:00100000:1.0:1789760794.245497:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7168 00000100:00100000:1.0:1789760794.245507:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 773738359119 time 773771334120 state 0x2 ref 0 00000100:00100000:0.0:1789760794.250174:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760794.250301:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512778368 00000100:00100000:2.0:1789760794.250312:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512778368 00000100:00100000:2.0:1789760794.250371:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 773738359119 time 773771334120 state 0x0 ref 1 00000100:00100000:2.0:1789760794.250386:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 773771594536 time 773771334120 state 0x1 ref 1 00000100:00100000:2.0:1789760794.250420:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 773771594536 time 773814719755 state 0x0 ref 1 00000100:00100000:2.0:1789760794.250432:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac437c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512778368:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760794.250482:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 29241344, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0xe007 00010000:00010000:2.0:1789760794.250492:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c3045f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.250507:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=773s v=(0 0) 00010000:00010000:2.0:1789760794.250511:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760794.250517:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c3045f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29241344->29245439] (req 29241344->29245439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.250531:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c3045f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29241344->29245439] (req 29241344->29245439) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760794.285170:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974458 is committed 00002000:00010000:0.0:1789760794.285271:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 29241344 -> 29245440 00010000:00010000:0.0:1789760794.285287:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c3045f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29241344->29245439] (req 29241344->29245439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.285301:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c3045f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29241344->29245439] (req 29241344->29245439) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.285311:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c3045f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29241344->29245439] (req 29241344->29245439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.285318:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760794.285323:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c3045f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29241344->29245439] (req 29241344->29245439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.285406:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c3045f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29241344->29245439] (req 29241344->29245439) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.285431:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c3045f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29241344->29245439] (req 29241344->29245439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.285438:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c3045f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29241344->29245439] (req 29241344->29245439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760794.285529:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac437c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512778368:12345-192.168.203.56@tcp:4:dd.0 Request processed in 35094us (35361us total) trans 4294974458 rc 0/0 00000100:00100000:0.0:1789760794.285541:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7169 00000100:00100000:0.0:1789760794.285550:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 773771594536 time 773814719755 state 0x2 ref 0 00000100:00100000:3.0:1789760794.288635:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760794.288734:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512778496 00000100:00100000:1.0:1789760794.288744:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512778496 00000100:00100000:1.0:1789760794.288795:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 773771594536 time 773814719755 state 0x0 ref 1 00000100:00100000:1.0:1789760794.288812:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 773814980171 time 773814719755 state 0x1 ref 1 00000100:00100000:1.0:1789760794.288835:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 773814980171 time 773853135293 state 0x0 ref 1 00000100:00100000:1.0:1789760794.288847:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febc6bd4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512778496:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760794.288892:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 29245440, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0xe007 00010000:00010000:1.0:1789760794.288902:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c30466 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.288917:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=773s v=(0 0) 00010000:00010000:1.0:1789760794.288922:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760794.288927:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c30466 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29245440->29249535] (req 29245440->29249535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.288957:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c30466 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29245440->29249535] (req 29245440->29249535) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760794.324759:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974459 is committed 00002000:00010000:1.0:1789760794.325003:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 29245440 -> 29249536 00010000:00010000:1.0:1789760794.325018:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c30466 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29245440->29249535] (req 29245440->29249535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.325031:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c30466 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29245440->29249535] (req 29245440->29249535) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.325040:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c30466 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29245440->29249535] (req 29245440->29249535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.325049:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760794.325053:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c30466 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29245440->29249535] (req 29245440->29249535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.325062:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c30466 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29245440->29249535] (req 29245440->29249535) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.325084:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c30466 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29245440->29249535] (req 29245440->29249535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.325093:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c30466 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29245440->29249535] (req 29245440->29249535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760794.325185:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febc6bd4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512778496:12345-192.168.203.56@tcp:4:dd.0 Request processed in 36336us (36555us total) trans 4294974459 rc 0/0 00000100:00100000:1.0:1789760794.325195:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7170 00000100:00100000:1.0:1789760794.325204:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 773814980171 time 773853135293 state 0x2 ref 0 00000100:00100000:3.0:1789760794.327854:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760794.327970:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512778624 00000100:00100000:1.0:1789760794.327980:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512778624 00000100:00100000:1.0:1789760794.328030:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 773814980171 time 773853135293 state 0x0 ref 1 00000100:00100000:1.0:1789760794.328045:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 773853395709 time 773853135293 state 0x1 ref 1 00000100:00100000:1.0:1789760794.328076:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 773853395709 time 773892374962 state 0x0 ref 1 00000100:00100000:1.0:1789760794.328088:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4360a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512778624:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760794.328144:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 29249536, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0xe007 00010000:00010000:1.0:1789760794.328153:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c3046d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.328169:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=773s v=(0 0) 00010000:00010000:1.0:1789760794.328174:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760794.328181:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c3046d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29249536->29253631] (req 29249536->29253631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.328195:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c3046d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29249536->29253631] (req 29249536->29253631) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760794.368773:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 29249536 -> 29253632 00010000:00010000:1.0:1789760794.368793:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c3046d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29249536->29253631] (req 29249536->29253631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.368807:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c3046d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29249536->29253631] (req 29249536->29253631) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.368817:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c3046d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29249536->29253631] (req 29249536->29253631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.368824:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760794.368830:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c3046d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29249536->29253631] (req 29249536->29253631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.368840:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c3046d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29249536->29253631] (req 29249536->29253631) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.368862:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c3046d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29249536->29253631] (req 29249536->29253631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.368875:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c3046d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29249536->29253631] (req 29249536->29253631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760794.368974:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4360a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2359:x1876699512778624:12345-192.168.203.56@tcp:4:dd.0 Request processed in 40883us (41124us total) trans 4294974460 rc 0/0 00000100:00100000:1.0:1789760794.368983:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7171 00000100:00100000:1.0:1789760794.368993:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 773853395709 time 773892374962 state 0x2 ref 0 00000001:00080000:2.0:1789760794.369366:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974460 is committed 00000100:00100000:0.0:1789760794.371730:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760794.371891:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512778752 00000100:00100000:2.0:1789760794.371901:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512778752 00000100:00100000:2.0:1789760794.371964:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 773853395709 time 773892374962 state 0x0 ref 1 00000100:00100000:2.0:1789760794.371978:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 773892635378 time 773892374962 state 0x1 ref 1 00000100:00100000:2.0:1789760794.372202:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 773892635378 time 773936500081 state 0x0 ref 1 00000100:00100000:2.0:1789760794.372218:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac437d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512778752:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760794.372266:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 29253632, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0xe007 00010000:00010000:2.0:1789760794.372275:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c30474 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.372291:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=773s v=(0 0) 00010000:00010000:2.0:1789760794.372295:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760794.372302:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c30474 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29253632->29257727] (req 29253632->29257727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.372317:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c30474 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29253632->29257727] (req 29253632->29257727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760794.414619:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 29253632 -> 29257728 00010000:00010000:1.0:1789760794.414640:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c30474 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29253632->29257727] (req 29253632->29257727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.414653:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c30474 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29253632->29257727] (req 29253632->29257727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.414661:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c30474 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29253632->29257727] (req 29253632->29257727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.414668:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760794.414672:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c30474 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29253632->29257727] (req 29253632->29257727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.414681:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c30474 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29253632->29257727] (req 29253632->29257727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.414736:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c30474 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29253632->29257727] (req 29253632->29257727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.414745:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c30474 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29253632->29257727] (req 29253632->29257727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760794.414828:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac437d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2360:x1876699512778752:12345-192.168.203.56@tcp:4:dd.0 Request processed in 42610us (43134us total) trans 4294974461 rc 0/0 00000100:00100000:1.0:1789760794.414838:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7172 00000100:00100000:1.0:1789760794.414848:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 773892635378 time 773936500081 state 0x2 ref 0 00000001:00080000:2.0:1789760794.415255:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974461 is committed 00000100:00100000:3.0:1789760794.418086:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760794.418186:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512778880 00000100:00100000:1.0:1789760794.418201:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512778880 00000100:00100000:1.0:1789760794.418693:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 773892635378 time 773936500081 state 0x0 ref 1 00000100:00100000:1.0:1789760794.418738:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 773936760497 time 773936500081 state 0x1 ref 1 00000100:00100000:1.0:1789760794.418760:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 773936760497 time 773983060381 state 0x0 ref 1 00000100:00100000:1.0:1789760794.418772:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4362d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512778880:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760794.418824:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 29257728, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0xe007 00010000:00010000:1.0:1789760794.418833:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c3047b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.418848:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=773s v=(0 0) 00010000:00010000:1.0:1789760794.418852:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760794.418860:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c3047b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29257728->29261823] (req 29257728->29261823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.418873:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c3047b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29257728->29261823] (req 29257728->29261823) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760794.445578:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974462 is committed 00002000:00010000:1.0:1789760794.446837:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 29257728 -> 29261824 00010000:00010000:1.0:1789760794.446857:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c3047b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29257728->29261823] (req 29257728->29261823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.446870:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c3047b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29257728->29261823] (req 29257728->29261823) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.446878:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c3047b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29257728->29261823] (req 29257728->29261823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.446884:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760794.446889:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c3047b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29257728->29261823] (req 29257728->29261823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.446897:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c3047b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29257728->29261823] (req 29257728->29261823) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.446919:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c3047b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29257728->29261823] (req 29257728->29261823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.446927:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c3047b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29257728->29261823] (req 29257728->29261823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760794.447036:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4362d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512778880:12345-192.168.203.56@tcp:4:dd.0 Request processed in 28262us (28954us total) trans 4294974462 rc 0/0 00000100:00100000:1.0:1789760794.447046:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7173 00000100:00100000:1.0:1789760794.447055:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 773936760497 time 773983060381 state 0x2 ref 0 00000100:00100000:0.0:1789760794.457021:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760794.457142:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512779008 00000100:00100000:1.0:1789760794.457152:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512779008 00000100:00100000:1.0:1789760794.457215:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 773936760497 time 773983060381 state 0x0 ref 1 00000100:00100000:1.0:1789760794.457230:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 773983320797 time 773983060381 state 0x1 ref 1 00000100:00100000:1.0:1789760794.457293:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 773983320797 time 774021592297 state 0x0 ref 1 00000100:00100000:1.0:1789760794.457307:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac437df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512779008:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760794.457357:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 29261824, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0xe007 00010000:00010000:1.0:1789760794.457365:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c30482 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.457382:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=774s v=(0 0) 00010000:00010000:1.0:1789760794.457387:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760794.457394:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c30482 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29261824->29265919] (req 29261824->29265919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.457419:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c30482 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29261824->29265919] (req 29261824->29265919) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760794.507869:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974463 is committed 00002000:00010000:1.0:1789760794.508750:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 29261824 -> 29265920 00010000:00010000:1.0:1789760794.508768:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c30482 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29261824->29265919] (req 29261824->29265919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.508781:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c30482 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29261824->29265919] (req 29261824->29265919) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.508790:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c30482 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29261824->29265919] (req 29261824->29265919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.508797:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760794.508802:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c30482 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29261824->29265919] (req 29261824->29265919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.508811:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c30482 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29261824->29265919] (req 29261824->29265919) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.508832:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c30482 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29261824->29265919] (req 29261824->29265919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.508840:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c30482 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29261824->29265919] (req 29261824->29265919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760794.509005:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac437df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512779008:12345-192.168.203.56@tcp:4:dd.0 Request processed in 51695us (51988us total) trans 4294974463 rc 0/0 00000100:00100000:1.0:1789760794.509019:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7174 00000100:00100000:1.0:1789760794.509029:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 773983320797 time 774021592297 state 0x2 ref 0 00000100:00100000:0.0:1789760794.516440:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760794.516575:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512779136 00000100:00100000:1.0:1789760794.516586:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512779136 00000100:00100000:1.0:1789760794.516647:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 773983320797 time 774021592297 state 0x0 ref 1 00000100:00100000:1.0:1789760794.516664:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 774021852713 time 774021592297 state 0x1 ref 1 00000100:00100000:1.0:1789760794.516693:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 774021852713 time 774080992675 state 0x0 ref 1 00000100:00100000:1.0:1789760794.516740:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac3353100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512779136:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760794.516791:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 29265920, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0xe007 00010000:00010000:1.0:1789760794.516800:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c30489 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.516815:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=774s v=(0 0) 00010000:00010000:1.0:1789760794.516820:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760794.516825:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c30489 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29265920->29270015] (req 29265920->29270015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.516839:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c30489 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29265920->29270015] (req 29265920->29270015) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760794.578178:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 29265920 -> 29270016 00002000:00010000:1.0:1789760794.578188:0:13063:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760793 -> 1789760794 00002000:00010000:1.0:1789760794.578191:0:13063:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760793 -> 1789760794 00010000:00010000:1.0:1789760794.578220:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c30489 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29265920->29270015] (req 29265920->29270015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.578234:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c30489 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29265920->29270015] (req 29265920->29270015) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.578245:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c30489 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29265920->29270015] (req 29265920->29270015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.578253:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760794.578262:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c30489 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29265920->29270015] (req 29265920->29270015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.578271:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c30489 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29265920->29270015] (req 29265920->29270015) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.578289:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c30489 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29265920->29270015] (req 29265920->29270015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.578297:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c30489 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29265920->29270015] (req 29265920->29270015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760794.578847:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac3353100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2361:x1876699512779136:12345-192.168.203.56@tcp:4:dd.0 Request processed in 61649us (61955us total) trans 4294974464 rc 0/0 00000100:00100000:1.0:1789760794.578858:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7175 00000100:00100000:1.0:1789760794.578868:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 774021852713 time 774080992675 state 0x2 ref 0 00000001:00080000:3.0:1789760794.579021:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974464 is committed 00000100:00100000:0.0:1789760794.591153:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760794.591508:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512779264 00000100:00100000:1.0:1789760794.591518:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512779264 00000100:00100000:1.0:1789760794.591573:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 774021852713 time 774080992675 state 0x0 ref 1 00000100:00100000:1.0:1789760794.591588:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 774081253091 time 774080992675 state 0x1 ref 1 00000100:00100000:1.0:1789760794.591617:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 774081253091 time 774155916272 state 0x0 ref 1 00000100:00100000:1.0:1789760794.591631:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac3351180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512779264:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760794.591677:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 29270016, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0xe007 00010000:00010000:1.0:1789760794.591685:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c30490 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.591731:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=774s v=(0 0) 00010000:00010000:1.0:1789760794.591736:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760794.591743:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c30490 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29270016->29274111] (req 29270016->29274111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.591756:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c30490 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29270016->29274111] (req 29270016->29274111) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760794.620569:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974465 is committed 00002000:00010000:1.0:1789760794.620666:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 29270016 -> 29274112 00010000:00010000:1.0:1789760794.620682:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c30490 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29270016->29274111] (req 29270016->29274111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.621018:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c30490 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29270016->29274111] (req 29270016->29274111) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.621031:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c30490 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29270016->29274111] (req 29270016->29274111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.621040:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760794.621053:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c30490 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29270016->29274111] (req 29270016->29274111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.621060:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c30490 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29270016->29274111] (req 29270016->29274111) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.621075:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c30490 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29270016->29274111] (req 29270016->29274111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.621080:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c30490 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29270016->29274111] (req 29270016->29274111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760794.621201:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac3351180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512779264:12345-192.168.203.56@tcp:4:dd.0 Request processed in 29568us (30053us total) trans 4294974465 rc 0/0 00000100:00100000:1.0:1789760794.621212:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7176 00000100:00100000:1.0:1789760794.621221:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 774081253091 time 774155916272 state 0x2 ref 0 00000100:00100000:0.0:1789760794.626806:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760794.629123:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512779392 00000100:00100000:1.0:1789760794.629146:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512779392 00000100:00100000:1.0:1789760794.629208:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 774081253091 time 774155916272 state 0x0 ref 1 00000100:00100000:1.0:1789760794.629223:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 774156176688 time 774155916272 state 0x1 ref 1 00000100:00100000:1.0:1789760794.629259:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 774156176688 time 774193558582 state 0x0 ref 1 00000100:00100000:1.0:1789760794.629272:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac3352a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512779392:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760794.629319:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 29274112, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0xe007 00010000:00010000:1.0:1789760794.629329:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c30497 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.629344:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=774s v=(0 0) 00010000:00010000:1.0:1789760794.629348:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760794.629355:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c30497 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29274112->29278207] (req 29274112->29278207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.629369:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c30497 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29274112->29278207] (req 29274112->29278207) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760794.679399:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974466 is committed 00002000:00010000:2.0:1789760794.679537:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 29274112 -> 29278208 00010000:00010000:2.0:1789760794.679558:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c30497 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29274112->29278207] (req 29274112->29278207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.679572:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c30497 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29274112->29278207] (req 29274112->29278207) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.679581:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c30497 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29274112->29278207] (req 29274112->29278207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.679591:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760794.679595:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c30497 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29274112->29278207] (req 29274112->29278207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.679621:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c30497 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29274112->29278207] (req 29274112->29278207) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.679641:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c30497 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29274112->29278207] (req 29274112->29278207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.679652:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c30497 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29274112->29278207] (req 29274112->29278207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760794.679783:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac3352a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512779392:12345-192.168.203.56@tcp:4:dd.0 Request processed in 50509us (52983us total) trans 4294974466 rc 0/0 00000100:00100000:2.0:1789760794.679797:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7177 00000100:00100000:2.0:1789760794.679811:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 774156176688 time 774193558582 state 0x2 ref 0 00000100:00100000:0.0:1789760794.683243:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760794.683391:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512779520 00000100:00100000:1.0:1789760794.683402:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512779520 00000100:00100000:1.0:1789760794.683525:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 774156176688 time 774193558582 state 0x0 ref 1 00000100:00100000:1.0:1789760794.683543:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 774193818998 time 774193558582 state 0x1 ref 1 00000100:00100000:1.0:1789760794.683569:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 774193818998 time 774247869501 state 0x0 ref 1 00000100:00100000:1.0:1789760794.683580:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac3352300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512779520:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760794.683628:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 29278208, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0xe007 00010000:00010000:1.0:1789760794.683637:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c3049e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.683653:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=774s v=(0 0) 00010000:00010000:1.0:1789760794.683658:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760794.683665:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c3049e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29278208->29282303] (req 29278208->29282303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.683679:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c3049e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29278208->29282303] (req 29278208->29282303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760794.726515:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974467 is committed 00002000:00010000:3.0:1789760794.726675:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 29278208 -> 29282304 00010000:00010000:3.0:1789760794.726691:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c3049e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29278208->29282303] (req 29278208->29282303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.726732:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c3049e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29278208->29282303] (req 29278208->29282303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.726741:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c3049e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29278208->29282303] (req 29278208->29282303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.726748:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760794.726752:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c3049e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29278208->29282303] (req 29278208->29282303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.726761:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c3049e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29278208->29282303] (req 29278208->29282303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.726779:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c3049e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29278208->29282303] (req 29278208->29282303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.726786:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c3049e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29278208->29282303] (req 29278208->29282303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760794.726860:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac3352300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512779520:12345-192.168.203.56@tcp:4:dd.0 Request processed in 43278us (43624us total) trans 4294974467 rc 0/0 00000100:00100000:3.0:1789760794.726873:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7178 00000100:00100000:3.0:1789760794.726881:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 774193818998 time 774247869501 state 0x2 ref 0 00000100:00100000:0.0:1789760794.729808:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760794.729953:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512779648 00000100:00100000:1.0:1789760794.729963:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512779648 00000100:00100000:1.0:1789760794.730019:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 774193818998 time 774247869501 state 0x0 ref 1 00000100:00100000:1.0:1789760794.730035:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 774248129917 time 774247869501 state 0x1 ref 1 00000100:00100000:1.0:1789760794.730067:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 774248129917 time 774294366584 state 0x0 ref 1 00000100:00100000:1.0:1789760794.730080:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac3353800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512779648:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760794.730131:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 29282304, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0xe007 00010000:00010000:1.0:1789760794.730140:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c304a5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.730158:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=774s v=(0 0) 00010000:00010000:1.0:1789760794.730162:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760794.730170:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c304a5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29282304->29286399] (req 29282304->29286399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.730184:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c304a5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29282304->29286399] (req 29282304->29286399) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760794.773466:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974468 is committed 00002000:00010000:3.0:1789760794.774125:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 29282304 -> 29286400 00010000:00010000:3.0:1789760794.774144:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c304a5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29282304->29286399] (req 29282304->29286399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.774157:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c304a5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29282304->29286399] (req 29282304->29286399) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.774166:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c304a5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29282304->29286399] (req 29282304->29286399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.774174:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760794.774179:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c304a5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29282304->29286399] (req 29282304->29286399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.774188:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c304a5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29282304->29286399] (req 29282304->29286399) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.774208:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c304a5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29282304->29286399] (req 29282304->29286399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.774216:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c304a5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29282304->29286399] (req 29282304->29286399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760794.774428:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac3353800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512779648:12345-192.168.203.56@tcp:4:dd.0 Request processed in 44346us (44625us total) trans 4294974468 rc 0/0 00000100:00100000:3.0:1789760794.774440:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7179 00000100:00100000:3.0:1789760794.774449:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 774248129917 time 774294366584 state 0x2 ref 0 00000100:00100000:1.0:1789760794.777393:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760794.777526:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512779776 00000100:00100000:2.0:1789760794.777537:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512779776 00000100:00100000:2.0:1789760794.777589:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 774248129917 time 774294366584 state 0x0 ref 1 00000100:00100000:2.0:1789760794.777614:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 774294627000 time 774294366584 state 0x1 ref 1 00000100:00100000:2.0:1789760794.777643:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 774294627000 time 774341942639 state 0x0 ref 1 00000100:00100000:2.0:1789760794.777654:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac531e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512779776:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760794.777733:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 29286400, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0xe007 00010000:00010000:2.0:1789760794.777741:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c304ac lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.777757:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=774s v=(0 0) 00010000:00010000:2.0:1789760794.777761:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760794.777768:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c304ac lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29286400->29290495] (req 29286400->29290495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.777781:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c304ac lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29286400->29290495] (req 29286400->29290495) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760794.807992:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974469 is committed 00002000:00010000:2.0:1789760794.808084:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 29286400 -> 29290496 00010000:00010000:2.0:1789760794.808100:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c304ac lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29286400->29290495] (req 29286400->29290495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.808113:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c304ac lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29286400->29290495] (req 29286400->29290495) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.808122:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c304ac lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29286400->29290495] (req 29286400->29290495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.808130:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760794.808135:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c304ac lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29286400->29290495] (req 29286400->29290495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.808144:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c304ac lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29286400->29290495] (req 29286400->29290495) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.808164:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c304ac lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29286400->29290495] (req 29286400->29290495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.808172:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c304ac lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29286400->29290495] (req 29286400->29290495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760794.808270:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac531e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512779776:12345-192.168.203.56@tcp:4:dd.0 Request processed in 30612us (30881us total) trans 4294974469 rc 0/0 00000100:00100000:2.0:1789760794.808280:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7180 00000100:00100000:2.0:1789760794.808290:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 774294627000 time 774341942639 state 0x2 ref 0 00000100:00100000:3.0:1789760794.811119:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760794.811751:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512779904 00000100:00100000:1.0:1789760794.811761:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512779904 00000100:00100000:1.0:1789760794.811812:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 774294627000 time 774341942639 state 0x0 ref 1 00000100:00100000:1.0:1789760794.811826:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 774342203055 time 774341942639 state 0x1 ref 1 00000100:00100000:1.0:1789760794.811853:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 774342203055 time 774376153529 state 0x0 ref 1 00000100:00100000:1.0:1789760794.811864:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4363100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512779904:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760794.811912:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 29290496, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0xe007 00010000:00010000:1.0:1789760794.811921:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c304b3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.811946:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=774s v=(0 0) 00010000:00010000:1.0:1789760794.811951:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760794.811957:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c304b3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29290496->29294591] (req 29290496->29294591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.811972:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c304b3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29290496->29294591] (req 29290496->29294591) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760794.839571:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974470 is committed 00002000:00010000:3.0:1789760794.839768:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 29290496 -> 29294592 00010000:00010000:3.0:1789760794.839784:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c304b3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29290496->29294591] (req 29290496->29294591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.839798:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c304b3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29290496->29294591] (req 29290496->29294591) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.839807:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c304b3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29290496->29294591] (req 29290496->29294591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.839815:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760794.839820:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c304b3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29290496->29294591] (req 29290496->29294591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.839829:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c304b3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29290496->29294591] (req 29290496->29294591) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.839850:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c304b3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29290496->29294591] (req 29290496->29294591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.839858:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ba7118fa/0xe6aa143190c304b3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29290496->29294591] (req 29290496->29294591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760794.840008:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4363100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512779904:12345-192.168.203.56@tcp:4:dd.0 Request processed in 28139us (28895us total) trans 4294974470 rc 0/0 00000100:00100000:3.0:1789760794.840019:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7181 00000100:00100000:3.0:1789760794.840029:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 774342203055 time 774376153529 state 0x2 ref 0 00000100:00100000:1.0:1789760794.842958:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760794.843040:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512780032 00000100:00100000:3.0:1789760794.843050:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512780032 00000100:00100000:3.0:1789760794.843102:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 774342203055 time 774376153529 state 0x0 ref 1 00000100:00100000:3.0:1789760794.843118:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 774376413945 time 774376153529 state 0x1 ref 1 00000100:00100000:3.0:1789760794.843198:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 774376413945 time 774407497182 state 0x0 ref 1 00000100:00100000:3.0:1789760794.843211:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac531ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512780032:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760794.843258:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 29294592, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0xe007 00010000:00010000:3.0:1789760794.843267:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c304ba lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.843283:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=774s v=(0 0) 00010000:00010000:3.0:1789760794.843288:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760794.843294:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c304ba lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29294592->29298687] (req 29294592->29298687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.843313:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c304ba lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29294592->29298687] (req 29294592->29298687) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760794.874630:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 29294592 -> 29298688 00010000:00010000:3.0:1789760794.874649:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c304ba lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29294592->29298687] (req 29294592->29298687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.874663:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c304ba lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29294592->29298687] (req 29294592->29298687) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.874672:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c304ba lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29294592->29298687] (req 29294592->29298687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.874679:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760794.874684:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c304ba lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29294592->29298687] (req 29294592->29298687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.874694:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c304ba lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29294592->29298687] (req 29294592->29298687) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.874736:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c304ba lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29294592->29298687] (req 29294592->29298687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.874744:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c304ba lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29294592->29298687] (req 29294592->29298687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760794.874828:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac531ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2358:x1876699512780032:12345-192.168.203.56@tcp:4:dd.0 Request processed in 31616us (31876us total) trans 4294974471 rc 0/0 00000100:00100000:3.0:1789760794.874838:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7182 00000100:00100000:3.0:1789760794.874847:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 774376413945 time 774407497182 state 0x2 ref 0 00000001:00080000:2.0:1789760794.877118:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974471 is committed 00000100:00100000:1.0:1789760794.877453:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760794.877527:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512780160 00000100:00100000:2.0:1789760794.877537:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512780160 00000100:00100000:2.0:1789760794.877590:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 774376413945 time 774407497182 state 0x0 ref 1 00000100:00100000:2.0:1789760794.877605:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 774407757598 time 774407497182 state 0x1 ref 1 00000100:00100000:2.0:1789760794.877642:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 774407757598 time 774441942338 state 0x0 ref 1 00000100:00100000:2.0:1789760794.877654:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac531f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512780160:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760794.877736:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 29298688, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0xe007 00010000:00010000:2.0:1789760794.877744:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c304c1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.877761:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=774s v=(0 0) 00010000:00010000:2.0:1789760794.877765:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760794.877772:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c304c1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29298688->29302783] (req 29298688->29302783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.877786:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c304c1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29298688->29302783] (req 29298688->29302783) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760794.911116:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974472 is committed 00002000:00010000:0.0:1789760794.911230:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 29298688 -> 29302784 00010000:00010000:0.0:1789760794.911246:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c304c1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29298688->29302783] (req 29298688->29302783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.911260:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c304c1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29298688->29302783] (req 29298688->29302783) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.911276:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c304c1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29298688->29302783] (req 29298688->29302783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.911284:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760794.911288:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c304c1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29298688->29302783] (req 29298688->29302783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.911297:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c304c1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29298688->29302783] (req 29298688->29302783) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.911315:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c304c1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29298688->29302783] (req 29298688->29302783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.911323:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c304c1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29298688->29302783] (req 29298688->29302783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760794.911416:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac531f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512780160:12345-192.168.203.56@tcp:4:dd.0 Request processed in 33759us (33964us total) trans 4294974472 rc 0/0 00000100:00100000:0.0:1789760794.911426:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7183 00000100:00100000:0.0:1789760794.911436:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 774407757598 time 774441942338 state 0x2 ref 0 00000100:00100000:1.0:1789760794.914374:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760794.914453:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512780288 00000100:00100000:3.0:1789760794.914463:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512780288 00000100:00100000:3.0:1789760794.914514:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 774407757598 time 774441942338 state 0x0 ref 1 00000100:00100000:3.0:1789760794.914528:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 774442202754 time 774441942338 state 0x1 ref 1 00000100:00100000:3.0:1789760794.914553:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 774442202754 time 774478852507 state 0x0 ref 1 00000100:00100000:3.0:1789760794.914565:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac531ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512780288:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760794.914620:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 29302784, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0xe007 00010000:00010000:3.0:1789760794.914629:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c304c8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.914644:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=774s v=(0 0) 00010000:00010000:3.0:1789760794.914648:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760794.914654:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c304c8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29302784->29306879] (req 29302784->29306879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.914667:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c304c8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29302784->29306879] (req 29302784->29306879) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760794.948193:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974473 is committed 00002000:00010000:1.0:1789760794.948294:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 29302784 -> 29306880 00010000:00010000:1.0:1789760794.948310:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c304c8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29302784->29306879] (req 29302784->29306879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.948324:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c304c8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29302784->29306879] (req 29302784->29306879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.948395:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c304c8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29302784->29306879] (req 29302784->29306879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.948403:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760794.948408:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c304c8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29302784->29306879] (req 29302784->29306879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.948417:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c304c8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29302784->29306879] (req 29302784->29306879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.948436:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c304c8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29302784->29306879] (req 29302784->29306879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.948445:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000bf55689/0xe6aa143190c304c8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29302784->29306879] (req 29302784->29306879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760794.948576:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac531ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512780288:12345-192.168.203.56@tcp:4:dd.0 Request processed in 34008us (34205us total) trans 4294974473 rc 0/0 00000100:00100000:1.0:1789760794.948587:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7184 00000100:00100000:1.0:1789760794.948597:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 774442202754 time 774478852507 state 0x2 ref 0 00000100:00100000:1.0:1789760794.951744:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760794.952117:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512780416 00000100:00100000:3.0:1789760794.952128:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512780416 00000100:00100000:3.0:1789760794.952179:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 774442202754 time 774478852507 state 0x0 ref 1 00000100:00100000:3.0:1789760794.952194:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 774479112923 time 774478852507 state 0x1 ref 1 00000100:00100000:3.0:1789760794.952224:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 774479112923 time 774516524695 state 0x0 ref 1 00000100:00100000:3.0:1789760794.952236:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac531c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512780416:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760794.952280:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 29306880, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0xe007 00010000:00010000:3.0:1789760794.952288:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c304cf lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.952304:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=774s v=(0 0) 00010000:00010000:3.0:1789760794.952308:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760794.952314:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c304cf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29306880->29310975] (req 29306880->29310975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.952338:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c304cf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29306880->29310975] (req 29306880->29310975) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760794.980061:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974474 is committed 00002000:00010000:2.0:1789760794.980299:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 29306880 -> 29310976 00010000:00010000:2.0:1789760794.980316:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c304cf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29306880->29310975] (req 29306880->29310975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.980340:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c304cf lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29306880->29310975] (req 29306880->29310975) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.980349:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c304cf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29306880->29310975] (req 29306880->29310975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.980357:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f2c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760794.980362:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c304cf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29306880->29310975] (req 29306880->29310975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.980371:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c304cf lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29306880->29310975] (req 29306880->29310975) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.980389:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c304cf lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29306880->29310975] (req 29306880->29310975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.980397:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ad476f4c/0xe6aa143190c304cf lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29306880->29310975] (req 29306880->29310975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760794.980490:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac531c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512780416:12345-192.168.203.56@tcp:4:dd.0 Request processed in 28251us (28749us total) trans 4294974474 rc 0/0 00000100:00100000:2.0:1789760794.980500:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7185 00000100:00100000:2.0:1789760794.980508:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 774479112923 time 774516524695 state 0x2 ref 0 00000100:00100000:0.0:1789760794.983370:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760794.983484:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512780544 00000100:00100000:2.0:1789760794.983494:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512780544 00000100:00100000:2.0:1789760794.983547:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 774479112923 time 774516524695 state 0x0 ref 1 00000100:00100000:2.0:1789760794.983563:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 774516785111 time 774516524695 state 0x1 ref 1 00000100:00100000:2.0:1789760794.983592:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 774516785111 time 774547891452 state 0x0 ref 1 00000100:00100000:2.0:1789760794.983613:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac28a2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512780544:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760794.983661:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 29310976, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0xe007 00010000:00010000:2.0:1789760794.983669:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c304d6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.983685:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=774s v=(0 0) 00010000:00010000:2.0:1789760794.983689:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760794.983724:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c304d6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29310976->29315071] (req 29310976->29315071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.983739:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c304d6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29310976->29315071] (req 29310976->29315071) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.994847:0:5835:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:3.0:1789760795.014720:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974475 is committed 00002000:00010000:0.0:1789760795.014832:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 29310976 -> 29315072 00010000:00010000:0.0:1789760795.014848:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c304d6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29310976->29315071] (req 29310976->29315071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.014862:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c304d6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29310976->29315071] (req 29310976->29315071) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.014871:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c304d6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29310976->29315071] (req 29310976->29315071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.014878:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760795.014883:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c304d6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29310976->29315071] (req 29310976->29315071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.014892:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c304d6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29310976->29315071] (req 29310976->29315071) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.014979:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c304d6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29310976->29315071] (req 29310976->29315071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.014988:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004d051e87/0xe6aa143190c304d6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29310976->29315071] (req 29310976->29315071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760795.015108:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac28a2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512780544:12345-192.168.203.56@tcp:4:dd.0 Request processed in 31500us (31744us total) trans 4294974475 rc 0/0 00000100:00100000:0.0:1789760795.015118:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7186 00000100:00100000:0.0:1789760795.015128:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 774516785111 time 774547891452 state 0x2 ref 0 00000100:00100000:3.0:1789760795.018291:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760795.019110:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512780672 00000100:00100000:1.0:1789760795.019121:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512780672 00000100:00100000:1.0:1789760795.019223:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 774516785111 time 774547891452 state 0x0 ref 1 00000100:00100000:1.0:1789760795.019238:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 774548151868 time 774547891452 state 0x1 ref 1 00000100:00100000:1.0:1789760795.019304:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 774548151868 time 774583603951 state 0x0 ref 1 00000100:00100000:1.0:1789760795.019317:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4362a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512780672:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760795.019374:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 29315072, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0xe007 00010000:00010000:1.0:1789760795.019383:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c304dd lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.019399:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=774s v=(0 0) 00010000:00010000:1.0:1789760795.019403:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760795.019410:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c304dd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29315072->29319167] (req 29315072->29319167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.019438:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c304dd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29315072->29319167] (req 29315072->29319167) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1789760795.021574:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:2.0:1789760795.048363:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974476 is committed 00002000:00010000:3.0:1789760795.048455:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 29315072 -> 29319168 00010000:00010000:3.0:1789760795.048471:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c304dd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29315072->29319167] (req 29315072->29319167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.048485:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c304dd lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29315072->29319167] (req 29315072->29319167) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.048495:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c304dd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29315072->29319167] (req 29315072->29319167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.048501:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760795.048507:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c304dd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29315072->29319167] (req 29315072->29319167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.048517:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c304dd lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29315072->29319167] (req 29315072->29319167) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.048535:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c304dd lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29315072->29319167] (req 29315072->29319167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.048543:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c304dd lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29315072->29319167] (req 29315072->29319167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760795.048632:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4362a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512780672:12345-192.168.203.56@tcp:4:dd.0 Request processed in 29313us (30344us total) trans 4294974476 rc 0/0 00000100:00100000:3.0:1789760795.048640:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7187 00000100:00100000:3.0:1789760795.048650:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 774548151868 time 774583603951 state 0x2 ref 0 00000100:00100000:3.0:1789760795.051429:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760795.051557:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512780800 00000100:00100000:2.0:1789760795.051566:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512780800 00000100:00100000:2.0:1789760795.051623:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 774548151868 time 774583603951 state 0x0 ref 1 00000100:00100000:2.0:1789760795.051637:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 774583864367 time 774583603951 state 0x1 ref 1 00000100:00100000:2.0:1789760795.051665:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 774583864367 time 774615964397 state 0x0 ref 1 00000100:00100000:2.0:1789760795.051675:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4360700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512780800:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760795.051743:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 29319168, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0xe007 00010000:00010000:2.0:1789760795.051752:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c304e4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.051766:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=774s v=(0 0) 00010000:00010000:2.0:1789760795.051770:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760795.051776:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c304e4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29319168->29323263] (req 29319168->29323263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.051790:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c304e4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29319168->29323263] (req 29319168->29323263) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760795.081537:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974477 is committed 00002000:00010000:0.0:1789760795.081731:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 29319168 -> 29323264 00010000:00010000:0.0:1789760795.081749:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c304e4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29319168->29323263] (req 29319168->29323263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.081762:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c304e4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29319168->29323263] (req 29319168->29323263) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.081772:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c304e4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29319168->29323263] (req 29319168->29323263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.081779:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760795.081785:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c304e4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29319168->29323263] (req 29319168->29323263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.081794:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c304e4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29319168->29323263] (req 29319168->29323263) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.081811:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c304e4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29319168->29323263] (req 29319168->29323263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.081819:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e19f1cf6/0xe6aa143190c304e4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29319168->29323263] (req 29319168->29323263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760795.081947:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4360700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512780800:12345-192.168.203.56@tcp:4:dd.0 Request processed in 30259us (30518us total) trans 4294974477 rc 0/0 00000100:00100000:0.0:1789760795.081957:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7188 00000100:00100000:0.0:1789760795.081967:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 774583864367 time 774615964397 state 0x2 ref 0 00000100:00100000:3.0:1789760795.084848:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760795.085497:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512780928 00000100:00100000:2.0:1789760795.085509:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512780928 00000100:00100000:2.0:1789760795.085567:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 774583864367 time 774615964397 state 0x0 ref 1 00000100:00100000:2.0:1789760795.085583:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 774616224813 time 774615964397 state 0x1 ref 1 00000100:00100000:2.0:1789760795.085652:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 774616224813 time 774649950936 state 0x0 ref 1 00000100:00100000:2.0:1789760795.085665:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5304700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512780928:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760795.085741:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 29323264, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0xe007 00010000:00010000:2.0:1789760795.085750:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c304eb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.085766:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=774s v=(0 0) 00010000:00010000:2.0:1789760795.085770:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760795.085776:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c304eb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29323264->29327359] (req 29323264->29327359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.085790:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c304eb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29323264->29327359] (req 29323264->29327359) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760795.136929:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974478 is committed 00002000:00010000:0.0:1789760795.137087:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 29323264 -> 29327360 00010000:00010000:0.0:1789760795.137103:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c304eb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29323264->29327359] (req 29323264->29327359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.137116:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c304eb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29323264->29327359] (req 29323264->29327359) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.137125:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c304eb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29323264->29327359] (req 29323264->29327359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.137133:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760795.137137:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c304eb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29323264->29327359] (req 29323264->29327359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.137146:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c304eb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29323264->29327359] (req 29323264->29327359) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.137167:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c304eb lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29323264->29327359] (req 29323264->29327359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.137175:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000758dc457/0xe6aa143190c304eb lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29323264->29327359] (req 29323264->29327359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760795.137272:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5304700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512780928:12345-192.168.203.56@tcp:4:dd.0 Request processed in 51602us (52426us total) trans 4294974478 rc 0/0 00000100:00100000:0.0:1789760795.137282:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7189 00000100:00100000:0.0:1789760795.137292:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 774616224813 time 774649950936 state 0x2 ref 0 00000100:00100000:3.0:1789760795.139763:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760795.147130:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512781056 00000100:00100000:1.0:1789760795.147144:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512781056 00000100:00100000:1.0:1789760795.147202:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 774616224813 time 774649950936 state 0x0 ref 1 00000100:00100000:1.0:1789760795.147217:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 774650211352 time 774649950936 state 0x1 ref 1 00000100:00100000:1.0:1789760795.147248:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 774650211352 time 774711548261 state 0x0 ref 1 00000100:00100000:1.0:1789760795.147261:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5304380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512781056:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760795.147313:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 29327360, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0xe007 00010000:00010000:1.0:1789760795.147323:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c304f2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.147338:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=774s v=(0 0) 00010000:00010000:1.0:1789760795.147343:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760795.147349:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c304f2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29327360->29331455] (req 29327360->29331455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.147362:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c304f2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29327360->29331455] (req 29327360->29331455) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760795.215962:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974479 is committed 00002000:00010000:2.0:1789760795.216170:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 29327360 -> 29331456 00010000:00010000:2.0:1789760795.216186:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c304f2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29327360->29331455] (req 29327360->29331455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.216200:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c304f2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29327360->29331455] (req 29327360->29331455) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.216213:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c304f2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29327360->29331455] (req 29327360->29331455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.216221:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760795.216226:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c304f2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29327360->29331455] (req 29327360->29331455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.216235:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c304f2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29327360->29331455] (req 29327360->29331455) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.216254:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c304f2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29327360->29331455] (req 29327360->29331455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.216262:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002eb32303/0xe6aa143190c304f2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29327360->29331455] (req 29327360->29331455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760795.216350:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5304380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512781056:12345-192.168.203.56@tcp:4:dd.0 Request processed in 69085us (76593us total) trans 4294974479 rc 0/0 00000100:00100000:2.0:1789760795.216360:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7190 00000100:00100000:2.0:1789760795.216370:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 774650211352 time 774711548261 state 0x2 ref 0 00000100:00100000:2.0:1789760795.222523:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760795.222580:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512781184 00000100:00100000:0.0:1789760795.222590:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512781184 00000100:00100000:0.0:1789760795.222644:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 774650211352 time 774711548261 state 0x0 ref 1 00000100:00100000:0.0:1789760795.222659:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 774711808677 time 774711548261 state 0x1 ref 1 00000100:00100000:0.0:1789760795.222689:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 774711808677 time 774786988905 state 0x0 ref 1 00000100:00100000:0.0:1789760795.222728:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2878380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512781184:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760795.222776:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 29331456, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0xe007 00010000:00010000:0.0:1789760795.222785:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c304f9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.222799:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=774s v=(0 0) 00010000:00010000:0.0:1789760795.222804:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760795.222810:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c304f9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29331456->29335551] (req 29331456->29335551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.222824:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c304f9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29331456->29335551] (req 29331456->29335551) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760795.261971:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974480 is committed 00002000:00010000:0.0:1789760795.262221:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 29331456 -> 29335552 00010000:00010000:0.0:1789760795.262237:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c304f9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29331456->29335551] (req 29331456->29335551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.262256:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c304f9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29331456->29335551] (req 29331456->29335551) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.262265:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c304f9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29331456->29335551] (req 29331456->29335551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.262272:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760795.262277:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c304f9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29331456->29335551] (req 29331456->29335551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.262286:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c304f9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29331456->29335551] (req 29331456->29335551) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.262303:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c304f9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29331456->29335551] (req 29331456->29335551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.262311:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c304f9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29331456->29335551] (req 29331456->29335551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760795.262460:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2878380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512781184:12345-192.168.203.56@tcp:4:dd.0 Request processed in 39730us (39940us total) trans 4294974480 rc 0/0 00000100:00100000:0.0:1789760795.262471:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7191 00000100:00100000:0.0:1789760795.262479:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 774711808677 time 774786988905 state 0x2 ref 0 00000100:00100000:0.0:1789760795.273141:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760795.273252:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512781312 00000100:00100000:2.0:1789760795.273263:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512781312 00000100:00100000:2.0:1789760795.273315:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 774711808677 time 774786988905 state 0x0 ref 1 00000100:00100000:2.0:1789760795.273333:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 774787249321 time 774786988905 state 0x1 ref 1 00000100:00100000:2.0:1789760795.273364:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 774787249321 time 774837663469 state 0x0 ref 1 00000100:00100000:2.0:1789760795.273376:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac28a2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512781312:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760795.273427:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 29335552, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0xe007 00010000:00010000:2.0:1789760795.273436:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c30500 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.273454:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=774s v=(0 0) 00010000:00010000:2.0:1789760795.273459:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760795.273466:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c30500 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29335552->29339647] (req 29335552->29339647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.273481:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c30500 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29335552->29339647] (req 29335552->29339647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760795.341439:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974481 is committed 00002000:00010000:0.0:1789760795.341532:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 29335552 -> 29339648 00010000:00010000:0.0:1789760795.341550:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c30500 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29335552->29339647] (req 29335552->29339647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.341566:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c30500 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29335552->29339647] (req 29335552->29339647) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.341576:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c30500 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29335552->29339647] (req 29335552->29339647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.341584:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760795.341590:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c30500 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29335552->29339647] (req 29335552->29339647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.341600:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c30500 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29335552->29339647] (req 29335552->29339647) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.341842:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c30500 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29335552->29339647] (req 29335552->29339647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.341851:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000eb862760/0xe6aa143190c30500 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29335552->29339647] (req 29335552->29339647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760795.341990:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac28a2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512781312:12345-192.168.203.56@tcp:4:dd.0 Request processed in 68611us (68857us total) trans 4294974481 rc 0/0 00000100:00100000:0.0:1789760795.342001:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7192 00000100:00100000:0.0:1789760795.342011:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 774787249321 time 774837663469 state 0x2 ref 0 00000100:00100000:3.0:1789760795.348665:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760795.348845:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512781440 00000100:00100000:2.0:1789760795.348856:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512781440 00000100:00100000:2.0:1789760795.348915:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 774787249321 time 774837663469 state 0x0 ref 1 00000100:00100000:2.0:1789760795.348942:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 774837923885 time 774837663469 state 0x1 ref 1 00000100:00100000:2.0:1789760795.348971:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 774837923885 time 774913270475 state 0x0 ref 1 00000100:00100000:2.0:1789760795.348983:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5306680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512781440:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760795.349035:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 29339648, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0xe007 00010000:00010000:2.0:1789760795.349044:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c30507 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.349059:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=774s v=(0 0) 00010000:00010000:2.0:1789760795.349063:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760795.349069:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c30507 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29339648->29343743] (req 29339648->29343743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.349083:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c30507 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29339648->29343743] (req 29339648->29343743) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760795.418320:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 29339648 -> 29343744 00010000:00010000:0.0:1789760795.418344:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c30507 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29339648->29343743] (req 29339648->29343743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.418360:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c30507 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29339648->29343743] (req 29339648->29343743) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.418370:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c30507 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29339648->29343743] (req 29339648->29343743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.418379:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760795.418384:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c30507 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29339648->29343743] (req 29339648->29343743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.418394:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c30507 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29339648->29343743] (req 29339648->29343743) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.418555:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c30507 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29339648->29343743] (req 29339648->29343743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.418565:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009e9d50a3/0xe6aa143190c30507 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29339648->29343743] (req 29339648->29343743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760795.418852:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5306680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2358:x1876699512781440:12345-192.168.203.56@tcp:4:dd.0 Request processed in 69865us (70194us total) trans 4294974482 rc 0/0 00000100:00100000:0.0:1789760795.418863:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7193 00000100:00100000:0.0:1789760795.418874:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 774837923885 time 774913270475 state 0x2 ref 0 00000001:00080000:1.0:1789760795.421298:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974482 is committed 00000100:00100000:0.0:1789760795.428859:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760795.428957:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512781568 00000100:00100000:1.0:1789760795.428983:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512781568 00000100:00100000:1.0:1789760795.429037:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 774837923885 time 774913270475 state 0x0 ref 1 00000100:00100000:1.0:1789760795.429051:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 774913530891 time 774913270475 state 0x1 ref 1 00000100:00100000:1.0:1789760795.429082:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 774913530891 time 774993382330 state 0x0 ref 1 00000100:00100000:1.0:1789760795.429095:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac28a0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512781568:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760795.429140:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 29343744, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0xe007 00010000:00010000:1.0:1789760795.429148:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c3050e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.429162:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=774s v=(0 0) 00010000:00010000:1.0:1789760795.429166:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760795.429172:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c3050e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29343744->29347839] (req 29343744->29347839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.429185:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c3050e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29343744->29347839] (req 29343744->29347839) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760795.472458:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 29343744 -> 29347840 00010000:00010000:1.0:1789760795.472476:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c3050e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29343744->29347839] (req 29343744->29347839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.472489:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c3050e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29343744->29347839] (req 29343744->29347839) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.472504:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c3050e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29343744->29347839] (req 29343744->29347839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.472512:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760795.472525:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c3050e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29343744->29347839] (req 29343744->29347839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.472534:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c3050e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29343744->29347839] (req 29343744->29347839) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.472553:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c3050e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29343744->29347839] (req 29343744->29347839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.472561:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ed8ff223/0xe6aa143190c3050e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29343744->29347839] (req 29343744->29347839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760795.472648:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac28a0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2358:x1876699512781568:12345-192.168.203.56@tcp:4:dd.0 Request processed in 43551us (43793us total) trans 4294974483 rc 0/0 00000100:00100000:1.0:1789760795.472656:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7194 00000100:00100000:1.0:1789760795.472665:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 774913530891 time 774993382330 state 0x2 ref 0 00000001:00080000:2.0:1789760795.473232:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974483 is committed 00000100:00100000:0.0:1789760795.484155:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760795.489454:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512781696 00000100:00100000:2.0:1789760795.489471:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512781696 00000100:00100000:2.0:1789760795.489541:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 774913530891 time 774993382330 state 0x0 ref 1 00000100:00100000:2.0:1789760795.489557:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 774993642746 time 774993382330 state 0x1 ref 1 00000100:00100000:2.0:1789760795.489677:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 774993642746 time 775053974866 state 0x0 ref 1 00000100:00100000:2.0:1789760795.489691:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac28a3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512781696:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760795.489760:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 29347840, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0xe007 00010000:00010000:2.0:1789760795.489769:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c30515 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.489808:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=775s v=(0 0) 00010000:00010000:2.0:1789760795.489812:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760795.489820:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c30515 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29347840->29351935] (req 29347840->29351935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.489835:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c30515 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29347840->29351935] (req 29347840->29351935) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760795.545280:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 29347840 -> 29351936 00010000:00010000:2.0:1789760795.545302:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c30515 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29347840->29351935] (req 29347840->29351935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.545316:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c30515 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29347840->29351935] (req 29347840->29351935) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.545324:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c30515 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29347840->29351935] (req 29347840->29351935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.545331:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760795.545335:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c30515 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29347840->29351935] (req 29347840->29351935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.545343:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c30515 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29347840->29351935] (req 29347840->29351935) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.545364:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c30515 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29347840->29351935] (req 29347840->29351935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.545372:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c30515 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29347840->29351935] (req 29347840->29351935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760795.545471:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac28a3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2361:x1876699512781696:12345-192.168.203.56@tcp:4:dd.0 Request processed in 55776us (61320us total) trans 4294974484 rc 0/0 00000100:00100000:2.0:1789760795.545482:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7195 00000100:00100000:2.0:1789760795.545492:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 774993642746 time 775053974866 state 0x2 ref 0 00000100:00100000:3.0:1789760795.548501:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760795.548594:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512781824 00000100:00100000:2.0:1789760795.548614:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512781824 00000100:00100000:2.0:1789760795.548666:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 774993642746 time 775053974866 state 0x0 ref 1 00000100:00100000:2.0:1789760795.548683:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 775054235282 time 775053974866 state 0x1 ref 1 00000100:00100000:2.0:1789760795.548749:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 775054235282 time 775113048461 state 0x0 ref 1 00000100:00100000:2.0:1789760795.548762:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5307100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2360:x1876699512781824:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760795.548812:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 29351936, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0xe007 00010000:00010000:2.0:1789760795.548821:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c3051c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.548836:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=775s v=(0 0) 00010000:00010000:2.0:1789760795.548840:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00000001:00080000:0.0:1789760795.548843:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974484 is committed 00010000:00010000:2.0:1789760795.548847:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c3051c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29351936->29356031] (req 29351936->29356031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.548867:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c3051c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29351936->29356031] (req 29351936->29356031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760795.583047:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974485 is committed 00002000:00010000:2.0:1789760795.583178:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 29351936 -> 29356032 00002000:00010000:2.0:1789760795.583183:0:13063:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760794 -> 1789760795 00002000:00010000:2.0:1789760795.583186:0:13063:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760794 -> 1789760795 00010000:00010000:2.0:1789760795.583201:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c3051c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29351936->29356031] (req 29351936->29356031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.583214:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c3051c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29351936->29356031] (req 29351936->29356031) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.583223:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c3051c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29351936->29356031] (req 29351936->29356031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.583230:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760795.583235:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c3051c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29351936->29356031] (req 29351936->29356031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.583245:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c3051c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29351936->29356031] (req 29351936->29356031) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.583265:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c3051c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29351936->29356031] (req 29351936->29356031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.583281:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000070a6f9fe/0xe6aa143190c3051c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29351936->29356031] (req 29351936->29356031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760795.583378:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5307100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512781824:12345-192.168.203.56@tcp:4:dd.0 Request processed in 34613us (34882us total) trans 4294974485 rc 0/0 00000100:00100000:2.0:1789760795.583388:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7196 00000100:00100000:2.0:1789760795.583397:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 775054235282 time 775113048461 state 0x2 ref 0 00000100:00100000:1.0:1789760795.589902:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760795.590307:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512781952 00000100:00100000:0.0:1789760795.590319:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512781952 00000100:00100000:0.0:1789760795.590380:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 775054235282 time 775113048461 state 0x0 ref 1 00000100:00100000:0.0:1789760795.590396:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 775113308877 time 775113048461 state 0x1 ref 1 00000100:00100000:0.0:1789760795.590422:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 775113308877 time 775154721528 state 0x0 ref 1 00000100:00100000:0.0:1789760795.590434:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4074000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512781952:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760795.590490:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 29356032, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0xe007 00010000:00010000:0.0:1789760795.590504:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c30523 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.590520:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=775s v=(0 0) 00010000:00010000:0.0:1789760795.590525:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760795.590531:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c30523 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29356032->29360127] (req 29356032->29360127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.590544:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c30523 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29356032->29360127] (req 29356032->29360127) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760795.620684:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974486 is committed 00002000:00010000:3.0:1789760795.621002:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 29356032 -> 29360128 00010000:00010000:3.0:1789760795.621018:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c30523 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29356032->29360127] (req 29356032->29360127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.621032:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c30523 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29356032->29360127] (req 29356032->29360127) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.621041:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c30523 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29356032->29360127] (req 29356032->29360127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.621050:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760795.621055:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c30523 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29356032->29360127] (req 29356032->29360127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.621065:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c30523 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29356032->29360127] (req 29356032->29360127) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.621093:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c30523 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29356032->29360127] (req 29356032->29360127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.621101:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c30523 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29356032->29360127] (req 29356032->29360127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760795.621194:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4074000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512781952:12345-192.168.203.56@tcp:4:dd.0 Request processed in 30757us (31299us total) trans 4294974486 rc 0/0 00000100:00100000:3.0:1789760795.621203:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7197 00000100:00100000:3.0:1789760795.621213:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 775113308877 time 775154721528 state 0x2 ref 0 00000100:00100000:0.0:1789760795.626910:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760795.627002:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512782080 00000100:00100000:2.0:1789760795.627012:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512782080 00000100:00100000:2.0:1789760795.627062:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 775113308877 time 775154721528 state 0x0 ref 1 00000100:00100000:2.0:1789760795.627086:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 775154981944 time 775154721528 state 0x1 ref 1 00000100:00100000:2.0:1789760795.627135:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 775154981944 time 775191435334 state 0x0 ref 1 00000100:00100000:2.0:1789760795.627149:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febff933b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512782080:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760795.627195:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 29360128, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0xe007 00010000:00010000:2.0:1789760795.627203:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c3052a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.627220:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=775s v=(0 0) 00010000:00010000:2.0:1789760795.627224:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760795.627230:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c3052a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29360128->29364223] (req 29360128->29364223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.627243:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c3052a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29360128->29364223] (req 29360128->29364223) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760795.660730:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974487 is committed 00002000:00010000:2.0:1789760795.660832:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 29360128 -> 29364224 00002000:00010000:2.0:1789760795.660837:0:13062:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb blocks from disk: 57351 -> 59399 00010000:00010000:2.0:1789760795.660850:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c3052a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29360128->29364223] (req 29360128->29364223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.660863:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c3052a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29360128->29364223] (req 29360128->29364223) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.660872:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c3052a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29360128->29364223] (req 29360128->29364223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.660880:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760795.660885:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c3052a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29360128->29364223] (req 29360128->29364223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.660895:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c3052a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29360128->29364223] (req 29360128->29364223) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.660914:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c3052a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29360128->29364223] (req 29360128->29364223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.660921:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009c8269d9/0xe6aa143190c3052a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29360128->29364223] (req 29360128->29364223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760795.661052:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febff933b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512782080:12345-192.168.203.56@tcp:4:dd.0 Request processed in 33901us (34146us total) trans 4294974487 rc 0/0 00000100:00100000:2.0:1789760795.661063:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7198 00000100:00100000:2.0:1789760795.661072:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 775154981944 time 775191435334 state 0x2 ref 0 00000100:00100000:3.0:1789760795.670134:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760795.670263:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512782208 00000100:00100000:1.0:1789760795.670272:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512782208 00000100:00100000:1.0:1789760795.670317:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 775154981944 time 775191435334 state 0x0 ref 1 00000100:00100000:1.0:1789760795.670395:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 775191695750 time 775191435334 state 0x1 ref 1 00000100:00100000:1.0:1789760795.670422:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 775191695750 time 775234721992 state 0x0 ref 1 00000100:00100000:1.0:1789760795.670519:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5304a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512782208:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760795.670564:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 29364224, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0xe807 00010000:00010000:1.0:1789760795.670573:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c30531 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.670590:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=775s v=(0 0) 00010000:00010000:1.0:1789760795.670594:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760795.670601:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c30531 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29364224->29368319] (req 29364224->29368319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.670615:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c30531 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29364224->29368319] (req 29364224->29368319) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760795.702153:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974488 is committed 00002000:00010000:0.0:1789760795.702516:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 29364224 -> 29368320 00010000:00010000:0.0:1789760795.702531:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c30531 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29364224->29368319] (req 29364224->29368319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.702544:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c30531 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29364224->29368319] (req 29364224->29368319) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.702554:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c30531 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29364224->29368319] (req 29364224->29368319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.702562:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760795.702567:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c30531 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29364224->29368319] (req 29364224->29368319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.702577:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c30531 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29364224->29368319] (req 29364224->29368319) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.702596:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c30531 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29364224->29368319] (req 29364224->29368319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.702603:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c30531 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29364224->29368319] (req 29364224->29368319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760795.702852:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5304a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512782208:12345-192.168.203.56@tcp:4:dd.0 Request processed in 32413us (32721us total) trans 4294974488 rc 0/0 00000100:00100000:0.0:1789760795.702863:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7199 00000100:00100000:0.0:1789760795.702872:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 775191695750 time 775234721992 state 0x2 ref 0 00000100:00100000:0.0:1789760795.706871:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760795.707093:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512782336 00000100:00100000:3.0:1789760795.707102:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512782336 00000100:00100000:3.0:1789760795.707157:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 775191695750 time 775234721992 state 0x0 ref 1 00000100:00100000:3.0:1789760795.707173:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 775234982408 time 775234721992 state 0x1 ref 1 00000100:00100000:3.0:1789760795.707277:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 775234982408 time 775271576242 state 0x0 ref 1 00000100:00100000:3.0:1789760795.707291:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febff932a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512782336:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760795.707508:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 29368320, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0xe807 00010000:00010000:3.0:1789760795.707517:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c30538 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.707532:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=775s v=(0 0) 00010000:00010000:3.0:1789760795.707536:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760795.707542:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c30538 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29368320->29372415] (req 29368320->29372415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.707562:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c30538 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29368320->29372415] (req 29368320->29372415) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760795.749145:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974489 is committed 00002000:00010000:3.0:1789760795.749287:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 29368320 -> 29372416 00010000:00010000:3.0:1789760795.749303:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c30538 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29368320->29372415] (req 29368320->29372415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.749315:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c30538 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29368320->29372415] (req 29368320->29372415) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.749325:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c30538 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29368320->29372415] (req 29368320->29372415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.749332:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760795.749338:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c30538 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29368320->29372415] (req 29368320->29372415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.749346:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c30538 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29368320->29372415] (req 29368320->29372415) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.749364:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c30538 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29368320->29372415] (req 29368320->29372415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.749372:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c30538 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29368320->29372415] (req 29368320->29372415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760795.749468:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febff932a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512782336:12345-192.168.203.56@tcp:4:dd.0 Request processed in 42175us (42602us total) trans 4294974489 rc 0/0 00000100:00100000:3.0:1789760795.749479:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7200 00000100:00100000:3.0:1789760795.749488:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 775234982408 time 775271576242 state 0x2 ref 0 00000100:00100000:2.0:1789760795.753055:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760795.753159:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512782464 00000100:00100000:0.0:1789760795.753169:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512782464 00000100:00100000:0.0:1789760795.753223:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 775234982408 time 775271576242 state 0x0 ref 1 00000100:00100000:0.0:1789760795.753238:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 775271836658 time 775271576242 state 0x1 ref 1 00000100:00100000:0.0:1789760795.753268:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 775271836658 time 775317567381 state 0x0 ref 1 00000100:00100000:0.0:1789760795.753280:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2879c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512782464:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760795.753333:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 29372416, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0xe807 00010000:00010000:0.0:1789760795.753342:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c3053f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.753357:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=775s v=(0 0) 00010000:00010000:0.0:1789760795.753361:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760795.753368:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c3053f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29372416->29376511] (req 29372416->29376511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.753382:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c3053f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29372416->29376511] (req 29372416->29376511) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760795.773237:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974490 is committed 00002000:00010000:0.0:1789760795.773307:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 29372416 -> 29376512 00010000:00010000:0.0:1789760795.773321:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c3053f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29372416->29376511] (req 29372416->29376511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.773334:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c3053f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29372416->29376511] (req 29372416->29376511) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.773344:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c3053f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29372416->29376511] (req 29372416->29376511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.773352:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760795.773357:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c3053f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29372416->29376511] (req 29372416->29376511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.773367:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c3053f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29372416->29376511] (req 29372416->29376511) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.773830:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c3053f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29372416->29376511] (req 29372416->29376511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.773841:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004e1e8227/0xe6aa143190c3053f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29372416->29376511] (req 29372416->29376511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760795.773977:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2879c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512782464:12345-192.168.203.56@tcp:4:dd.0 Request processed in 20695us (20926us total) trans 4294974490 rc 0/0 00000100:00100000:0.0:1789760795.773988:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7201 00000100:00100000:0.0:1789760795.773997:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 775271836658 time 775317567381 state 0x2 ref 0 00000100:00100000:0.0:1789760795.777600:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760795.777740:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512782592 00000100:00100000:1.0:1789760795.777750:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512782592 00000100:00100000:1.0:1789760795.777800:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 775271836658 time 775317567381 state 0x0 ref 1 00000100:00100000:1.0:1789760795.777815:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 775317827797 time 775317567381 state 0x1 ref 1 00000100:00100000:1.0:1789760795.777839:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 775317827797 time 775342140053 state 0x0 ref 1 00000100:00100000:1.0:1789760795.777851:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febff930e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512782592:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760795.777895:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 29376512, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0xe807 00010000:00010000:1.0:1789760795.777904:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c30546 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.777919:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=775s v=(0 0) 00010000:00010000:1.0:1789760795.777923:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760795.777941:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c30546 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29376512->29380607] (req 29376512->29380607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.777955:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c30546 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29376512->29380607] (req 29376512->29380607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760795.812349:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974491 is committed 00002000:00010000:1.0:1789760795.812448:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 29376512 -> 29380608 00010000:00010000:1.0:1789760795.812464:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c30546 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29376512->29380607] (req 29376512->29380607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.812478:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c30546 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29376512->29380607] (req 29376512->29380607) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.812488:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c30546 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29376512->29380607] (req 29376512->29380607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.812496:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760795.812501:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c30546 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29376512->29380607] (req 29376512->29380607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.812511:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c30546 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29376512->29380607] (req 29376512->29380607) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.812532:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c30546 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29376512->29380607] (req 29376512->29380607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.812541:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c30546 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29376512->29380607] (req 29376512->29380607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760795.812748:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febff930e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512782592:12345-192.168.203.56@tcp:4:dd.0 Request processed in 34894us (35152us total) trans 4294974491 rc 0/0 00000100:00100000:1.0:1789760795.812757:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7202 00000100:00100000:1.0:1789760795.812766:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 775317827797 time 775342140053 state 0x2 ref 0 00000100:00100000:2.0:1789760795.815267:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760795.815350:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512782720 00000100:00100000:3.0:1789760795.815368:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512782720 00000100:00100000:3.0:1789760795.815421:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 775317827797 time 775342140053 state 0x0 ref 1 00000100:00100000:3.0:1789760795.815436:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 775342400469 time 775342140053 state 0x1 ref 1 00000100:00100000:3.0:1789760795.815462:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 775342400469 time 775379762411 state 0x0 ref 1 00000100:00100000:3.0:1789760795.815474:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac287bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512782720:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760795.815518:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 29380608, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0xe807 00010000:00010000:3.0:1789760795.815527:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c3054d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.815543:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=775s v=(0 0) 00010000:00010000:3.0:1789760795.815547:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760795.815554:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c3054d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29380608->29384703] (req 29380608->29384703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.815568:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c3054d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29380608->29384703] (req 29380608->29384703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760795.840268:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974492 is committed 00002000:00010000:2.0:1789760795.840434:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 29380608 -> 29384704 00010000:00010000:2.0:1789760795.840448:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c3054d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29380608->29384703] (req 29380608->29384703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.840465:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c3054d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29380608->29384703] (req 29380608->29384703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.840475:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c3054d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29380608->29384703] (req 29380608->29384703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.840482:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760795.840488:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c3054d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29380608->29384703] (req 29380608->29384703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.840498:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c3054d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29380608->29384703] (req 29380608->29384703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.840516:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c3054d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29380608->29384703] (req 29380608->29384703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.840524:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c3054d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29380608->29384703] (req 29380608->29384703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760795.841522:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac287bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512782720:12345-192.168.203.56@tcp:4:dd.0 Request processed in 25273us (25487us total) trans 4294974492 rc 0/0 00000100:00100000:2.0:1789760795.841538:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7203 00000100:00100000:2.0:1789760795.841549:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 775342400469 time 775379762411 state 0x2 ref 0 00000100:00100000:2.0:1789760795.843946:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760795.844094:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512782848 00000100:00100000:0.0:1789760795.844104:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512782848 00000100:00100000:0.0:1789760795.844155:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 775342400469 time 775379762411 state 0x0 ref 1 00000100:00100000:0.0:1789760795.844169:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 775380022827 time 775379762411 state 0x1 ref 1 00000100:00100000:0.0:1789760795.844199:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 775380022827 time 775408498819 state 0x0 ref 1 00000100:00100000:0.0:1789760795.844210:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febfc974700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512782848:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760795.844251:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 29384704, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0xe807 00010000:00010000:0.0:1789760795.844259:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c30554 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.844274:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=775s v=(0 0) 00010000:00010000:0.0:1789760795.844278:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760795.844283:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c30554 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29384704->29388799] (req 29384704->29388799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.844296:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c30554 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29384704->29388799] (req 29384704->29388799) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760795.871717:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974493 is committed 00002000:00010000:1.0:1789760795.871775:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 29384704 -> 29388800 00010000:00010000:1.0:1789760795.871789:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c30554 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29384704->29388799] (req 29384704->29388799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.871813:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c30554 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29384704->29388799] (req 29384704->29388799) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.871823:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c30554 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29384704->29388799] (req 29384704->29388799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.871830:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760795.871835:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c30554 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29384704->29388799] (req 29384704->29388799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.871845:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c30554 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29384704->29388799] (req 29384704->29388799) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.871863:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c30554 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29384704->29388799] (req 29384704->29388799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.871870:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f1151685/0xe6aa143190c30554 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29384704->29388799] (req 29384704->29388799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760795.872590:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febfc974700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512782848:12345-192.168.203.56@tcp:4:dd.0 Request processed in 28375us (28657us total) trans 4294974493 rc 0/0 00000100:00100000:1.0:1789760795.872600:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7204 00000100:00100000:1.0:1789760795.872610:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 775380022827 time 775408498819 state 0x2 ref 0 00000100:00100000:0.0:1789760795.882547:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760795.882631:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512782976 00000100:00100000:1.0:1789760795.882641:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512782976 00000100:00100000:1.0:1789760795.882691:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 775380022827 time 775408498819 state 0x0 ref 1 00000100:00100000:1.0:1789760795.882744:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 775408759235 time 775408498819 state 0x1 ref 1 00000100:00100000:1.0:1789760795.882772:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 775408759235 time 775447072098 state 0x0 ref 1 00000100:00100000:1.0:1789760795.882784:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febe7907800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512782976:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760795.882838:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 29388800, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0xe807 00010000:00010000:1.0:1789760795.882847:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c3055b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.882864:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=775s v=(0 0) 00010000:00010000:1.0:1789760795.882868:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760795.882874:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c3055b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29388800->29392895] (req 29388800->29392895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.882888:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c3055b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29388800->29392895] (req 29388800->29392895) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760795.911734:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760795.913570:0:8229:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512783232 00000100:00100000:0.0:1789760795.913583:0:8229:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512783232 00000100:00100000:0.0:1789760795.914015:0:8229:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 524 00000100:00100000:0.0:1789760795.914025:0:8229:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febe7904700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+8:2360:x1876699512783232:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:0.0:1789760795.914096:0:8229:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febe7904700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+8:2360:x1876699512783232:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 73us (2371us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760795.914104:0:8229:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 524 00000100:00080000:3.0:1789760795.914432:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1789760795.915011:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760795.915236:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760795.915261:0:5848:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512783360 00000100:00100000:1.0:1789760795.915267:0:5848:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512783360 00000100:00100000:1.0:1789760795.915305:0:5848:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 525 00000100:00100000:1.0:1789760795.915313:0:5848:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4074380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:d2c40150-e97b-40da-952b-cb9f18bf7e96+5:2360:x1876699512783360:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:1.0:1789760795.915356:0:5848:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4074380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:d2c40150-e97b-40da-952b-cb9f18bf7e96+5:2360:x1876699512783360:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 43us (120us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760795.915362:0:5848:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 525 00000100:00080000:1.0:1789760795.915376:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789760795.916957:0:5842:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512783104 00000100:00100000:3.0:1789760795.916969:0:5842:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512783104 00000100:00100000:3.0:1789760795.917016:0:5842:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 418 00000100:00100000:3.0:1789760795.917024:0:5842:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febe00c4c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:e6cf7267-9b19-462a-a263-14115df15dab+13:2360:x1876699512783104:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:3.0:1789760795.917079:0:5842:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febe00c4c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:e6cf7267-9b19-462a-a263-14115df15dab+13:2360:x1876699512783104:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 56us (2071us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760795.917086:0:5842:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 418 00000001:00080000:2.0:1789760795.929810:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974494 is committed 00002000:00010000:0.0:1789760795.929985:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 29388800 -> 29392896 00010000:00010000:0.0:1789760795.929998:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c3055b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29388800->29392895] (req 29388800->29392895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.930007:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c3055b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29388800->29392895] (req 29388800->29392895) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.930013:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c3055b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29388800->29392895] (req 29388800->29392895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.930017:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760795.930020:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c3055b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29388800->29392895] (req 29388800->29392895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.930027:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c3055b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29388800->29392895] (req 29388800->29392895) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.930040:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c3055b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29388800->29392895] (req 29388800->29392895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.930044:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004c9e19b9/0xe6aa143190c3055b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29388800->29392895] (req 29388800->29392895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760795.930148:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febe7907800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512782976:12345-192.168.203.56@tcp:4:dd.0 Request processed in 47357us (47602us total) trans 4294974494 rc 0/0 00000100:00100000:0.0:1789760795.930158:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7205 00000100:00100000:0.0:1789760795.930169:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 775408759235 time 775447072098 state 0x2 ref 0 00000100:00100000:3.0:1789760795.934078:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760795.934208:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512783488 00000100:00100000:2.0:1789760795.934217:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512783488 00000100:00100000:2.0:1789760795.934264:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 775408759235 time 775447072098 state 0x0 ref 1 00000100:00100000:2.0:1789760795.934278:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 775447332514 time 775447072098 state 0x1 ref 1 00000100:00100000:2.0:1789760795.934307:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 775447332514 time 775498606882 state 0x0 ref 1 00000100:00100000:2.0:1789760795.934319:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5305880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512783488:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760795.934362:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 29392896, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0xe807 00010000:00010000:2.0:1789760795.934371:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c30562 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.934386:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=775s v=(0 0) 00010000:00010000:2.0:1789760795.934390:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760795.934396:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c30562 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29392896->29396991] (req 29392896->29396991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.934410:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c30562 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29392896->29396991] (req 29392896->29396991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760795.985484:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974495 is committed 00002000:00010000:0.0:1789760795.985925:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 29392896 -> 29396992 00010000:00010000:0.0:1789760795.985954:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c30562 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29392896->29396991] (req 29392896->29396991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.985968:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c30562 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29392896->29396991] (req 29392896->29396991) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.985977:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c30562 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29392896->29396991] (req 29392896->29396991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.985985:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760795.985994:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c30562 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29392896->29396991] (req 29392896->29396991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.986002:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c30562 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29392896->29396991] (req 29392896->29396991) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.986021:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c30562 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29392896->29396991] (req 29392896->29396991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.986033:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d43cd646/0xe6aa143190c30562 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29392896->29396991] (req 29392896->29396991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760795.986126:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5305880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512783488:12345-192.168.203.56@tcp:4:dd.0 Request processed in 51804us (52052us total) trans 4294974495 rc 0/0 00000100:00100000:0.0:1789760795.986136:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7206 00000100:00100000:0.0:1789760795.986145:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 775447332514 time 775498606882 state 0x2 ref 0 00000100:00100000:1.0:1789760795.988859:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760795.988941:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512783616 00000100:00100000:0.0:1789760795.988951:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512783616 00000100:00100000:0.0:1789760795.989005:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 775447332514 time 775498606882 state 0x0 ref 1 00000100:00100000:0.0:1789760795.989020:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 775498867298 time 775498606882 state 0x1 ref 1 00000100:00100000:0.0:1789760795.989051:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 775498867298 time 775553351021 state 0x0 ref 1 00000100:00100000:0.0:1789760795.989063:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4076a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512783616:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760795.989114:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 29396992, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0xe807 00010000:00010000:0.0:1789760795.989123:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c30569 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.989139:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=775s v=(0 0) 00010000:00010000:0.0:1789760795.989144:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760795.989150:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c30569 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29396992->29401087] (req 29396992->29401087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.989172:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c30569 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29396992->29401087] (req 29396992->29401087) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760796.013553:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974496 is committed 00002000:00010000:1.0:1789760796.013748:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 29396992 -> 29401088 00010000:00010000:1.0:1789760796.013763:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c30569 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29396992->29401087] (req 29396992->29401087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.013776:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c30569 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29396992->29401087] (req 29396992->29401087) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.013792:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c30569 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29396992->29401087] (req 29396992->29401087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.013808:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760796.013813:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c30569 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29396992->29401087] (req 29396992->29401087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.013823:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c30569 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29396992->29401087] (req 29396992->29401087) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.013842:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c30569 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29396992->29401087] (req 29396992->29401087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.013850:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006df6e978/0xe6aa143190c30569 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29396992->29401087] (req 29396992->29401087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760796.013951:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4076a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512783616:12345-192.168.203.56@tcp:4:dd.0 Request processed in 24884us (25096us total) trans 4294974496 rc 0/0 00000100:00100000:1.0:1789760796.013960:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7207 00000100:00100000:1.0:1789760796.013970:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 775498867298 time 775553351021 state 0x2 ref 0 00000100:00100000:1.0:1789760796.017073:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760796.017162:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512783744 00000100:00100000:0.0:1789760796.017173:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512783744 00000100:00100000:0.0:1789760796.017227:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 775498867298 time 775553351021 state 0x0 ref 1 00000100:00100000:0.0:1789760796.017242:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 775553611437 time 775553351021 state 0x1 ref 1 00000100:00100000:0.0:1789760796.017271:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 775553611437 time 775581571236 state 0x0 ref 1 00000100:00100000:0.0:1789760796.017283:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febea48b450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512783744:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760796.017336:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 29401088, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0xe807 00010000:00010000:0.0:1789760796.017345:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c30570 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.017361:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=775s v=(0 0) 00010000:00010000:0.0:1789760796.017366:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760796.017373:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c30570 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29401088->29405183] (req 29401088->29405183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.017387:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c30570 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29401088->29405183] (req 29401088->29405183) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.021828:0:7647:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:2.0:1789760796.044210:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974497 is committed 00002000:00010000:0.0:1789760796.044353:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 29401088 -> 29405184 00010000:00010000:0.0:1789760796.044372:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c30570 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29401088->29405183] (req 29401088->29405183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.044383:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c30570 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29401088->29405183] (req 29401088->29405183) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.044392:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c30570 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29401088->29405183] (req 29401088->29405183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.044410:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760796.044415:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c30570 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29401088->29405183] (req 29401088->29405183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.044425:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c30570 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29401088->29405183] (req 29401088->29405183) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.044442:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c30570 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29401088->29405183] (req 29401088->29405183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.044448:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c30570 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29401088->29405183] (req 29401088->29405183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760796.044539:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febea48b450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512783744:12345-192.168.203.56@tcp:4:dd.0 Request processed in 27251us (27468us total) trans 4294974497 rc 0/0 00000100:00100000:0.0:1789760796.044548:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7208 00000100:00100000:0.0:1789760796.044559:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 775553611437 time 775581571236 state 0x2 ref 0 00000100:00100000:3.0:1789760796.049995:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760796.050098:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512783872 00000100:00100000:0.0:1789760796.050108:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512783872 00000100:00100000:0.0:1789760796.050161:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 775553611437 time 775581571236 state 0x0 ref 1 00000100:00100000:0.0:1789760796.050175:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 775581831652 time 775581571236 state 0x1 ref 1 00000100:00100000:0.0:1789760796.050258:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 775581831652 time 775614557262 state 0x0 ref 1 00000100:00100000:0.0:1789760796.050272:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5305f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512783872:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760796.050318:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 29405184, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0xe807 00010000:00010000:0.0:1789760796.050327:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c30577 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.050343:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=775s v=(0 0) 00010000:00010000:0.0:1789760796.050347:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760796.050354:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c30577 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29405184->29409279] (req 29405184->29409279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.050367:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c30577 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29405184->29409279] (req 29405184->29409279) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760796.084974:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974498 is committed 00002000:00010000:1.0:1789760796.086606:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 29405184 -> 29409280 00010000:00010000:1.0:1789760796.086627:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c30577 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29405184->29409279] (req 29405184->29409279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.086645:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c30577 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29405184->29409279] (req 29405184->29409279) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.086654:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c30577 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29405184->29409279] (req 29405184->29409279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.086660:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760796.086664:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c30577 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29405184->29409279] (req 29405184->29409279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.086673:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c30577 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29405184->29409279] (req 29405184->29409279) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.086692:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c30577 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29405184->29409279] (req 29405184->29409279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.086716:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008401802b/0xe6aa143190c30577 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29405184->29409279] (req 29405184->29409279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760796.086901:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5305f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512783872:12345-192.168.203.56@tcp:4:dd.0 Request processed in 36625us (36908us total) trans 4294974498 rc 0/0 00000100:00100000:1.0:1789760796.086913:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7209 00000100:00100000:1.0:1789760796.086923:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 775581831652 time 775614557262 state 0x2 ref 0 00000100:00100000:2.0:1789760796.095095:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760796.095167:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512784000 00000100:00100000:0.0:1789760796.095176:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512784000 00000100:00100000:0.0:1789760796.095222:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 775581831652 time 775614557262 state 0x0 ref 1 00000100:00100000:0.0:1789760796.095236:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 775614817678 time 775614557262 state 0x1 ref 1 00000100:00100000:0.0:1789760796.095270:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 775614817678 time 775659570168 state 0x0 ref 1 00000100:00100000:0.0:1789760796.095281:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febfc975f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512784000:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760796.095335:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 29409280, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0xe807 00010000:00010000:0.0:1789760796.095344:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c3057e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.095358:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=775s v=(0 0) 00010000:00010000:0.0:1789760796.095362:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760796.095369:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c3057e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29409280->29413375] (req 29409280->29413375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.095382:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c3057e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29409280->29413375] (req 29409280->29413375) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760796.151865:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 29409280 -> 29413376 00010000:00010000:0.0:1789760796.151886:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c3057e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29409280->29413375] (req 29409280->29413375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.151899:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c3057e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29409280->29413375] (req 29409280->29413375) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760796.151906:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974499 is committed 00010000:00010000:0.0:1789760796.151907:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c3057e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29409280->29413375] (req 29409280->29413375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.151914:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760796.151919:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c3057e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29409280->29413375] (req 29409280->29413375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.151938:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c3057e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29409280->29413375] (req 29409280->29413375) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.151959:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c3057e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29409280->29413375] (req 29409280->29413375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.151966:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c3057e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29409280->29413375] (req 29409280->29413375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760796.152057:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febfc975f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512784000:12345-192.168.203.56@tcp:4:dd.0 Request processed in 56771us (56964us total) trans 4294974499 rc 0/0 00000100:00100000:0.0:1789760796.152067:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7210 00000100:00100000:0.0:1789760796.152077:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 775614817678 time 775659570168 state 0x2 ref 0 00000100:00100000:2.0:1789760796.154809:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760796.154915:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512784128 00000100:00100000:0.0:1789760796.154937:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512784128 00000100:00100000:0.0:1789760796.154992:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 775614817678 time 775659570168 state 0x0 ref 1 00000100:00100000:0.0:1789760796.155007:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 775659830584 time 775659570168 state 0x1 ref 1 00000100:00100000:0.0:1789760796.155035:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 775659830584 time 775719335424 state 0x0 ref 1 00000100:00100000:0.0:1789760796.155047:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febfc975180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512784128:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760796.155107:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 29413376, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0xe807 00010000:00010000:0.0:1789760796.155117:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c30585 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.155132:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=775s v=(0 0) 00010000:00010000:0.0:1789760796.155136:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760796.155143:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c30585 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29413376->29417471] (req 29413376->29417471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.155156:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c30585 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29413376->29417471] (req 29413376->29417471) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760796.187314:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974500 is committed 00002000:00010000:1.0:1789760796.187439:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 29413376 -> 29417472 00010000:00010000:1.0:1789760796.187454:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c30585 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29413376->29417471] (req 29413376->29417471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.187466:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c30585 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29413376->29417471] (req 29413376->29417471) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.187474:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c30585 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29413376->29417471] (req 29413376->29417471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.187481:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760796.187486:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c30585 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29413376->29417471] (req 29413376->29417471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.187495:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c30585 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29413376->29417471] (req 29413376->29417471) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.187517:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c30585 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29413376->29417471] (req 29413376->29417471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.187524:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c30585 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29413376->29417471] (req 29413376->29417471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760796.187629:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febfc975180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512784128:12345-192.168.203.56@tcp:4:dd.0 Request processed in 32578us (32823us total) trans 4294974500 rc 0/0 00000100:00100000:1.0:1789760796.187641:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7211 00000100:00100000:1.0:1789760796.187651:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 775659830584 time 775719335424 state 0x2 ref 0 00000100:00100000:3.0:1789760796.196883:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760796.196994:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512784256 00000100:00100000:1.0:1789760796.197015:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512784256 00000100:00100000:1.0:1789760796.197070:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 775659830584 time 775719335424 state 0x0 ref 1 00000100:00100000:1.0:1789760796.197086:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 775719595840 time 775719335424 state 0x1 ref 1 00000100:00100000:1.0:1789760796.197183:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 775719595840 time 775761481298 state 0x0 ref 1 00000100:00100000:1.0:1789760796.197197:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5307480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512784256:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760796.197246:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 29417472, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0xe807 00010000:00010000:1.0:1789760796.197256:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c3058c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.197284:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=775s v=(0 0) 00010000:00010000:1.0:1789760796.197290:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760796.197297:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c3058c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29417472->29421567] (req 29417472->29421567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.197311:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c3058c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29417472->29421567] (req 29417472->29421567) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760796.232237:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974501 is committed 00002000:00010000:1.0:1789760796.232396:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 29417472 -> 29421568 00010000:00010000:1.0:1789760796.232411:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c3058c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29417472->29421567] (req 29417472->29421567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.232425:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c3058c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29417472->29421567] (req 29417472->29421567) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.232436:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c3058c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29417472->29421567] (req 29417472->29421567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.232444:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760796.232450:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c3058c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29417472->29421567] (req 29417472->29421567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.232459:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c3058c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29417472->29421567] (req 29417472->29421567) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.232478:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c3058c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29417472->29421567] (req 29417472->29421567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.232485:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c3058c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29417472->29421567] (req 29417472->29421567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760796.232612:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5307480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512784256:12345-192.168.203.56@tcp:4:dd.0 Request processed in 35412us (35733us total) trans 4294974501 rc 0/0 00000100:00100000:1.0:1789760796.232622:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7212 00000100:00100000:1.0:1789760796.232632:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 775719595840 time 775761481298 state 0x2 ref 0 00000100:00100000:2.0:1789760796.237937:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760796.238040:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512784384 00000100:00100000:0.0:1789760796.238050:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512784384 00000100:00100000:0.0:1789760796.238106:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 775719595840 time 775761481298 state 0x0 ref 1 00000100:00100000:0.0:1789760796.238121:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 775761741714 time 775761481298 state 0x1 ref 1 00000100:00100000:0.0:1789760796.238187:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 775761741714 time 775802486245 state 0x0 ref 1 00000100:00100000:0.0:1789760796.238201:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febfc977100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512784384:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760796.238250:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 29421568, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0xe807 00010000:00010000:0.0:1789760796.238259:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c30593 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.238275:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=775s v=(0 0) 00010000:00010000:0.0:1789760796.238280:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760796.238286:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c30593 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29421568->29425663] (req 29421568->29425663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.238299:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c30593 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29421568->29425663] (req 29421568->29425663) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760796.273261:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 29421568 -> 29425664 00010000:00010000:1.0:1789760796.273280:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c30593 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29421568->29425663] (req 29421568->29425663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.273299:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c30593 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29421568->29425663] (req 29421568->29425663) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.273307:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c30593 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29421568->29425663] (req 29421568->29425663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.273314:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9800 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760796.273332:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c30593 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29421568->29425663] (req 29421568->29425663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.273341:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c30593 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29421568->29425663] (req 29421568->29425663) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.273362:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c30593 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29421568->29425663] (req 29421568->29425663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.273370:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000abc861c/0xe6aa143190c30593 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29421568->29425663] (req 29421568->29425663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760796.273460:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febfc977100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2360:x1876699512784384:12345-192.168.203.56@tcp:4:dd.0 Request processed in 35257us (35538us total) trans 4294974502 rc 0/0 00000100:00100000:1.0:1789760796.273471:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7213 00000100:00100000:1.0:1789760796.273481:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 775761741714 time 775802486245 state 0x2 ref 0 00000100:00100000:3.0:1789760796.276581:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000001:00080000:2.0:1789760796.277267:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974502 is committed 00000100:00100000:1.0:1789760796.277531:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512784512 00000100:00100000:1.0:1789760796.277543:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512784512 00000100:00100000:1.0:1789760796.278162:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 775761741714 time 775802486245 state 0x0 ref 1 00000100:00100000:1.0:1789760796.278182:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 775802746661 time 775802486245 state 0x1 ref 1 00000100:00100000:1.0:1789760796.278256:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 775802746661 time 775842554735 state 0x0 ref 1 00000100:00100000:1.0:1789760796.278272:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5304700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512784512:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760796.278330:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 29425664, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0xe807 00010000:00010000:1.0:1789760796.278340:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c3059a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.278355:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=775s v=(0 0) 00010000:00010000:1.0:1789760796.278360:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760796.278366:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c3059a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29425664->29429759] (req 29425664->29429759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.278382:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c3059a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29425664->29429759] (req 29425664->29429759) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760796.316792:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 29425664 -> 29429760 00010000:00010000:3.0:1789760796.316812:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c3059a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29425664->29429759] (req 29425664->29429759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.316826:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c3059a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29425664->29429759] (req 29425664->29429759) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.316836:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c3059a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29425664->29429759] (req 29425664->29429759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.316843:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760796.316847:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c3059a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29425664->29429759] (req 29425664->29429759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.316856:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c3059a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29425664->29429759] (req 29425664->29429759) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.316875:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c3059a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29425664->29429759] (req 29425664->29429759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.316884:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c3059a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29425664->29429759] (req 29425664->29429759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760796.316907:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974503 is committed 00000100:00100000:3.0:1789760796.316990:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5304700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512784512:12345-192.168.203.56@tcp:4:dd.0 Request processed in 38718us (40412us total) trans 4294974503 rc 0/0 00000100:00100000:3.0:1789760796.317000:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7214 00000100:00100000:3.0:1789760796.317010:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 775802746661 time 775842554735 state 0x2 ref 0 00000100:00100000:3.0:1789760796.321715:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760796.321890:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512784640 00000100:00100000:1.0:1789760796.321899:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512784640 00000100:00100000:1.0:1789760796.321972:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 775802746661 time 775842554735 state 0x0 ref 1 00000100:00100000:1.0:1789760796.321989:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 775842815151 time 775842554735 state 0x1 ref 1 00000100:00100000:1.0:1789760796.322020:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 775842815151 time 775886320508 state 0x0 ref 1 00000100:00100000:1.0:1789760796.322034:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5304380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512784640:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760796.322081:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 29429760, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0xe807 00010000:00010000:1.0:1789760796.322089:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c305a1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.322104:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=775s v=(0 0) 00010000:00010000:1.0:1789760796.322108:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760796.322114:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c305a1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29429760->29433855] (req 29429760->29433855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.322128:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c305a1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29429760->29433855] (req 29429760->29433855) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760796.358195:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 29429760 -> 29433856 00010000:00010000:1.0:1789760796.358216:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c305a1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29429760->29433855] (req 29429760->29433855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.358229:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c305a1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29429760->29433855] (req 29429760->29433855) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.358238:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c305a1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29429760->29433855] (req 29429760->29433855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.358246:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760796.358251:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c305a1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29429760->29433855] (req 29429760->29433855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.358262:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c305a1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29429760->29433855] (req 29429760->29433855) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.358283:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c305a1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29429760->29433855] (req 29429760->29433855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.358290:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c305a1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29429760->29433855] (req 29429760->29433855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760796.358395:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5304380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2361:x1876699512784640:12345-192.168.203.56@tcp:4:dd.0 Request processed in 36359us (36704us total) trans 4294974504 rc 0/0 00000100:00100000:1.0:1789760796.358406:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7215 00000100:00100000:1.0:1789760796.358415:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 775842815151 time 775886320508 state 0x2 ref 0 00000001:00080000:1.0:1789760796.358452:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974504 is committed 00000100:00100000:1.0:1789760796.363462:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760796.363558:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512784768 00000100:00100000:0.0:1789760796.363569:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512784768 00000100:00100000:0.0:1789760796.363625:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 775842815151 time 775886320508 state 0x0 ref 1 00000100:00100000:0.0:1789760796.363640:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 775886580924 time 775886320508 state 0x1 ref 1 00000100:00100000:0.0:1789760796.363673:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 775886580924 time 775927972938 state 0x0 ref 1 00000100:00100000:0.0:1789760796.363685:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4074e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512784768:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760796.363765:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 29433856, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0xe807 00010000:00010000:0.0:1789760796.363774:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c305a8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.363789:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=775s v=(0 0) 00010000:00010000:0.0:1789760796.363794:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760796.363800:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c305a8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29433856->29437951] (req 29433856->29437951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.363813:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c305a8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29433856->29437951] (req 29433856->29437951) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760796.425399:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 29433856 -> 29437952 00010000:00010000:1.0:1789760796.425417:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c305a8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29433856->29437951] (req 29433856->29437951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.425429:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c305a8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29433856->29437951] (req 29433856->29437951) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.425438:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c305a8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29433856->29437951] (req 29433856->29437951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.425449:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760796.425454:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c305a8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29433856->29437951] (req 29433856->29437951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.425463:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c305a8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29433856->29437951] (req 29433856->29437951) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.425482:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c305a8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29433856->29437951] (req 29433856->29437951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.425489:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c305a8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29433856->29437951] (req 29433856->29437951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760796.425586:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4074e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2359:x1876699512784768:12345-192.168.203.56@tcp:4:dd.0 Request processed in 61897us (62129us total) trans 4294974505 rc 0/0 00000100:00100000:1.0:1789760796.425977:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7216 00000100:00100000:1.0:1789760796.425987:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 775886580924 time 775927972938 state 0x2 ref 0 00000001:00080000:3.0:1789760796.426350:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974505 is committed 00000100:00100000:3.0:1789760796.432461:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760796.432565:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512784896 00000100:00100000:1.0:1789760796.432575:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512784896 00000100:00100000:1.0:1789760796.432752:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 775886580924 time 775927972938 state 0x0 ref 1 00000100:00100000:1.0:1789760796.432768:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 775928233354 time 775927972938 state 0x1 ref 1 00000100:00100000:1.0:1789760796.432800:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 775928233354 time 775997099459 state 0x0 ref 1 00000100:00100000:1.0:1789760796.432817:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5306680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512784896:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760796.432867:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 29437952, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0xe807 00010000:00010000:1.0:1789760796.432876:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c305af lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.432892:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=775s v=(0 0) 00010000:00010000:1.0:1789760796.432897:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760796.432903:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c305af lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29437952->29442047] (req 29437952->29442047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.432918:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c305af lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29437952->29442047] (req 29437952->29442047) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760796.466124:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974506 is committed 00002000:00010000:1.0:1789760796.466488:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 29437952 -> 29442048 00010000:00010000:1.0:1789760796.466504:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c305af lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29437952->29442047] (req 29437952->29442047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.466518:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c305af lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29437952->29442047] (req 29437952->29442047) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.466527:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c305af lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29437952->29442047] (req 29437952->29442047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.466534:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760796.466539:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c305af lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29437952->29442047] (req 29437952->29442047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.466557:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c305af lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29437952->29442047] (req 29437952->29442047) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.466579:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c305af lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29437952->29442047] (req 29437952->29442047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.466587:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c305af lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29437952->29442047] (req 29437952->29442047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760796.466688:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5306680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512784896:12345-192.168.203.56@tcp:4:dd.0 Request processed in 33868us (34231us total) trans 4294974506 rc 0/0 00000100:00100000:1.0:1789760796.466831:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7217 00000100:00100000:1.0:1789760796.466840:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 775928233354 time 775997099459 state 0x2 ref 0 00000100:00100000:2.0:1789760796.473522:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760796.473592:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512785024 00000100:00100000:1.0:1789760796.473603:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512785024 00000100:00100000:1.0:1789760796.473658:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 775928233354 time 775997099459 state 0x0 ref 1 00000100:00100000:1.0:1789760796.473673:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 775997359875 time 775997099459 state 0x1 ref 1 00000100:00100000:1.0:1789760796.473736:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 775997359875 time 776038035419 state 0x0 ref 1 00000100:00100000:1.0:1789760796.473748:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febfc974700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512785024:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760796.473795:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 29442048, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0xe807 00010000:00010000:1.0:1789760796.473803:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c305b6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.473820:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=776s v=(0 0) 00010000:00010000:1.0:1789760796.473825:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760796.473831:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c305b6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29442048->29446143] (req 29442048->29446143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.473845:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c305b6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29442048->29446143] (req 29442048->29446143) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760796.517363:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 29442048 -> 29446144 00010000:00010000:0.0:1789760796.517383:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c305b6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29442048->29446143] (req 29442048->29446143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.517396:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c305b6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29442048->29446143] (req 29442048->29446143) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.517405:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c305b6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29442048->29446143] (req 29442048->29446143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.517413:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760796.517419:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c305b6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29442048->29446143] (req 29442048->29446143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.517438:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c305b6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29442048->29446143] (req 29442048->29446143) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.517457:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c305b6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29442048->29446143] (req 29442048->29446143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.517465:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c305b6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29442048->29446143] (req 29442048->29446143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760796.517560:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febfc974700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2359:x1876699512785024:12345-192.168.203.56@tcp:4:dd.0 Request processed in 43809us (44042us total) trans 4294974507 rc 0/0 00000100:00100000:0.0:1789760796.517584:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7218 00000100:00100000:0.0:1789760796.517602:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 775997359875 time 776038035419 state 0x2 ref 0 00000001:00080000:3.0:1789760796.518252:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974507 is committed 00000100:00100000:2.0:1789760796.524319:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760796.524455:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512785152 00000100:00100000:1.0:1789760796.524465:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512785152 00000100:00100000:1.0:1789760796.524515:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 775997359875 time 776038035419 state 0x0 ref 1 00000100:00100000:1.0:1789760796.524530:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 776038295835 time 776038035419 state 0x1 ref 1 00000100:00100000:1.0:1789760796.524597:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 776038295835 time 776088896078 state 0x0 ref 1 00000100:00100000:1.0:1789760796.524617:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac41d5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512785152:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760796.524674:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 29446144, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0xe807 00010000:00010000:1.0:1789760796.524683:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c305bd lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.525739:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=776s v=(0 0) 00010000:00010000:1.0:1789760796.525747:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760796.525755:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c305bd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29446144->29450239] (req 29446144->29450239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.525777:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c305bd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29446144->29450239] (req 29446144->29450239) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760796.549871:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974508 is committed 00002000:00010000:1.0:1789760796.550135:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 29446144 -> 29450240 00002000:00010000:1.0:1789760796.550140:0:13062:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760795 -> 1789760796 00002000:00010000:1.0:1789760796.550142:0:13062:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760795 -> 1789760796 00010000:00010000:1.0:1789760796.550155:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c305bd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29446144->29450239] (req 29446144->29450239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.550169:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c305bd lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29446144->29450239] (req 29446144->29450239) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.550178:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c305bd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29446144->29450239] (req 29446144->29450239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.550185:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760796.550190:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c305bd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29446144->29450239] (req 29446144->29450239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.550198:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c305bd lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29446144->29450239] (req 29446144->29450239) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.550217:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c305bd lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29446144->29450239] (req 29446144->29450239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.550230:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c305bd lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29446144->29450239] (req 29446144->29450239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760796.550359:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac41d5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512785152:12345-192.168.203.56@tcp:4:dd.0 Request processed in 25738us (26044us total) trans 4294974508 rc 0/0 00000100:00100000:1.0:1789760796.550371:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7219 00000100:00100000:1.0:1789760796.550381:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 776038295835 time 776088896078 state 0x2 ref 0 00000100:00100000:2.0:1789760796.553772:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760796.553857:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512785280 00000100:00100000:1.0:1789760796.553866:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512785280 00000100:00100000:1.0:1789760796.553921:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 776038295835 time 776088896078 state 0x0 ref 1 00000100:00100000:1.0:1789760796.553946:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 776089156494 time 776088896078 state 0x1 ref 1 00000100:00100000:1.0:1789760796.553971:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 776089156494 time 776118270954 state 0x0 ref 1 00000100:00100000:1.0:1789760796.553983:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac41d7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512785280:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760796.554029:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 29450240, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0xe807 00010000:00010000:1.0:1789760796.554038:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c305c4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.554055:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=776s v=(0 0) 00010000:00010000:1.0:1789760796.554060:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760796.554067:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c305c4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29450240->29454335] (req 29450240->29454335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.554088:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c305c4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29450240->29454335] (req 29450240->29454335) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760796.620407:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974509 is committed 00002000:00010000:0.0:1789760796.620969:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 29450240 -> 29454336 00010000:00010000:0.0:1789760796.620986:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c305c4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29450240->29454335] (req 29450240->29454335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.620998:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c305c4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29450240->29454335] (req 29450240->29454335) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.621007:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c305c4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29450240->29454335] (req 29450240->29454335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.621013:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760796.621018:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c305c4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29450240->29454335] (req 29450240->29454335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.621026:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c305c4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29450240->29454335] (req 29450240->29454335) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.621111:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c305c4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29450240->29454335] (req 29450240->29454335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.621118:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c305c4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29450240->29454335] (req 29450240->29454335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760796.621220:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac41d7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512785280:12345-192.168.203.56@tcp:4:dd.0 Request processed in 67233us (67452us total) trans 4294974509 rc 0/0 00000100:00100000:0.0:1789760796.621228:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7220 00000100:00100000:0.0:1789760796.621238:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 776089156494 time 776118270954 state 0x2 ref 0 00000100:00100000:2.0:1789760796.625402:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760796.625489:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512785408 00000100:00100000:0.0:1789760796.625499:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512785408 00000100:00100000:0.0:1789760796.625552:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 776089156494 time 776118270954 state 0x0 ref 1 00000100:00100000:0.0:1789760796.625568:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 776118531370 time 776118270954 state 0x1 ref 1 00000100:00100000:0.0:1789760796.625634:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 776118531370 time 776189933050 state 0x0 ref 1 00000100:00100000:0.0:1789760796.625646:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac41d6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512785408:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760796.625691:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 29454336, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0xe807 00010000:00010000:0.0:1789760796.625739:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c305cb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.625754:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=776s v=(0 0) 00010000:00010000:0.0:1789760796.625758:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760796.625764:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c305cb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29454336->29458431] (req 29454336->29458431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.625777:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c305cb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29454336->29458431] (req 29454336->29458431) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760796.659984:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 29454336 -> 29458432 00010000:00010000:1.0:1789760796.660005:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c305cb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29454336->29458431] (req 29454336->29458431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.660029:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c305cb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29454336->29458431] (req 29454336->29458431) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.660039:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c305cb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29454336->29458431] (req 29454336->29458431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.660047:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760796.660052:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c305cb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29454336->29458431] (req 29454336->29458431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.660062:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c305cb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29454336->29458431] (req 29454336->29458431) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.660081:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c305cb lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29454336->29458431] (req 29454336->29458431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.660090:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cb1ad1a9/0xe6aa143190c305cb lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29454336->29458431] (req 29454336->29458431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760796.660193:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac41d6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2360:x1876699512785408:12345-192.168.203.56@tcp:4:dd.0 Request processed in 34543us (34796us total) trans 4294974510 rc 0/0 00000100:00100000:1.0:1789760796.660204:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7221 00000100:00100000:1.0:1789760796.660214:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 776118531370 time 776189933050 state 0x2 ref 0 00000001:00080000:2.0:1789760796.660975:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974510 is committed 00000100:00100000:2.0:1789760796.664648:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760796.665089:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512785536 00000100:00100000:1.0:1789760796.665099:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512785536 00000100:00100000:1.0:1789760796.665150:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 776118531370 time 776189933050 state 0x0 ref 1 00000100:00100000:1.0:1789760796.665165:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 776190193466 time 776189933050 state 0x1 ref 1 00000100:00100000:1.0:1789760796.665270:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 776190193466 time 776229568566 state 0x0 ref 1 00000100:00100000:1.0:1789760796.665293:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac41d5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512785536:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760796.665339:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 29458432, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0xe807 00010000:00010000:1.0:1789760796.665347:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c305d2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.665373:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=776s v=(0 0) 00010000:00010000:1.0:1789760796.665378:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760796.665384:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c305d2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29458432->29462527] (req 29458432->29462527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.665398:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c305d2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29458432->29462527] (req 29458432->29462527) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760796.702116:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974511 is committed 00002000:00010000:1.0:1789760796.702187:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 29458432 -> 29462528 00010000:00010000:1.0:1789760796.702201:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c305d2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29458432->29462527] (req 29458432->29462527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.702213:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c305d2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29458432->29462527] (req 29458432->29462527) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.702223:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c305d2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29458432->29462527] (req 29458432->29462527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.702231:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760796.702236:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c305d2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29458432->29462527] (req 29458432->29462527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.702246:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c305d2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29458432->29462527] (req 29458432->29462527) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.702265:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c305d2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29458432->29462527] (req 29458432->29462527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.702273:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c305d2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29458432->29462527] (req 29458432->29462527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760796.702351:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac41d5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512785536:12345-192.168.203.56@tcp:4:dd.0 Request processed in 37056us (37706us total) trans 4294974511 rc 0/0 00000100:00100000:1.0:1789760796.702362:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7222 00000100:00100000:1.0:1789760796.702371:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 776190193466 time 776229568566 state 0x2 ref 0 00000100:00100000:0.0:1789760796.708681:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760796.708805:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512785664 00000100:00100000:1.0:1789760796.708815:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512785664 00000100:00100000:1.0:1789760796.708870:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 776190193466 time 776229568566 state 0x0 ref 1 00000100:00100000:1.0:1789760796.708885:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 776229828982 time 776229568566 state 0x1 ref 1 00000100:00100000:1.0:1789760796.708918:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 776229828982 time 776273217048 state 0x0 ref 1 00000100:00100000:1.0:1789760796.708939:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febe7907800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512785664:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760796.708984:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 29462528, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0xe807 00010000:00010000:1.0:1789760796.708994:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c305d9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.709009:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=776s v=(0 0) 00010000:00010000:1.0:1789760796.709013:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760796.709020:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c305d9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29462528->29466623] (req 29462528->29466623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.709034:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c305d9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29462528->29466623] (req 29462528->29466623) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760796.760324:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974512 is committed 00002000:00010000:0.0:1789760796.760630:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 29462528 -> 29466624 00010000:00010000:0.0:1789760796.760647:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c305d9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29462528->29466623] (req 29462528->29466623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.760660:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c305d9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29462528->29466623] (req 29462528->29466623) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.760670:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c305d9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29462528->29466623] (req 29462528->29466623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.760677:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760796.760682:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c305d9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29462528->29466623] (req 29462528->29466623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.760691:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c305d9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29462528->29466623] (req 29462528->29466623) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.760738:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c305d9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29462528->29466623] (req 29462528->29466623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.760746:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000064e0e5ac/0xe6aa143190c305d9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29462528->29466623] (req 29462528->29466623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760796.760834:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febe7907800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512785664:12345-192.168.203.56@tcp:4:dd.0 Request processed in 51893us (52158us total) trans 4294974512 rc 0/0 00000100:00100000:0.0:1789760796.760845:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7223 00000100:00100000:0.0:1789760796.760855:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 776229828982 time 776273217048 state 0x2 ref 0 00000100:00100000:1.0:1789760796.772274:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760796.772366:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512785792 00000100:00100000:0.0:1789760796.772377:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512785792 00000100:00100000:0.0:1789760796.772438:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 776229828982 time 776273217048 state 0x0 ref 1 00000100:00100000:0.0:1789760796.772457:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 776273477464 time 776273217048 state 0x1 ref 1 00000100:00100000:0.0:1789760796.772492:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 776273477464 time 776336791735 state 0x0 ref 1 00000100:00100000:0.0:1789760796.772504:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4075500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512785792:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760796.772557:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 29466624, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0xe807 00010000:00010000:0.0:1789760796.772566:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c305e0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.772580:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=776s v=(0 0) 00010000:00010000:0.0:1789760796.772585:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760796.772592:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c305e0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29466624->29470719] (req 29466624->29470719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.772605:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c305e0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29466624->29470719] (req 29466624->29470719) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760796.801839:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974513 is committed 00002000:00010000:1.0:1789760796.802020:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 29466624 -> 29470720 00010000:00010000:1.0:1789760796.802036:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c305e0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29466624->29470719] (req 29466624->29470719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.802050:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c305e0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29466624->29470719] (req 29466624->29470719) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.802060:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c305e0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29466624->29470719] (req 29466624->29470719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.802068:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760796.802073:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c305e0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29466624->29470719] (req 29466624->29470719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.802083:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c305e0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29466624->29470719] (req 29466624->29470719) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.802102:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c305e0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29466624->29470719] (req 29466624->29470719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.802110:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000133c1022/0xe6aa143190c305e0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29466624->29470719] (req 29466624->29470719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760796.802199:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4075500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512785792:12345-192.168.203.56@tcp:4:dd.0 Request processed in 29691us (29931us total) trans 4294974513 rc 0/0 00000100:00100000:1.0:1789760796.802209:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7224 00000100:00100000:1.0:1789760796.802218:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 776273477464 time 776336791735 state 0x2 ref 0 00000100:00100000:1.0:1789760796.806800:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760796.806948:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512785920 00000100:00100000:2.0:1789760796.806958:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512785920 00000100:00100000:2.0:1789760796.807009:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 776273477464 time 776336791735 state 0x0 ref 1 00000100:00100000:2.0:1789760796.807025:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 776337052151 time 776336791735 state 0x1 ref 1 00000100:00100000:2.0:1789760796.807047:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 776337052151 time 776371347682 state 0x0 ref 1 00000100:00100000:2.0:1789760796.807060:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4076300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512785920:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760796.807111:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 29470720, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0xe807 00010000:00010000:2.0:1789760796.807120:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c305e7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.807136:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=776s v=(0 0) 00010000:00010000:2.0:1789760796.807141:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760796.807147:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c305e7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29470720->29474815] (req 29470720->29474815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.807160:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c305e7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29470720->29474815] (req 29470720->29474815) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760796.870976:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 29470720 -> 29474816 00010000:00010000:3.0:1789760796.870996:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c305e7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29470720->29474815] (req 29470720->29474815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.871008:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c305e7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29470720->29474815] (req 29470720->29474815) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.871020:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c305e7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29470720->29474815] (req 29470720->29474815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.871028:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760796.871033:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c305e7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29470720->29474815] (req 29470720->29474815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.871042:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c305e7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29470720->29474815] (req 29470720->29474815) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.871059:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c305e7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29470720->29474815] (req 29470720->29474815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.871066:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d89a9821/0xe6aa143190c305e7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29470720->29474815] (req 29470720->29474815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760796.871168:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4076300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2360:x1876699512785920:12345-192.168.203.56@tcp:4:dd.0 Request processed in 64105us (64373us total) trans 4294974514 rc 0/0 00000100:00100000:3.0:1789760796.871177:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7225 00000100:00100000:3.0:1789760796.871186:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 776337052151 time 776371347682 state 0x2 ref 0 00000001:00080000:3.0:1789760796.871226:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974514 is committed 00000100:00100000:0.0:1789760796.880810:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760796.881005:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512786048 00000100:00100000:1.0:1789760796.881015:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512786048 00000100:00100000:1.0:1789760796.881068:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 776337052151 time 776371347682 state 0x0 ref 1 00000100:00100000:1.0:1789760796.881082:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 776371608098 time 776371347682 state 0x1 ref 1 00000100:00100000:1.0:1789760796.881123:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 776371608098 time 776445422791 state 0x0 ref 1 00000100:00100000:1.0:1789760796.881135:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febe7905c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512786048:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760796.881182:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 29474816, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0xe807 00010000:00010000:1.0:1789760796.881191:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c305ee lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.881208:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=776s v=(0 0) 00010000:00010000:1.0:1789760796.881212:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760796.881218:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c305ee lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29474816->29478911] (req 29474816->29478911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.881231:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c305ee lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29474816->29478911] (req 29474816->29478911) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760796.924525:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974515 is committed 00002000:00010000:2.0:1789760796.924815:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 29474816 -> 29478912 00010000:00010000:2.0:1789760796.924832:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c305ee lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29474816->29478911] (req 29474816->29478911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.924845:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c305ee lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29474816->29478911] (req 29474816->29478911) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.924855:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c305ee lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29474816->29478911] (req 29474816->29478911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.924862:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760796.924866:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c305ee lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29474816->29478911] (req 29474816->29478911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.924875:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c305ee lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29474816->29478911] (req 29474816->29478911) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.924893:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c305ee lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29474816->29478911] (req 29474816->29478911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.924901:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002117afc5/0xe6aa143190c305ee lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29474816->29478911] (req 29474816->29478911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760796.924992:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febe7905c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512786048:12345-192.168.203.56@tcp:4:dd.0 Request processed in 43854us (44186us total) trans 4294974515 rc 0/0 00000100:00100000:2.0:1789760796.925001:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7226 00000100:00100000:2.0:1789760796.925010:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 776371608098 time 776445422791 state 0x2 ref 0 00000100:00100000:2.0:1789760796.927292:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760796.927670:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512786176 00000100:00100000:1.0:1789760796.927680:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512786176 00000100:00100000:1.0:1789760796.927775:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 776371608098 time 776445422791 state 0x0 ref 1 00000100:00100000:1.0:1789760796.927791:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 776445683207 time 776445422791 state 0x1 ref 1 00000100:00100000:1.0:1789760796.927820:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 776445683207 time 776492119315 state 0x0 ref 1 00000100:00100000:1.0:1789760796.927832:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febfb897100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512786176:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760796.927876:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 29478912, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0xe807 00010000:00010000:1.0:1789760796.927885:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c305f5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.927900:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=776s v=(0 0) 00010000:00010000:1.0:1789760796.927904:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760796.927910:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c305f5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29478912->29483007] (req 29478912->29483007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.927938:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c305f5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29478912->29483007] (req 29478912->29483007) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760796.963791:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974516 is committed 00002000:00010000:3.0:1789760796.963881:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 29478912 -> 29483008 00010000:00010000:3.0:1789760796.963896:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c305f5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29478912->29483007] (req 29478912->29483007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.963909:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c305f5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29478912->29483007] (req 29478912->29483007) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.963917:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c305f5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29478912->29483007] (req 29478912->29483007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.963932:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760796.963937:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c305f5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29478912->29483007] (req 29478912->29483007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.963945:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c305f5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29478912->29483007] (req 29478912->29483007) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.963966:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c305f5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29478912->29483007] (req 29478912->29483007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.963973:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000be33b024/0xe6aa143190c305f5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29478912->29483007] (req 29478912->29483007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760796.964175:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febfb897100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512786176:12345-192.168.203.56@tcp:4:dd.0 Request processed in 36341us (36887us total) trans 4294974516 rc 0/0 00000100:00100000:3.0:1789760796.964185:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7227 00000100:00100000:3.0:1789760796.964194:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 776445683207 time 776492119315 state 0x2 ref 0 00000100:00100000:1.0:1789760796.967639:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760796.967810:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512786304 00000100:00100000:3.0:1789760796.967821:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512786304 00000100:00100000:3.0:1789760796.967875:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 776445683207 time 776492119315 state 0x0 ref 1 00000100:00100000:3.0:1789760796.967891:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 776492379731 time 776492119315 state 0x1 ref 1 00000100:00100000:3.0:1789760796.967936:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 776492379731 time 776532235236 state 0x0 ref 1 00000100:00100000:3.0:1789760796.967949:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4075180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512786304:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760796.967999:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 29483008, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0xe807 00010000:00010000:3.0:1789760796.968007:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c305fc lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.968024:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=776s v=(0 0) 00010000:00010000:3.0:1789760796.968028:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760796.968035:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c305fc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29483008->29487103] (req 29483008->29487103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.968049:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c305fc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29483008->29487103] (req 29483008->29487103) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.044375:0:5836:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:2.0:1789760797.056167:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974517 is committed 00002000:00010000:1.0:1789760797.059125:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 29483008 -> 29487104 00010000:00010000:1.0:1789760797.059144:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c305fc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29483008->29487103] (req 29483008->29487103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.059156:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c305fc lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29483008->29487103] (req 29483008->29487103) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.059165:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c305fc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29483008->29487103] (req 29483008->29487103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.059178:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760797.059183:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c305fc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29483008->29487103] (req 29483008->29487103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.059192:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c305fc lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29483008->29487103] (req 29483008->29487103) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.059211:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c305fc lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29483008->29487103] (req 29483008->29487103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.059220:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c80eb7fb/0xe6aa143190c305fc lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29483008->29487103] (req 29483008->29487103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760797.059321:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4075180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512786304:12345-192.168.203.56@tcp:4:dd.0 Request processed in 91371us (91687us total) trans 4294974517 rc 0/0 00000100:00100000:1.0:1789760797.059331:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7228 00000100:00100000:1.0:1789760797.059341:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 776492379731 time 776532235236 state 0x2 ref 0 00000100:00100000:2.0:1789760797.063119:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760797.064107:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512786432 00000100:00100000:1.0:1789760797.064118:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512786432 00000100:00100000:1.0:1789760797.064191:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 776492379731 time 776532235236 state 0x0 ref 1 00000100:00100000:1.0:1789760797.064206:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 776532495652 time 776532235236 state 0x1 ref 1 00000100:00100000:1.0:1789760797.064236:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 776532495652 time 776628536032 state 0x0 ref 1 00000100:00100000:1.0:1789760797.064252:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febfb895880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512786432:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760797.064301:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 29487104, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0xe807 00010000:00010000:1.0:1789760797.064310:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c30603 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.064325:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=776s v=(0 0) 00010000:00010000:1.0:1789760797.064330:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760797.064336:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c30603 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29487104->29491199] (req 29487104->29491199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.064347:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c30603 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29487104->29491199] (req 29487104->29491199) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1789760797.065002:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00010000:3.0:1789760797.138301:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 29487104 -> 29491200 00010000:00010000:3.0:1789760797.138320:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c30603 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29487104->29491199] (req 29487104->29491199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.138333:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c30603 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29487104->29491199] (req 29487104->29491199) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.138342:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c30603 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29487104->29491199] (req 29487104->29491199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.138348:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760797.138353:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c30603 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29487104->29491199] (req 29487104->29491199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.138361:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c30603 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29487104->29491199] (req 29487104->29491199) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.138378:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c30603 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29487104->29491199] (req 29487104->29491199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.138387:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a3b1c979/0xe6aa143190c30603 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29487104->29491199] (req 29487104->29491199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760797.138482:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febfb895880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2360:x1876699512786432:12345-192.168.203.56@tcp:4:dd.0 Request processed in 74228us (75368us total) trans 4294974518 rc 0/0 00000100:00100000:3.0:1789760797.138494:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7229 00000100:00100000:3.0:1789760797.138504:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 776532495652 time 776628536032 state 0x2 ref 0 00000001:00080000:2.0:1789760797.139286:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974518 is committed 00000100:00100000:3.0:1789760797.143044:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760797.143332:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512786560 00000100:00100000:1.0:1789760797.143344:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512786560 00000100:00100000:1.0:1789760797.143396:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 776532495652 time 776628536032 state 0x0 ref 1 00000100:00100000:1.0:1789760797.143410:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 776628796448 time 776628536032 state 0x1 ref 1 00000100:00100000:1.0:1789760797.143441:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 776628796448 time 776707740828 state 0x0 ref 1 00000100:00100000:1.0:1789760797.143453:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5307100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512786560:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760797.143502:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 29491200, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0xe807 00010000:00010000:1.0:1789760797.143511:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c3060a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.143533:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=776s v=(0 0) 00010000:00010000:1.0:1789760797.143537:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760797.143544:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c3060a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29491200->29495295] (req 29491200->29495295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.143557:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c3060a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29491200->29495295] (req 29491200->29495295) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760797.189392:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974519 is committed 00002000:00010000:1.0:1789760797.189495:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 29491200 -> 29495296 00010000:00010000:1.0:1789760797.189511:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c3060a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29491200->29495295] (req 29491200->29495295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.189524:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c3060a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29491200->29495295] (req 29491200->29495295) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.189534:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c3060a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29491200->29495295] (req 29491200->29495295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.189542:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760797.189547:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c3060a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29491200->29495295] (req 29491200->29495295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.189558:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c3060a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29491200->29495295] (req 29491200->29495295) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.189577:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c3060a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29491200->29495295] (req 29491200->29495295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.189587:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c3060a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29491200->29495295] (req 29491200->29495295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760797.189878:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5307100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512786560:12345-192.168.203.56@tcp:4:dd.0 Request processed in 46421us (46837us total) trans 4294974519 rc 0/0 00000100:00100000:1.0:1789760797.189889:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7230 00000100:00100000:1.0:1789760797.189898:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 776628796448 time 776707740828 state 0x2 ref 0 00000100:00100000:0.0:1789760797.194275:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760797.194351:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512786688 00000100:00100000:1.0:1789760797.194362:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512786688 00000100:00100000:1.0:1789760797.194419:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 776628796448 time 776707740828 state 0x0 ref 1 00000100:00100000:1.0:1789760797.194435:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 776708001244 time 776707740828 state 0x1 ref 1 00000100:00100000:1.0:1789760797.194471:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 776708001244 time 776758770253 state 0x0 ref 1 00000100:00100000:1.0:1789760797.194485:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febe7905500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512786688:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760797.194535:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 29495296, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0xe807 00010000:00010000:1.0:1789760797.194545:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c30611 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.194560:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=776s v=(0 0) 00010000:00010000:1.0:1789760797.194564:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760797.194571:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c30611 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29495296->29499391] (req 29495296->29499391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.194591:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c30611 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29495296->29499391] (req 29495296->29499391) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760797.233095:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974520 is committed 00002000:00010000:2.0:1789760797.233301:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 29495296 -> 29499392 00010000:00010000:2.0:1789760797.233327:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c30611 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29495296->29499391] (req 29495296->29499391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.233341:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c30611 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29495296->29499391] (req 29495296->29499391) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.233351:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c30611 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29495296->29499391] (req 29495296->29499391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.233358:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760797.233363:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c30611 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29495296->29499391] (req 29495296->29499391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.233372:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c30611 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29495296->29499391] (req 29495296->29499391) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.233392:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c30611 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29495296->29499391] (req 29495296->29499391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.233401:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007980ff7a/0xe6aa143190c30611 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29495296->29499391] (req 29495296->29499391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760797.233490:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febe7905500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512786688:12345-192.168.203.56@tcp:4:dd.0 Request processed in 39003us (39220us total) trans 4294974520 rc 0/0 00000100:00100000:2.0:1789760797.233501:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7231 00000100:00100000:2.0:1789760797.233511:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 776708001244 time 776758770253 state 0x2 ref 0 00000100:00100000:2.0:1789760797.236830:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1789760797.237102:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512786816 00000100:00100000:3.0:1789760797.237119:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512786816 00000100:00100000:3.0:1789760797.237168:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 776708001244 time 776758770253 state 0x0 ref 1 00000100:00100000:3.0:1789760797.237184:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 776759030669 time 776758770253 state 0x1 ref 1 00000100:00100000:3.0:1789760797.237210:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 776759030669 time 776801509679 state 0x0 ref 1 00000100:00100000:3.0:1789760797.237221:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4214700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512786816:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:3.0:1789760797.237274:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 29499392, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0xe807 00010000:00010000:3.0:1789760797.237283:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c30618 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.237298:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=776s v=(0 0) 00010000:00010000:3.0:1789760797.237303:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760797.237309:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c30618 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29499392->29503487] (req 29499392->29503487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.237323:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c30618 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29499392->29503487] (req 29499392->29503487) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760797.281949:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 29499392 -> 29503488 00000001:00080000:2.0:1789760797.281954:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974521 is committed 00010000:00010000:1.0:1789760797.281968:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c30618 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29499392->29503487] (req 29499392->29503487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.281981:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c30618 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29499392->29503487] (req 29499392->29503487) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.281991:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c30618 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29499392->29503487] (req 29499392->29503487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.282008:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfc3f3600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760797.282014:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c30618 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29499392->29503487] (req 29499392->29503487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.282023:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c30618 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29499392->29503487] (req 29499392->29503487) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.282042:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c30618 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29499392->29503487] (req 29499392->29503487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.282050:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a8ad84a5/0xe6aa143190c30618 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29499392->29503487] (req 29499392->29503487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760797.282147:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4214700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512786816:12345-192.168.203.56@tcp:4:dd.0 Request processed in 44921us (45321us total) trans 4294974521 rc 0/0 00000100:00100000:1.0:1789760797.282157:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7232 00000100:00100000:1.0:1789760797.282167:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 776759030669 time 776801509679 state 0x2 ref 0 00000100:00100000:3.0:1789760797.284051:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760797.284165:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512786944 00000100:00100000:1.0:1789760797.284175:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512786944 00000100:00100000:1.0:1789760797.284227:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 776759030669 time 776801509679 state 0x0 ref 1 00000100:00100000:1.0:1789760797.284241:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 776801770095 time 776801509679 state 0x1 ref 1 00000100:00100000:1.0:1789760797.284273:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 776801770095 time 776848572178 state 0x0 ref 1 00000100:00100000:1.0:1789760797.284285:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5304a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512786944:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760797.284341:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 29503488, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0xe807 00010000:00010000:1.0:1789760797.284350:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c3061f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.284366:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=776s v=(0 0) 00010000:00010000:1.0:1789760797.284371:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760797.284378:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c3061f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29503488->29507583] (req 29503488->29507583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.284392:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c3061f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29503488->29507583] (req 29503488->29507583) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760797.329326:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974522 is committed 00002000:00010000:1.0:1789760797.329461:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 29503488 -> 29507584 00010000:00010000:1.0:1789760797.329477:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c3061f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29503488->29507583] (req 29503488->29507583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.329491:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c3061f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29503488->29507583] (req 29503488->29507583) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.329501:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c3061f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29503488->29507583] (req 29503488->29507583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.329509:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760797.329514:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c3061f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29503488->29507583] (req 29503488->29507583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.329523:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c3061f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29503488->29507583] (req 29503488->29507583) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.329543:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c3061f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29503488->29507583] (req 29503488->29507583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.329550:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c3061f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29503488->29507583] (req 29503488->29507583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760797.329847:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5304a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512786944:12345-192.168.203.56@tcp:4:dd.0 Request processed in 45558us (45798us total) trans 4294974522 rc 0/0 00000100:00100000:1.0:1789760797.329862:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7233 00000100:00100000:1.0:1789760797.329872:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 776801770095 time 776848572178 state 0x2 ref 0 00000100:00100000:0.0:1789760797.334108:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760797.334198:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512787072 00000100:00100000:2.0:1789760797.334207:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512787072 00000100:00100000:2.0:1789760797.334257:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 776801770095 time 776848572178 state 0x0 ref 1 00000100:00100000:2.0:1789760797.334272:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 776848832594 time 776848572178 state 0x1 ref 1 00000100:00100000:2.0:1789760797.334302:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 776848832594 time 776898601534 state 0x0 ref 1 00000100:00100000:2.0:1789760797.334313:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febe7905f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512787072:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760797.334356:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 29507584, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0xe807 00010000:00010000:2.0:1789760797.334365:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c30626 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.334381:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=776s v=(0 0) 00010000:00010000:2.0:1789760797.334385:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760797.334391:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c30626 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29507584->29511679] (req 29507584->29511679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.334406:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c30626 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29507584->29511679] (req 29507584->29511679) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760797.383322:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 29507584 -> 29511680 00010000:00010000:1.0:1789760797.383343:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c30626 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29507584->29511679] (req 29507584->29511679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.383357:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c30626 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29507584->29511679] (req 29507584->29511679) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.383366:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c30626 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29507584->29511679] (req 29507584->29511679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.383375:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760797.383379:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c30626 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29507584->29511679] (req 29507584->29511679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.383389:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c30626 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29507584->29511679] (req 29507584->29511679) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.383408:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c30626 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29507584->29511679] (req 29507584->29511679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.383416:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001a9a8278/0xe6aa143190c30626 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29507584->29511679] (req 29507584->29511679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760797.383514:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febe7905f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2358:x1876699512787072:12345-192.168.203.56@tcp:4:dd.0 Request processed in 49197us (49409us total) trans 4294974523 rc 0/0 00000100:00100000:1.0:1789760797.383524:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7234 00000100:00100000:1.0:1789760797.383534:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 776848832594 time 776898601534 state 0x2 ref 0 00000001:00080000:2.0:1789760797.383656:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974523 is committed 00000100:00100000:0.0:1789760797.386953:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760797.387054:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512787200 00000100:00100000:1.0:1789760797.387065:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512787200 00000100:00100000:1.0:1789760797.387118:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 776848832594 time 776898601534 state 0x0 ref 1 00000100:00100000:1.0:1789760797.387133:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 776898861950 time 776898601534 state 0x1 ref 1 00000100:00100000:1.0:1789760797.387166:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 776898861950 time 776951465589 state 0x0 ref 1 00000100:00100000:1.0:1789760797.387180:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febe7905880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512787200:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760797.387229:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 29511680, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0xe807 00010000:00010000:1.0:1789760797.387238:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c3062d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.387254:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=776s v=(0 0) 00010000:00010000:1.0:1789760797.387258:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760797.387264:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c3062d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29511680->29515775] (req 29511680->29515775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.387278:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c3062d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29511680->29515775] (req 29511680->29515775) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760797.421984:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 29511680 -> 29515776 00000001:00080000:2.0:1789760797.422123:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974524 is committed 00010000:00010000:1.0:1789760797.422202:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c3062d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29511680->29515775] (req 29511680->29515775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.422216:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c3062d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29511680->29515775] (req 29511680->29515775) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.422224:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c3062d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29511680->29515775] (req 29511680->29515775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.422232:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760797.422236:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c3062d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29511680->29515775] (req 29511680->29515775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.422246:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c3062d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29511680->29515775] (req 29511680->29515775) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.422266:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c3062d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29511680->29515775] (req 29511680->29515775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.422275:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c3062d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29511680->29515775] (req 29511680->29515775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760797.422398:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febe7905880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512787200:12345-192.168.203.56@tcp:4:dd.0 Request processed in 35216us (35450us total) trans 4294974524 rc 0/0 00000100:00100000:1.0:1789760797.422412:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7235 00000100:00100000:1.0:1789760797.422421:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 776898861950 time 776951465589 state 0x2 ref 0 00000100:00100000:3.0:1789760797.430811:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760797.430850:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512787328 00000100:00100000:1.0:1789760797.430860:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512787328 00000100:00100000:1.0:1789760797.430911:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 776898861950 time 776951465589 state 0x0 ref 1 00000100:00100000:1.0:1789760797.430939:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 776951726005 time 776951465589 state 0x1 ref 1 00000100:00100000:1.0:1789760797.430975:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 776951726005 time 776995274704 state 0x0 ref 1 00000100:00100000:1.0:1789760797.431003:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac5305880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512787328:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760797.431057:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 29515776, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0xe807 00010000:00010000:1.0:1789760797.431066:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c30634 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.431081:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=776s v=(0 0) 00010000:00010000:1.0:1789760797.431086:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760797.431092:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c30634 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29515776->29519871] (req 29515776->29519871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.431116:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c30634 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29515776->29519871] (req 29515776->29519871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760797.477781:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974525 is committed 00002000:00010000:2.0:1789760797.477875:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 29515776 -> 29519872 00010000:00010000:2.0:1789760797.477890:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c30634 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29515776->29519871] (req 29515776->29519871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.477902:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c30634 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29515776->29519871] (req 29515776->29519871) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.477912:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c30634 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29515776->29519871] (req 29515776->29519871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.477931:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5400 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760797.477936:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c30634 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29515776->29519871] (req 29515776->29519871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.477945:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c30634 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29515776->29519871] (req 29515776->29519871) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.477966:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c30634 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29515776->29519871] (req 29515776->29519871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.477974:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009a0f4074/0xe6aa143190c30634 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29515776->29519871] (req 29515776->29519871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760797.478078:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac5305880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512787328:12345-192.168.203.56@tcp:4:dd.0 Request processed in 47080us (47271us total) trans 4294974525 rc 0/0 00000100:00100000:2.0:1789760797.478087:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7236 00000100:00100000:2.0:1789760797.478097:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 776951726005 time 776995274704 state 0x2 ref 0 00000100:00100000:2.0:1789760797.482036:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760797.482310:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512787456 00000100:00100000:1.0:1789760797.482320:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512787456 00000100:00100000:1.0:1789760797.482475:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 776951726005 time 776995274704 state 0x0 ref 1 00000100:00100000:1.0:1789760797.482496:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 776995535120 time 776995274704 state 0x1 ref 1 00000100:00100000:1.0:1789760797.482561:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 776995535120 time 777046859977 state 0x0 ref 1 00000100:00100000:1.0:1789760797.482575:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4214a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512787456:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760797.482622:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 29519872, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0xe807 00010000:00010000:1.0:1789760797.482631:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c3063b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.482646:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=777s v=(0 0) 00010000:00010000:1.0:1789760797.482651:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760797.482657:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c3063b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29519872->29523967] (req 29519872->29523967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.482671:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c3063b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29519872->29523967] (req 29519872->29523967) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760797.547596:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974526 is committed 00002000:00010000:1.0:1789760797.547960:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 29519872 -> 29523968 00010000:00010000:1.0:1789760797.547976:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c3063b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29519872->29523967] (req 29519872->29523967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.547990:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c3063b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29519872->29523967] (req 29519872->29523967) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.547999:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c3063b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29519872->29523967] (req 29519872->29523967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.548007:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760797.548012:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c3063b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29519872->29523967] (req 29519872->29523967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.548021:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c3063b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29519872->29523967] (req 29519872->29523967) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.548040:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c3063b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29519872->29523967] (req 29519872->29523967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.548048:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c3063b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29519872->29523967] (req 29519872->29523967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760797.548182:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4214a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2359:x1876699512787456:12345-192.168.203.56@tcp:4:dd.0 Request processed in 65604us (66149us total) trans 4294974526 rc 0/0 00000100:00100000:1.0:1789760797.548192:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7237 00000100:00100000:1.0:1789760797.548202:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 776995535120 time 777046859977 state 0x2 ref 0 00000100:00100000:0.0:1789760797.559486:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760797.559673:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512787584 00000100:00100000:1.0:1789760797.559684:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512787584 00000100:00100000:1.0:1789760797.559770:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 776995535120 time 777046859977 state 0x0 ref 1 00000100:00100000:1.0:1789760797.559787:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 777047120393 time 777046859977 state 0x1 ref 1 00000100:00100000:1.0:1789760797.559818:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 777047120393 time 777124117219 state 0x0 ref 1 00000100:00100000:1.0:1789760797.559833:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febe7907100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512787584:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760797.559881:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 29523968, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0xe807 00010000:00010000:1.0:1789760797.559890:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c30642 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.559907:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=777s v=(0 0) 00010000:00010000:1.0:1789760797.559911:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760797.559928:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c30642 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29523968->29528063] (req 29523968->29528063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.559944:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c30642 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29523968->29528063] (req 29523968->29528063) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760797.590251:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974527 is committed 00002000:00010000:3.0:1789760797.590417:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 29523968 -> 29528064 00002000:00010000:3.0:1789760797.590422:0:13063:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760796 -> 1789760797 00002000:00010000:3.0:1789760797.590424:0:13063:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760796 -> 1789760797 00010000:00010000:3.0:1789760797.590437:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c30642 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29523968->29528063] (req 29523968->29528063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.590457:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c30642 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29523968->29528063] (req 29523968->29528063) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.590466:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c30642 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29523968->29528063] (req 29523968->29528063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.590473:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760797.590478:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c30642 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29523968->29528063] (req 29523968->29528063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.590487:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c30642 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29523968->29528063] (req 29523968->29528063) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.590506:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c30642 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29523968->29528063] (req 29523968->29528063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.590514:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006e0e4e61/0xe6aa143190c30642 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29523968->29528063] (req 29523968->29528063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760797.590739:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febe7907100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512787584:12345-192.168.203.56@tcp:4:dd.0 Request processed in 30866us (31219us total) trans 4294974527 rc 0/0 00000100:00100000:3.0:1789760797.590750:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7238 00000100:00100000:3.0:1789760797.590760:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 777047120393 time 777124117219 state 0x2 ref 0 00000100:00100000:1.0:1789760797.594757:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760797.594949:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512787712 00000100:00100000:2.0:1789760797.594958:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512787712 00000100:00100000:2.0:1789760797.595011:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 777047120393 time 777124117219 state 0x0 ref 1 00000100:00100000:2.0:1789760797.595030:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 777124377635 time 777124117219 state 0x1 ref 1 00000100:00100000:2.0:1789760797.595063:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 777124377635 time 777159361451 state 0x0 ref 1 00000100:00100000:2.0:1789760797.595075:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4074a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512787712:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760797.595121:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 29528064, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0xe807 00010000:00010000:2.0:1789760797.595130:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c30649 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.595146:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=777s v=(0 0) 00010000:00010000:2.0:1789760797.595151:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760797.595158:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c30649 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29528064->29532159] (req 29528064->29532159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.595170:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c30649 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29528064->29532159] (req 29528064->29532159) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760797.618995:0:3310:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8feac4074000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3310:1876699540234368:0@lo:400:kworker.0 00000100:00100000:1.0:1789760797.619061:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760797.619122:0:3310:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8febfbddca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3310:1876699540234880:0@lo:400:kworker.0 00000100:00100000:1.0:1789760797.619150:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760797.619180:0:3310:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:1.0:1789760797.619187:0:3310:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8feac4074700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:6df98d54-eb52-4c5c-8fc9-ee9d74f90f98:3310:1876699540234240:0@lo:400:kworker.0 00000100:00100000:1.0:1789760797.619217:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760797.619236:0:3310:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8febfbddc000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3310:1876699540234752:0@lo:400:kworker.0 00000100:00100000:1.0:1789760797.619259:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760797.619283:0:3311:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8feac4076a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3311:1876699540234496:0@lo:400:kworker.0 00000100:00100000:1.0:1789760797.619427:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760797.619450:0:3311:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1789760797.619456:0:3311:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8febfbdddc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3311:1876699540234624:0@lo:400:kworker.0 00000100:00100000:1.0:1789760797.619481:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760797.619512:0:8229:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540234368 00000100:00100000:1.0:1789760797.619518:0:8229:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540234368 00000100:00100000:1.0:1789760797.619558:0:8229:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540234752 00000100:00100000:1.0:1789760797.619562:0:8229:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540234752 00000100:00100000:1.0:1789760797.619594:0:8229:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540234496 00000100:00100000:1.0:1789760797.619598:0:8229:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540234496 00000100:00100000:1.0:1789760797.619620:0:8229:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 526 00000100:00100000:1.0:1789760797.619627:0:8229:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febff92aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3310:x1876699540234368:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760797.619684:0:8229:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febff92aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3310:x1876699540234368:12345-0@lo:400:kworker.0 Request processed in 57us (623us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760797.619691:0:8229:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 526 00000100:00100000:1.0:1789760797.619727:0:8229:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 527 00000100:00100000:1.0:1789760797.619732:0:8229:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febfabf4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3310:x1876699540234752:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760797.619757:0:8229:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febfabf4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3310:x1876699540234752:12345-0@lo:400:kworker.0 Request processed in 25us (498us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760797.619763:0:8229:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 527 00000100:00100000:1.0:1789760797.619772:0:8229:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 528 00000100:00100000:1.0:1789760797.619778:0:8229:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febfabf6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3311:x1876699540234496:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760797.619804:0:8229:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febfabf6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3311:x1876699540234496:12345-0@lo:400:kworker.0 Request processed in 26us (496us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760797.619809:0:8229:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 528 00000100:00100000:1.0:1789760797.619824:0:13064:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540234880 00000100:00100000:1.0:1789760797.619826:0:13064:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540234880 00000100:00100000:1.0:1789760797.619848:0:13064:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540234624 00000100:00100000:1.0:1789760797.619851:0:13064:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540234624 00000100:00100000:1.0:1789760797.619866:0:13064:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 199 00000100:00100000:1.0:1789760797.619872:0:13064:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febfabf4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3310:x1876699540234880:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760797.619899:0:13064:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febfabf4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3310:x1876699540234880:12345-0@lo:400:kworker.0 Request processed in 27us (749us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760797.619905:0:13064:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 199 00000100:00100000:0.0:1789760797.620814:0:3310:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8febfbddc000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3310:1876699540234752:0@lo:400:kworker.0 00000100:00100000:0.0:1789760797.620831:0:3310:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8feac4074000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3310:1876699540234368:0@lo:400:kworker.0 00000100:00100000:0.0:1789760797.620841:0:3310:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8febfbddca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3310:1876699540234880:0@lo:400:kworker.0 00000100:00100000:1.0:1789760797.620926:0:13064:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 200 00000100:00100000:1.0:1789760797.620940:0:13064:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febea772680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3311:x1876699540234624:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760797.620991:0:13064:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febea772680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3311:x1876699540234624:12345-0@lo:400:kworker.0 Request processed in 55us (1508us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760797.620997:0:13064:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 200 00000100:00100000:1.0:1789760797.621025:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540234240 00000100:00100000:1.0:1789760797.621029:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540234240 00000100:00100000:1.0:1789760797.621056:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 419 00000100:00100000:1.0:1789760797.621061:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febe00bf450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:6df98d54-eb52-4c5c-8fc9-ee9d74f90f98+19:3310:x1876699540234240:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760797.621111:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febe00bf450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:6df98d54-eb52-4c5c-8fc9-ee9d74f90f98+19:3310:x1876699540234240:12345-0@lo:400:kworker.0 Request processed in 49us (1894us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760797.621118:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 419 00000100:00100000:1.0:1789760797.621161:0:3312:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1789760797.621173:0:3312:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8feac4214e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3312:1876699540235008:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1789760797.621202:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760797.621230:0:6707:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540235008 00000100:00100000:1.0:1789760797.621234:0:6707:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540235008 00000100:00100000:1.0:1789760797.621258:0:6707:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 218 00000100:00100000:1.0:1789760797.621264:0:6707:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febea770380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3312:x1876699540235008:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1789760797.621277:0:6707:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 528547840 pending 1703936 free 3828350976 avail 3804233728 00000100:00100000:1.0:1789760797.621387:0:6707:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febea770380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3312:x1876699540235008:12345-0@lo:13:osp-pre-0-0.0 Request processed in 122us (183us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760797.621393:0:6707:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 218 00000100:00100000:1.0:1789760797.621427:0:3312:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8feac4214e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3312:1876699540235008:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1789760797.621451:0:3311:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8febfbdddc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3311:1876699540234624:0@lo:400:kworker.0 00000100:00100000:1.0:1789760797.621462:0:3311:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8feac4076a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3311:1876699540234496:0@lo:400:kworker.0 00000100:00100000:1.0:1789760797.621482:0:3310:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8feac4074700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:6df98d54-eb52-4c5c-8fc9-ee9d74f90f98:3310:1876699540234240:0@lo:400:kworker.0 00000100:00080000:3.0:1789760797.621788:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:3.0:1789760797.621793:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:3.0:1789760797.621795:0:5843:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000001:00080000:0.0:1789760797.623331:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974528 is committed 00002000:00010000:2.0:1789760797.623503:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 29528064 -> 29532160 00010000:00010000:2.0:1789760797.623518:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c30649 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29528064->29532159] (req 29528064->29532159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.623532:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c30649 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29528064->29532159] (req 29528064->29532159) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.623541:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c30649 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29528064->29532159] (req 29528064->29532159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.623555:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760797.623561:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c30649 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29528064->29532159] (req 29528064->29532159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.623571:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c30649 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29528064->29532159] (req 29528064->29532159) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.623655:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c30649 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29528064->29532159] (req 29528064->29532159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.623665:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c30649 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29528064->29532159] (req 29528064->29532159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760797.623832:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4074a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512787712:12345-192.168.203.56@tcp:4:dd.0 Request processed in 28754us (29080us total) trans 4294974528 rc 0/0 00000100:00100000:2.0:1789760797.623843:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7239 00000100:00100000:2.0:1789760797.623853:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 777124377635 time 777159361451 state 0x2 ref 0 00000100:00100000:3.0:1789760797.626078:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760797.626214:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512787840 00000100:00100000:1.0:1789760797.626224:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512787840 00000100:00100000:1.0:1789760797.626271:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 777124377635 time 777159361451 state 0x0 ref 1 00000100:00100000:1.0:1789760797.626286:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 777159621867 time 777159361451 state 0x1 ref 1 00000100:00100000:1.0:1789760797.626323:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 777159621867 time 777190622774 state 0x0 ref 1 00000100:00100000:1.0:1789760797.626335:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febfb89ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512787840:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760797.626379:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 29532160, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0xe807 00010000:00010000:1.0:1789760797.626387:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c30650 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.626401:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=777s v=(0 0) 00010000:00010000:1.0:1789760797.626405:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760797.626412:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c30650 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29532160->29536255] (req 29532160->29536255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.626429:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c30650 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29532160->29536255] (req 29532160->29536255) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760797.649184:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 29532160 -> 29536256 00010000:00010000:2.0:1789760797.649203:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c30650 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29532160->29536255] (req 29532160->29536255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.649216:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c30650 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29532160->29536255] (req 29532160->29536255) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.649230:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c30650 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29532160->29536255] (req 29532160->29536255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.649236:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760797.649241:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c30650 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29532160->29536255] (req 29532160->29536255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.649250:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c30650 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29532160->29536255] (req 29532160->29536255) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.649268:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c30650 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29532160->29536255] (req 29532160->29536255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.649275:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007a0dc4dd/0xe6aa143190c30650 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29532160->29536255] (req 29532160->29536255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760797.649370:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febfb89ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2360:x1876699512787840:12345-192.168.203.56@tcp:4:dd.0 Request processed in 23031us (23295us total) trans 4294974529 rc 0/0 00000100:00100000:2.0:1789760797.649379:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7240 00000100:00100000:2.0:1789760797.649389:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 777159621867 time 777190622774 state 0x2 ref 0 00000100:00100000:3.0:1789760797.652670:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760797.652899:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512787968 00000100:00100000:1.0:1789760797.652909:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512787968 00000100:00100000:1.0:1789760797.652974:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 777159621867 time 777190622774 state 0x0 ref 1 00000100:00100000:1.0:1789760797.652989:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 777190883190 time 777190622774 state 0x1 ref 1 00000100:00100000:1.0:1789760797.653020:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 777190883190 time 777217319502 state 0x0 ref 1 00000100:00100000:1.0:1789760797.653033:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febfb89fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2361:x1876699512787968:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760797.653083:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 29536256, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0xe807 00010000:00010000:1.0:1789760797.653092:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c30657 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.653107:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=777s v=(0 0) 00010000:00010000:1.0:1789760797.653112:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760797.653118:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c30657 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29536256->29540351] (req 29536256->29540351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.653133:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c30657 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29536256->29540351] (req 29536256->29540351) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760797.654241:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974529 is committed 00000001:00080000:1.0:1789760797.681304:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974530 is committed 00002000:00010000:0.0:1789760797.681418:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 29536256 -> 29540352 00010000:00010000:0.0:1789760797.681439:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c30657 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29536256->29540351] (req 29536256->29540351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.681452:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c30657 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29536256->29540351] (req 29536256->29540351) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.681465:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c30657 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29536256->29540351] (req 29536256->29540351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.681474:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af4c00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760797.681479:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c30657 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29536256->29540351] (req 29536256->29540351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.681487:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c30657 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29536256->29540351] (req 29536256->29540351) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.681507:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c30657 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29536256->29540351] (req 29536256->29540351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.681516:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000096cdc479/0xe6aa143190c30657 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29536256->29540351] (req 29536256->29540351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760797.681627:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febfb89fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+7:2361:x1876699512787968:12345-192.168.203.56@tcp:4:dd.0 Request processed in 28592us (28962us total) trans 4294974530 rc 0/0 00000100:00100000:0.0:1789760797.681637:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7241 00000100:00100000:0.0:1789760797.681647:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 777190883190 time 777217319502 state 0x2 ref 0 00000100:00100000:1.0:1789760797.684404:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760797.684625:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512788096 00000100:00100000:2.0:1789760797.684635:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512788096 00000100:00100000:2.0:1789760797.684687:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 777190883190 time 777217319502 state 0x0 ref 1 00000100:00100000:2.0:1789760797.684732:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 777217579918 time 777217319502 state 0x1 ref 1 00000100:00100000:2.0:1789760797.684763:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 777217579918 time 777249063455 state 0x0 ref 1 00000100:00100000:2.0:1789760797.684775:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac38b6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512788096:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760797.684948:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 29540352, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0xe807 00010000:00010000:2.0:1789760797.684958:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c3065e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.684973:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=777s v=(0 0) 00010000:00010000:2.0:1789760797.684978:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760797.684984:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c3065e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29540352->29544447] (req 29540352->29544447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.685163:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c3065e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29540352->29544447] (req 29540352->29544447) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760797.712808:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974531 is committed 00002000:00010000:0.0:1789760797.713316:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 29540352 -> 29544448 00010000:00010000:0.0:1789760797.713332:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c3065e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29540352->29544447] (req 29540352->29544447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.713346:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c3065e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29540352->29544447] (req 29540352->29544447) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.713355:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c3065e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29540352->29544447] (req 29540352->29544447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.713363:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40a00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760797.713369:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c3065e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29540352->29544447] (req 29540352->29544447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.713384:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c3065e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29540352->29544447] (req 29540352->29544447) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.713404:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c3065e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29540352->29544447] (req 29540352->29544447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.713410:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000271eff56/0xe6aa143190c3065e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29540352->29544447] (req 29540352->29544447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760797.713551:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac38b6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512788096:12345-192.168.203.56@tcp:4:dd.0 Request processed in 28772us (29151us total) trans 4294974531 rc 0/0 00000100:00100000:0.0:1789760797.713563:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7242 00000100:00100000:0.0:1789760797.713574:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 777217579918 time 777249063455 state 0x2 ref 0 00000100:00100000:3.0:1789760797.720923:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760797.721065:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512788224 00000100:00100000:2.0:1789760797.721075:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512788224 00000100:00100000:2.0:1789760797.721128:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 777217579918 time 777249063455 state 0x0 ref 1 00000100:00100000:2.0:1789760797.721144:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 777249323871 time 777249063455 state 0x1 ref 1 00000100:00100000:2.0:1789760797.721173:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 777249323871 time 777285472563 state 0x0 ref 1 00000100:00100000:2.0:1789760797.721185:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2746300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512788224:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760797.721245:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 29544448, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0xe807 00010000:00010000:2.0:1789760797.721254:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c30665 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.721270:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=777s v=(0 0) 00010000:00010000:2.0:1789760797.721274:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760797.721280:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c30665 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29544448->29548543] (req 29544448->29548543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.721292:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c30665 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29544448->29548543] (req 29544448->29548543) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760797.752555:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974532 is committed 00002000:00010000:3.0:1789760797.752690:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 29544448 -> 29548544 00010000:00010000:3.0:1789760797.752720:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c30665 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29544448->29548543] (req 29544448->29548543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.752732:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c30665 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29544448->29548543] (req 29544448->29548543) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.752741:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c30665 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29544448->29548543] (req 29544448->29548543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.752748:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb41600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:3.0:1789760797.752752:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c30665 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29544448->29548543] (req 29544448->29548543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.752761:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c30665 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29544448->29548543] (req 29544448->29548543) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.752780:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c30665 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29544448->29548543] (req 29544448->29548543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.752788:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000107850e9/0xe6aa143190c30665 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29544448->29548543] (req 29544448->29548543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760797.752885:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2746300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512788224:12345-192.168.203.56@tcp:4:dd.0 Request processed in 31698us (31976us total) trans 4294974532 rc 0/0 00000100:00100000:3.0:1789760797.752895:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7243 00000100:00100000:3.0:1789760797.752904:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 777249323871 time 777285472563 state 0x2 ref 0 00000100:00100000:3.0:1789760797.755156:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760797.755320:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512788352 00000100:00100000:0.0:1789760797.755331:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512788352 00000100:00100000:0.0:1789760797.755383:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 777249323871 time 777285472563 state 0x0 ref 1 00000100:00100000:0.0:1789760797.755399:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 777285732979 time 777285472563 state 0x1 ref 1 00000100:00100000:0.0:1789760797.755429:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 777285732979 time 777319728083 state 0x0 ref 1 00000100:00100000:0.0:1789760797.755440:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2744a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512788352:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760797.755485:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 29548544, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0xe807 00010000:00010000:0.0:1789760797.755493:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c3066c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.755508:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=777s v=(0 0) 00010000:00010000:0.0:1789760797.755513:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760797.755519:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c3066c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29548544->29552639] (req 29548544->29552639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.755533:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c3066c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29548544->29552639] (req 29548544->29552639) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760797.784685:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974533 is committed 00002000:00010000:1.0:1789760797.785115:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 29548544 -> 29552640 00010000:00010000:1.0:1789760797.785132:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c3066c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29548544->29552639] (req 29548544->29552639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.785145:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c3066c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29548544->29552639] (req 29548544->29552639) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.785154:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c3066c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29548544->29552639] (req 29548544->29552639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.785162:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760797.785167:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c3066c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29548544->29552639] (req 29548544->29552639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.785176:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c3066c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29548544->29552639] (req 29548544->29552639) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.785196:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c3066c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29548544->29552639] (req 29548544->29552639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.785204:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c3066c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29548544->29552639] (req 29548544->29552639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760797.785331:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2744a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512788352:12345-192.168.203.56@tcp:4:dd.0 Request processed in 29888us (30179us total) trans 4294974533 rc 0/0 00000100:00100000:1.0:1789760797.785345:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7244 00000100:00100000:1.0:1789760797.785355:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 777285732979 time 777319728083 state 0x2 ref 0 00000100:00100000:1.0:1789760797.791129:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760797.791231:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512788480 00000100:00100000:0.0:1789760797.791242:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512788480 00000100:00100000:0.0:1789760797.791298:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 777285732979 time 777319728083 state 0x0 ref 1 00000100:00100000:0.0:1789760797.791313:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 777319988499 time 777319728083 state 0x1 ref 1 00000100:00100000:0.0:1789760797.791383:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 777319988499 time 777355681624 state 0x0 ref 1 00000100:00100000:0.0:1789760797.791397:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac38b4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512788480:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760797.791445:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 29552640, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0xe807 00010000:00010000:0.0:1789760797.791454:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c30673 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.791469:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=777s v=(0 0) 00010000:00010000:0.0:1789760797.791474:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760797.791480:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c30673 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29552640->29556735] (req 29552640->29556735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.791494:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c30673 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29552640->29556735] (req 29552640->29556735) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760797.823100:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974534 is committed 00002000:00010000:0.0:1789760797.823430:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 29552640 -> 29556736 00010000:00010000:0.0:1789760797.823448:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c30673 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29552640->29556735] (req 29552640->29556735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.823460:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c30673 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29552640->29556735] (req 29552640->29556735) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.823471:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c30673 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29552640->29556735] (req 29552640->29556735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.823478:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760797.823483:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c30673 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29552640->29556735] (req 29552640->29556735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.823492:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c30673 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29552640->29556735] (req 29552640->29556735) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.823512:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c30673 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29552640->29556735] (req 29552640->29556735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.823519:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c30673 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29552640->29556735] (req 29552640->29556735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760797.823632:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac38b4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2358:x1876699512788480:12345-192.168.203.56@tcp:4:dd.0 Request processed in 32232us (32508us total) trans 4294974534 rc 0/0 00000100:00100000:0.0:1789760797.823642:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7245 00000100:00100000:0.0:1789760797.823651:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 777319988499 time 777355681624 state 0x2 ref 0 00000100:00100000:3.0:1789760797.827287:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1789760797.827501:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512788608 00000100:00100000:2.0:1789760797.827512:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512788608 00000100:00100000:2.0:1789760797.827572:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 777319988499 time 777355681624 state 0x0 ref 1 00000100:00100000:2.0:1789760797.827589:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 777355942040 time 777355681624 state 0x1 ref 1 00000100:00100000:2.0:1789760797.827618:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 777355942040 time 777391918405 state 0x0 ref 1 00000100:00100000:2.0:1789760797.827632:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2745f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512788608:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:2.0:1789760797.827687:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 29556736, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0xe807 00010000:00010000:2.0:1789760797.827718:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c3067a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.827735:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=777s v=(0 0) 00010000:00010000:2.0:1789760797.827740:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760797.827746:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c3067a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29556736->29560831] (req 29556736->29560831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.827844:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c3067a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29556736->29560831] (req 29556736->29560831) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760797.872554:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 29556736 -> 29560832 00010000:00010000:0.0:1789760797.872571:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c3067a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29556736->29560831] (req 29556736->29560831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.872581:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c3067a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29556736->29560831] (req 29556736->29560831) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.872587:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c3067a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29556736->29560831] (req 29556736->29560831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.872591:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febfcb40e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760797.872595:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c3067a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29556736->29560831] (req 29556736->29560831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.872601:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c3067a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29556736->29560831] (req 29556736->29560831) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.872615:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c3067a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29556736->29560831] (req 29556736->29560831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.872623:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004c40742d/0xe6aa143190c3067a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29556736->29560831] (req 29556736->29560831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760797.872678:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974535 is committed 00000100:00100000:0.0:1789760797.872726:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2745f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512788608:12345-192.168.203.56@tcp:4:dd.0 Request processed in 45064us (45418us total) trans 4294974535 rc 0/0 00000100:00100000:0.0:1789760797.872734:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7246 00000100:00100000:0.0:1789760797.872742:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 777355942040 time 777391918405 state 0x2 ref 0 00000100:00100000:3.0:1789760797.875573:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760797.875715:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512788736 00000100:00100000:1.0:1789760797.875725:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512788736 00000100:00100000:1.0:1789760797.875776:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 777355942040 time 777391918405 state 0x0 ref 1 00000100:00100000:1.0:1789760797.875791:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 777392178821 time 777391918405 state 0x1 ref 1 00000100:00100000:1.0:1789760797.875820:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 777392178821 time 777440120567 state 0x0 ref 1 00000100:00100000:1.0:1789760797.875831:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2745500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512788736:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760797.875875:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 29560832, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0xe807 00010000:00010000:1.0:1789760797.875883:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c30681 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.875899:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=777s v=(0 0) 00010000:00010000:1.0:1789760797.875903:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760797.875909:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c30681 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29560832->29564927] (req 29560832->29564927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.875934:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c30681 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29560832->29564927] (req 29560832->29564927) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760797.946859:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974536 is committed 00002000:00010000:0.0:1789760797.947418:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 29560832 -> 29564928 00010000:00010000:0.0:1789760797.947435:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c30681 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29560832->29564927] (req 29560832->29564927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.947450:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c30681 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29560832->29564927] (req 29560832->29564927) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.947459:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c30681 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29560832->29564927] (req 29560832->29564927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.947467:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5000 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:0.0:1789760797.947471:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c30681 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29560832->29564927] (req 29560832->29564927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.947480:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c30681 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29560832->29564927] (req 29560832->29564927) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.947505:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c30681 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29560832->29564927] (req 29560832->29564927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.947513:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000017862c60/0xe6aa143190c30681 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29560832->29564927] (req 29560832->29564927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760797.947620:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2745500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512788736:12345-192.168.203.56@tcp:4:dd.0 Request processed in 71784us (72051us total) trans 4294974536 rc 0/0 00000100:00100000:0.0:1789760797.947630:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7247 00000100:00100000:0.0:1789760797.947641:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 777392178821 time 777440120567 state 0x2 ref 0 00000100:00100000:3.0:1789760797.951449:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760797.952168:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512788864 00000100:00100000:1.0:1789760797.952179:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512788864 00000100:00100000:1.0:1789760797.952236:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 777392178821 time 777440120567 state 0x0 ref 1 00000100:00100000:1.0:1789760797.952252:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 777440380983 time 777440120567 state 0x1 ref 1 00000100:00100000:1.0:1789760797.952329:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 777440380983 time 777516627627 state 0x0 ref 1 00000100:00100000:1.0:1789760797.952342:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac2744e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512788864:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760797.952391:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 29564928, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0xe807 00010000:00010000:1.0:1789760797.952401:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c30688 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.952419:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=777s v=(0 0) 00010000:00010000:1.0:1789760797.952423:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760797.952430:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c30688 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29564928->29569023] (req 29564928->29569023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.952445:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c30688 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29564928->29569023] (req 29564928->29569023) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760797.990025:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974537 is committed 00002000:00010000:1.0:1789760797.990211:0:6709:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 29564928 -> 29569024 00010000:00010000:1.0:1789760797.990226:0:6709:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c30688 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29564928->29569023] (req 29564928->29569023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.990243:0:6709:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c30688 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29564928->29569023] (req 29564928->29569023) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.990252:0:6709:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c30688 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29564928->29569023] (req 29564928->29569023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.990259:0:6709:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8feac3af5200 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760797.990264:0:6709:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c30688 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29564928->29569023] (req 29564928->29569023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.990274:0:6709:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c30688 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29564928->29569023] (req 29564928->29569023) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.990293:0:6709:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c30688 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29564928->29569023] (req 29564928->29569023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.990301:0:6709:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c2379ace/0xe6aa143190c30688 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29564928->29569023] (req 29564928->29569023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760797.990496:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac2744e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512788864:12345-192.168.203.56@tcp:4:dd.0 Request processed in 38152us (39055us total) trans 4294974537 rc 0/0 00000100:00100000:1.0:1789760797.990508:0:6709:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7248 00000100:00100000:1.0:1789760797.990518:0:6709:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 777440380983 time 777516627627 state 0x2 ref 0 00000100:00100000:2.0:1789760797.994356:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760797.994506:0:13062:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512788992 00000100:00100000:0.0:1789760797.994517:0:13062:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512788992 00000100:00100000:0.0:1789760797.994569:0:13062:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 777440380983 time 777516627627 state 0x0 ref 1 00000100:00100000:0.0:1789760797.994584:0:13062:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 777516888043 time 777516627627 state 0x1 ref 1 00000100:00100000:0.0:1789760797.994812:0:13062:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 777516888043 time 777559110017 state 0x0 ref 1 00000100:00100000:0.0:1789760797.994826:0:13062:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4215500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512788992:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760797.994876:0:13062:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 29569024, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0xe807 00010000:00010000:0.0:1789760797.994885:0:13062:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c3068f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.994900:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=777s v=(0 0) 00010000:00010000:0.0:1789760797.994904:0:13062:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760797.994920:0:13062:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c3068f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29569024->29573119] (req 29569024->29573119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.994935:0:13062:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c3068f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29569024->29573119] (req 29569024->29573119) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760798.040212:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974538 is committed 00002000:00010000:1.0:1789760798.040396:0:13062:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 29569024 -> 29573120 00010000:00010000:1.0:1789760798.040413:0:13062:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c3068f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29569024->29573119] (req 29569024->29573119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.040426:0:13062:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c3068f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29569024->29573119] (req 29569024->29573119) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.040436:0:13062:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c3068f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29569024->29573119] (req 29569024->29573119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.040443:0:13062:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f8e00 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:1.0:1789760798.040449:0:13062:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c3068f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29569024->29573119] (req 29569024->29573119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.040457:0:13062:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c3068f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29569024->29573119] (req 29569024->29573119) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.040476:0:13062:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c3068f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29569024->29573119] (req 29569024->29573119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.040484:0:13062:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e8ca9324/0xe6aa143190c3068f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29569024->29573119] (req 29569024->29573119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13062 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760798.040572:0:13062:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4215500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512788992:12345-192.168.203.56@tcp:4:dd.0 Request processed in 45744us (46220us total) trans 4294974538 rc 0/0 00000100:00100000:1.0:1789760798.040592:0:13062:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7249 00000100:00100000:1.0:1789760798.040604:0:13062:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 777516888043 time 777559110017 state 0x2 ref 0 00000100:00100000:2.0:1789760798.043843:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1789760798.043972:0:13063:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512789120 00000100:00100000:0.0:1789760798.043982:0:13063:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512789120 00000100:00100000:0.0:1789760798.044034:0:13063:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 777516888043 time 777559110017 state 0x0 ref 1 00000100:00100000:0.0:1789760798.044049:0:13063:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 777559370433 time 777559110017 state 0x1 ref 1 00000100:00100000:0.0:1789760798.044078:0:13063:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 777559370433 time 777608378328 state 0x0 ref 1 00000100:00100000:0.0:1789760798.044091:0:13063:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4217100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512789120:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:0.0:1789760798.044422:0:13063:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 29573120, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0xe807 00010000:00010000:0.0:1789760798.044431:0:13063:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c30696 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.044447:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=777s v=(0 0) 00010000:00010000:0.0:1789760798.044451:0:13063:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760798.044458:0:13063:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c30696 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29573120->29577215] (req 29573120->29577215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.044472:0:13063:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c30696 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29573120->29577215] (req 29573120->29577215) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.066926:0:5835:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:3.0:1789760798.097021:0:6729:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974539 is committed 00002000:00010000:2.0:1789760798.097276:0:13063:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 29573120 -> 29577216 00010000:00010000:2.0:1789760798.097293:0:13063:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c30696 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29573120->29577215] (req 29573120->29577215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.097384:0:13063:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c30696 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29573120->29577215] (req 29573120->29577215) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.097394:0:13063:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c30696 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29573120->29577215] (req 29573120->29577215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.097402:0:13063:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8febc61f9600 already unused, calling callback (ffffffffc15d8350) 00010000:00010000:2.0:1789760798.097407:0:13063:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c30696 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29573120->29577215] (req 29573120->29577215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.097417:0:13063:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c30696 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29573120->29577215] (req 29573120->29577215) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.097441:0:13063:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c30696 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29573120->29577215] (req 29573120->29577215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.097449:0:13063:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e7c7b91a/0xe6aa143190c30696 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29573120->29577215] (req 29573120->29577215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13063 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760798.097578:0:13063:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8feac4217100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2360:x1876699512789120:12345-192.168.203.56@tcp:4:dd.0 Request processed in 53484us (53739us total) trans 4294974539 rc 0/0 00000100:00100000:2.0:1789760798.097596:0:13063:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.56@tcp, seq: 7250 00000100:00100000:2.0:1789760798.097605:0:13063:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 777559370433 time 777608378328 state 0x2 ref 0 00000100:00100000:2.0:1789760798.102548:0:3302:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1789760798.102623:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699512789248 00000100:00100000:1.0:1789760798.102633:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699512789248 00000100:00100000:1.0:1789760798.102686:0:6709:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 777559370433 time 777608378328 state 0x0 ref 1 00000100:00100000:1.0:1789760798.102733:0:6709:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 777608638744 time 777608378328 state 0x1 ref 1 00000100:00100000:1.0:1789760798.102758:0:6709:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8febfbe81600 key nid={192.168.203.56@tcp}&opcode={ost_write} rule@0xffff8febff902600 gen 0 rate 15 depth 3 token 3 deadline 777608638744 time 777667057985 state 0x0 ref 1 00000100:00100000:1.0:1789760798.102770:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8feac4214e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:50a9c642-d267-4bb6-8271-a3ab7ad8230f+5:2361:x1876699512789248:12345-192.168.203.56@tcp:4:dd.0 00002000:00010000:1.0:1789760798.102819:0:6709:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 29577216, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0xe807 00010000:00010000:1.0:1789760798.102828:0:6709:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c3069d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.102844:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=777s v=(0 0) 00010000:00010000:1.0:1789760798.102849:0:6709:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760798.102856:0:6709:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c3069d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29577216->29581311] (req 29577216->29581311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.102870:0:6709:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aa47624f/0xe6aa143190c3069d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [29577216->29581311] (req 29577216->29581311) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6709 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760798.131939:0:3312:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8feac2747100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3312:1876699540235136:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1789760798.132022:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760798.132117:0:6707:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699540235136 00000100:00100000:0.0:1789760798.132127:0:6707:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699540235136 00000100:00100000:0.0:1789760798.132185:0:6707:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 219 00000100:00100000:0.0:1789760798.132195:0:6707:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8febe7906300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3312:x1876699540235136:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1789760798.132234:0:6707:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3859808256 avail 3857711104 00000100:00100000:0.0:1789760798.132285:0:6707:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8febe7906300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3312:x1876699540235136:12345-0@lo:13:osp-pre-1-0.0 Request processed in 91us (263us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760798.132292:0:6707:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 219 00000100:00100000:0.0:1789760798.132348:0:3312:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8feac2747100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3312:1876699540235136:0@lo:13:osp-pre-1-0.0 Debug log: 56500 lines, 56500 kept, 0 dropped, 0 bad.